 In module 1, we introduced various aspects of data mining, data simulation and prediction and the relations to other branches of science and engineering. This course is a mathematically oriented course on data simulation. So we are going to provide all the mathematical tools and techniques that would be needed to be able to pursue research and education in data simulation areas. With that in mind, in module 2, we have several sub modules. In 2.1, we are going to quickly review the concepts of finite dimensional vector spaces. The notion of finite dimensional vector spaces is fundamental to performing any computational process. In module 2.2, we are going to be talking about all the results that one would need from matrix theory and then we will review concepts from multivariate calculus. Then as a last part in this module, we will also review some of the basic principles from optimization theory. So a strong grounding in basic understanding in finite dimensional vector space, matrix theory, multivariate calculus and optimization tools and techniques are fundamental to any serious pursuit of data simulation. So we will start with a quick review of fundamental principles from finite dimensional vector spaces. I am also going to use this module to set up all our notations and basic concepts. So R is a set of real numbers, they are also called real scalars. C is a set of complex numbers, it is called complex scalars, Rn, R to the power n refers to set of all real vectors of size n, Cfn, set of all complex vectors of size n. We are giving some examples now. This belongs to Rn implies x is a vector of the n components, the components are written column wise, each of the component xi is a real number, 0 is a vector, 0 vector consists of all 0s is called a null vector. When n is 3, here is an example of a vector of 3.2, 1.59, 9.9, 3.2 is the x1, 1.5 is x2, 9.9 is x3. Here is an example of a complex vector. In a complex vector again there are n components, each component is a complex number. So first one is x1 plus iy1, the ith one is xi plus iyi and xn plus yn is the nth element of the complex vector. Here xi and yi are real numbers, i is the unit imaginary number square root of minus 1. Here is an example of a complex vector, 1 plus i, 1 minus i, 1 minus 2i, this is a complex vector of size 3. Even though we talked about complex as well as real spaces of vectors, largely in this course we will deal with real spaces especially R to the n. I am going to quickly review some of the concepts from operations on vectors, x, y, z, b vectors, let a, b, c, b vectors, I am sorry there is an error, a, b, c belongs to R not to R to the n. So there is an error we will correct that, x is a vector, y is a vector, z is a vector, z is a sum of x and y or difference of x and y, zi is the nth component. So zi is either sum of the 2 components or the difference of the 2 components, this is called vector addition, vector subtraction, y is equal to a times x, a is a scalar. So this is called scalar multiplication of a vector, yi is equal to a times xi, scalar multiplication of a vector x by a scalar a, z is equal to ax plus y, in this case ath component zi is equal to a times xi plus yi, this is called saxp, scalar times a vector plus a vector. So these are the basic operations on vectors that we will be dealing with. Now I would like to introduce the notion of what is called a vector space, let V denote a collection of real vectors of size n. For this V to be called a linear space or a vector space or a linear vector space there are several names associated with it, if it satisfies the following 3 condition. The first condition is V1 is a group under addition, what does it mean? If I took any vectors in V, the sum is also in V, it is closed under addition. This operation of vector addition is also associative. So if I am given 3 vectors x plus y plus z in order to find, please remember addition is a binary operation, I can only add 2 numbers at a given time. So if I have given 3 vectors, I have to make 2 additions, you do one at a time, either you add y plus z and to the sum you add x or you add x plus y to the sum you add z, such a property is called associative property of addition. So what does associative property essentially tells you, the order in which you add does not affect the results of the computation. If V contains a 0 vector, what is the property of 0 vector, if you add 0 to any vector it remains the vector does not change, x plus 0 is 0 plus x is equal to x for all x. For every vector x there is a unique y such that x plus y is equal to y plus x is equal to 0 that is called additive inverse of x and y is called minus x. Every collection of vectors that satisfies these properties closed under addition, it is an associative property, it contains a 0 vector and it also has an additive inverse such a set is called a group. So V first be a group, second one there are properties that is to say scalar multiplication a times x, if x belongs to V, ax also belongs to V, so that means any vector if you multiply by a constant it is in the same set. If a and b are two scalars, you can multiply the vector x by b and then by a that is equal to multiplying a and b and then multiplying with x that is again a kind of an associative with respect to scalar multiplication. One times x is x, so one is the real number one, if you multiply any vector by the number one it does not change, it is itself. The third property is called distributivity property, a times x plus y is equal to a times x plus a times y for all x and y. So first one is scalar multiplication distributes itself with respect to addition, the second one is scalar addition distributes itself with respect to vector multiplication by a vector, so a plus b times x is ax plus bx for all x. So any collection of vectors that satisfy these three properties c1, c2, c3 is said to constitute what is called a linear space, a vector space or a linear vector space. For all computations there must underlie always a finite dimensional vector space as a base on which it is all the computations are done. So this is the general definition of what a vector space is, vector space comes in various shapes and forms. The set of all real numbers is a vector space, it satisfies all the properties. The set of all reals satisfy these properties is a vector space, set of all complex vectors satisfy all the things. The set of all n by n real matrices is a vector space, set of all polynomials of degree n is a vector space. If you have sequence, infinite sequence such that the sum of these squares is finite, it is called square summable sequence, the set of all square summable infinite sequences they also form a vector space. The set of all continuous functions over interval a, b is also a vector space. So you can see vector space of functions, vector space of sequences, vector space of polynomials, vector space of matrices, vector space of complex numbers, real numbers and real vectors. So vector spaces are abundant. Every one of these vector spaces constitute the basis for computational processes and data simulation is largely a computational problem because I need to be able to estimate, fit the model is a solving an inverse problem, being a computational problem I must always be concerned with what is the vector space in which I am performing all these computations. I am now going to quickly review operations and vectors. Some of you who have taken a course in linear algebra may already know this. I am assuming that all the people who are going to be reading this may not have the same background and so to bring uniformity in the reader I am going to quickly review many of these concepts. So let x, y, z be 3 vectors, a, b, c be 3 real numbers. I am going to introduce this bracket notation opening bracket dot, dot closing bracket that is going to be a binary operation on vectors. So that binary operation in here is called an inner product. So parenthesis inner product y, x and y defines an inner product which is defined at x transpose y which is defined as sum of xi, yi. xi, yi is also equal to yi, xi because multiplication of real numbers is commutative. So that is equal to y transpose x is equal to y, x. So this means the inner product I am not only defining the inner product but I am also showing the inner product as an intrinsic property it is symmetric. So the symmetry property. So the properties of inner product x dot y is greater than 0 if x is not equal to 0 it is 0 only if x is equal to 0. So this y must be x, one second I will tell that now. This y must be x then the definition is correct. So inner product of x of itself is greater than 0 when x is not 0, inner product of x with the x is 0 only when x is 0 that is called the positive definite property. We have already seen the symmetric property. Inner product is also additive, inner product of x plus y with z is inner product of x with z, inner product of y with z the sum of the two. The inner product is said to be homogeneous what does it mean? If I multiply one of the components by a constant a, so inner product of ax, y is a times inner product of x and y it is also same as x times inner product of ay that is called the homogeneity property. If the inner product of x and z and y and z are equal for all z then x and y must be equal that is again another fundamental property of inner product we will use all these properties in developing a joint technique or joint methods when we do 40 bar methods. When x and y belong to Cn the inner product is defined by xi yi bar, yi bar is the complex conjugate of y. So the inner product definition has to be appropriately modified when you go from real domain to the complex domain. Since we are going to be dealing only with the real domain these 5 properties of inner product are sufficient for our purposes. Now I am going to define other operations and vectors x is a vector, y is a vector by vector I always mean a column vector. So y transpose is a row vector. So x, y transpose is the product between a column a column vector and a row vector the product of column vector is called an outer product of 2 vectors. The result is a matrix x1, y1, x1, y2, xn, yn and so on you can see the elements of the matrix coming in here. So the outer product can be written in many ways. The first column is a multiple of the vector x by y1, the second column is a multiple of x by y2, the last column is a multiple of x by yn likewise I can also consider as a multiple of rows. The first row is the multiple of the row y with x1, the second row is the multiple of the row y with x2, the last row is the multiple of the row y with xn. So I can think of it as a matrix or multiples of column x or multiples of row y. All these are properties of the outer product of matrices, outer product of matrices is a fundamental operation. The next one is called the norm of x and the notion of a distance. The norm of x is denoted by x within that sign 2 vertical to the left 2 vertical to the right. A vector is one object the norm of a vector is another object. The norm is a scalar associated with every vector there is a norm. Norm is a measure of the size of the vector, the size of the vector is denoted as a scalar. The norm of the vectors arises in many ways. One is called the Euclidean norm, another is called the Manhattan norm, another is called Chebyshev norm, another called Minkowski's norm, another is called the energy norm. The Euclidean norm is a standard one that comes from the Pythagorean theorem. The norm of x is equal to square root of the sum of the squares of x that can also be expressed as square root of the inner product of x with x. The Manhattan norm or one norm is essentially sum of the absolute values of x. I would like to be able to bring the distinction between Manhattan norm and the Euclidean norm. So if I have a 2 dimensional plane if I have a point here if this is x1 this is x2 the Euclidean norm refers to this distance and what is the value of this distance? This distance is equal to x1 square plus x2 square to the power half that comes from the length of the hypotenuse is this is x1 this is x2 is the right angle triangle with 2 sides x1 and x2 that is the length of the hypotenuse. So that is the 2 norm, the one norm on the other hand is if I want to go from 0 to this point I have to go by x1 then I have to go by x2 it is sum of the distances from 0 to there. Let us talk about this now suppose you have to go so let us assume this is the point O this is the point P if I want to go by a taxi from point O to point B I will first go along the street to the east and then I will go along the street to the north. So the total distance traveled by a taxi cab is x1 plus x2 but if I had a helicopter I could fly directly from 0 to P and that is the Euclidean norm. So that is the 2 ways of differentiating the 2 norms Chebyshev norm is called the infinitely norm and that is Chebyshev is a famous Russian mathematician and he defined the norm to be the maximum of the absolute values of I that is another useful definition of a norm. Minkowski another mathematician from Russia he defined what is called a P norm the norm of P is given by what is that you need to do you take the absolute value of each component raise it to the power P it is the P so pth root of the sum of the pth powers of the absolute values of x I hope that is clear from the expression we will often talk about another useful in meteorology when we talk about error growth and other things it is called energy norm energy norm of a vector x with respect to matrix A is defined to be x transpose A x to the power of half A in this case is a symmetric positive definite matrix. So norm refers to the size size can be measured in many ways there are at least 5 different ways I have illustrated one can measure the size of a vector with once I have a size I have the notion of a distance so if x and y are 2 points the distance between x and y is simply the norm of the difference of the 2 vectors x is the vector y is the vector difference of a vector is a vector I can turn to the norm of the vector so the distance between 2 vectors is simply the norm of the vector associated with the difference what are the general properties of norm you can define norm any way you want no one is going to be able to come and dictate that this should be the only way to be able to design norm. So if you want to define your own norm I am going to tell you what are the basic properties a norm must possess so given a vector x n of x is a norm if it satisfies the following three condition n of x must be positive definite n of x must be go homogeneous in other words the norm of a scalar multiple of A x is simply A times the norm of x that is called a homogeneous the third property the norm is that the sum of the norm of the sum of the 2 vectors is less than or equal to norm of x plus norm of y that is called the triangle inequality that is called the triangle inequality. So the norm should be positive definite a norm must be homogeneous a norm must satisfy the triangle inequality I would like to point out that every norm that we define the phi norms we define all of them satisfy these properties in addition to these five you can define your own norm you can any norm that you want to use must satisfy these three conditions. Now a special note equally norm is very special because equally norm is the only norm that can be derived from inner product of the phi norms only equally norm is associated with the inner product and nothing else then I am I have a homework here verify that the norm square of the sum plus norm square of the difference is 2 times the square of the norm of x plus square of the norm of y. So this must be norm of y once again this must be norm of y so that rule is a very basic rule that is called the parallelogram law any the norm based on the Euclidean definition always satisfies this parallelogram law. Then the notion of what is called the unit sphere comes into play the unit sphere in two norm is given here please remember two norm is called the Euclidean norm the unit sphere is in one norm takes the shape one norm is the Manhattan norm this is the conventional geometric norm this is the infinity norm the unit sphere the infinity. So what is the unit sphere unit sphere is the locus of points which are at unit distance from the origin so if you take a circle of radius one centered at the origin if the circle is defined as a locus of all points at constant distance of one from the origin. So if you pick the norm to be the equally norm that is the circle the circle becomes this trapezoid when you change the norm the trapezoid becomes a square if you change the norm the unit circle becomes an ellipsoid if I change the norm. So when you pick a matrix A to be 5001 that is the symmetric positive definite matrix if you consider the square of this norm you get the equation to an ellipse which is given by here x1 square by a square plus x2 square by b square is equal to 1 is an equation to an ellipse so you can readily see the equation to the ellipse is depicted here. So what is why am I doing this I want you to understand that the geometrical figures naturally morphs the shapes changes if you change the definition of a norm again I want to insist here mathematics is a man made science you have total freedom to do whatever you want the only condition is you must be consistent. So for a norm to be consistent you have to satisfy those three rules so consistent with those three rules we have seen several different norms and this is one way to geometrically explain the intrinsic differences between the properties of these norms then you have the notion of what is called the unit vector the unit vector in the direction x is simply x divided by the norm of x we all know that very well then there are a couple of fundamental inequalities what is called Cauchy Schwarz inequality what does it say if I have an inner product between x and y the value of the inner product by definition is x transpose y and that is equal to the norm of x norm of y times the cosine of the angle between the two that is the cosine of theta cosine of theta is always less than 1 less than or equal to 1 therefore this product is always less than or equal to product of the norm of x and norm of y so this inequality namely inner product of x and y is less than or equal to the product of the norms of x and y that inequality is called Cauchy Schwarz inequality it is one of the most fundamental inequalities again I would like you to work as an exercise verify that this Cauchy Schwarz inequality becomes an equality only when the vectors x and y are parallel to each other is a very simple exercise and I would like you to prove it yourself to be able to understand the power of the Cauchy Schwarz inequality an extension of the Cauchy Schwarz inequality is called Minkowski inequality if p and q are two integers with the property 1 over p plus 1 over q is 1 then Cauchy Schwarz inequality can be extended to the inner product of x and y is equal to x transpose y is less than or equal to the p norm of x and a q norm of y when p is equal to q is equal to half 1 over half plus 1 over half is 1 p is 2 q is 2 I get the two norm so the Minkowski inequality reduces to Cauchy Schwarz inequality when I pick the two norm so you can see the generalization between two norm p norm q norm Cauchy Schwarz Minkowski all these related properties of vectors. So now that we have known that there is one norm there is two norm there is infinity norm all these norms are related I am not going to prove them but you can readily see what does it mean if I have a vector x if it is if the norm of a particular vector x is finite in one norm it has to finite in every norm that is what is essentially says the length of the vector in a two norm is less than or equal to the length of the vector in one norm which is less than or equal to square root of n times length vector in two norm likewise all the other inequalities I do not want to repeat it you can read for yourself this essentially tells you that all these norms are intrinsically interrelated so what does this mean this means that you as an analyst has total freedom you do not have to confine your analysis either to the one norm or two norm or the infinity norm or the energy now you can do the analysis by picking any norm that is convenient to you if you can prove one result in one norm you can extend it to any other norm using these inequalities so that is the fundamental that is the fundamental aspect of this relation between various norms. Now I am going to introduce the other concept which is called a functional let V be a vector space any function that means that maps V to R R is a set of real number F is a function that takes a vector x as input so let me give you little picture here so I have a box which is F I give an x x belongs to R of n it spits out a value f of x and f of x is a real number so what does it mean it takes vectors and converts them to real numbers any function that converts a vector into real number that is called a functional function is different from functional it is a very technical term so I would like you to be aware of the intrinsic differences between a functional is a function but not all functions are functionals so functionals are special cases of functions f is called a linear function so once I have a functional a functional can be a linear functional or a non-linear functional a functional is said to be a linear functional the emphasis linear functional f of x1 plus x2 is f of x1 plus f of x2 that means it satisfies the additive property it also satisfies what is called the homogeneity property f of ax is equal to a times f of x so any functional that satisfies these two properties is called a linear functional now I am going to give you examples of linear non-linear functional a norm is a non-linear functional given a vector x a norm is a number so norm converts a vector into numbers is a functional is a non-linear functional for any fixed vector a f of a mapping R into R that means f of a of x is a times x for a fix that is a linear functional another example of a non-linear functional given a matrix a given a matrix a I can talk now about one half of x transpose ax that is an example of a non-linear functional so functions linear functional non-linear functional functionals defined over vector space so vector space is the basis so you can think of a functional to be as follows a functional is here is a vector space v here is a real line R a functional takes a vector and maps it into a real number so that is how you can look at a functional mapping a vector to a real number now I am going to quickly talk about the notion of orthogonality and conjugacy of vectors why do I need conjugacy later when we are going to do optimization we are going to be talking about conjugate gradient methods I would like to be able to introduce the notion of conjugacy pretty early enough so let x and y be two vectors we denote a vector this this one I am sorry yeah good this symbol has to be perpendicular like this I think my computer did not have that is so we say x perpendicular y is equal to 0 to imply the inner product implies and implied by the inner product of x and y is 0 if the inner product to x and y is 0 I say the vector is orthogonal orthogonal vectors are denoted by this symbolism x perpendicular sign and y so two vectors are set to be orthogonal if the angle between them is 90 so if this is x if this is y x this is y angle is 90 degrees so we say x and y orthogonal now in extension of the notion of orthogonality is called a conjugacy two vectors are set to be a conjugate if x transpose a y is 0 now I can extend the notion of a conjugacy to a set of vectors let x be a set of k vectors each of them in Rn yes said to be mutually orthogonal if I pick any two vectors x i x j it is 0 if i is equal to j it is not 0 so we call it mutually orthogonal that means if I took any pair of vectors they are orthogonal so what is an example of any pair of vectors there is orthogonal you already know this example 1 0 0 0 1 0 and 0 0 1 we generally call this vector e1 we call this vector e2 we call this vector e3 you already know e1 is perpendicular to e2 e2 is perpendicular to e3 and e3 is also perpendicular to e1 so e1 e2 e3 are unit vectors they are mutually perpendicular to each other that is the notion of mutually orthogonality then a is set to be orthonormal if if I picked two distinct vectors the product is 0 the inner product is 0 if I pick the same vector and compute the inner product with itself then the value is 1 in which case it is called orthonormal orthonormal means the vectors are normalized they are also orthogonal so what do I mean by saying the norm of x i come on now inner product of x i is equal to 1 that simply is equal to the square of the norm of x is 1 say that is what that what this means that is that is what this means that is what that means that means vectors have unit length every two vectors are orthogonal now if I look at my vector e1 a this is a unit length this is a unit length this is a unit length so I have examples of three unit vectors which are mutually orthogonal so this these three vectors they are not they are not only mutually orthogonal but also orthonormal I hope you see the difference between normality and and and simple orthogonality the same set of vectors are set to be a conjugate if x i a trans x i transpose a x j equal to 0 if i is not equal to 0 the energy norm of x with respect to the matrix a square of it if i is equal to j so this is an extension of the notion of mutual orthogonality mutual orthogonality so these three concepts orthogonal orthonormal a conjugacy of a collection of vectors is one of the fundamental properties of vectors that we would be very much interested in in our analysis now I am going to introduce a very simple notion what is called what is called the linear combinations of vectors we will also have lot of occasions to talk about this concept let x be a set of k vectors each of the vectors are going to be in orthon so each of the vectors so I have k number of vectors each of them in orthon so I want you to distinguish two things the size of the vector is n but k of them x i the ith vector has n components the n components the ith vector is i 1 i 2 i n the first index refers to the index i of the vector x the second indices refer to the components of the vector let a 1 a 2 a k or the be the real scalars let us define y to be the sum a scalar times a vector plus a scalar times a vector plus scalar times a vector y is simply sum of the multiples of each of the vectors so y is called a linear combination of the vectors x in I have so this is this is called the linear combination that is very fundamental linear combination y is a 1 x 1 plus a 2 x 2 plus a k x k what is the standard example of a linear combination if I have a set of vectors x 1 x 2 x k if I compute the average x bar 1 over k times summation i equal to 1 to k x i a what is that that is called the centroid in geometry we consider centre of gravity the centre of gravity is the centroid centroid is simply a linear combination of vectors so I this is an example of the notion of the linear combination that often occurs in statistics in many computations so the notion of linear combination is fundamental once I have the notion of linear combination I am now going to talk about the notion of what is called linear independence and linear dependence again this is another fundamental property from the vector spaces that one needs to be very thorough way let x be k vectors the set of vectors the set of vectors in s are linearly dependent if there exists a linear combination y defined by a 1 x 1 a 2 x 2 a 3 x 3 a k x k whose sum is 0 but the condition is that not all a's are 0's when not all a's are 0's means even when I can annihilate them by I can annihilate them by picking by picking some of them to be not 0 as an example if I have a vector 1 0 0 if I have a vector 3 0 0 you can readily see this is the vector let us say x 1 this is the vector x 2 x 1 I can I can say minus 3 times x 1 plus x 2 is equal to 0 do you see that place so these 2 vectors are not linearly independent they are linearly dependent so the notion of a linear dependence is very clear so when do I say something is linearly independent the opposite of dependence is independence your set of vectors is set to be linearly independent if it is not linearly dependent so you define what dependence is and then say independence is something that is not dependent so the notion of linear dependence is fundamental is an absolute absolutely a very replace a very basic role when we deal with rank of matrices when we talk about solutions of linear systems and so on. The next concept is the notion of what is called span of a set of vectors so let us assume I am given a set x of k vectors in Rn n is the dimension of the space k is the number of vectors I have picked I am going to define a concept called span of a vector what is that it is a set of all vectors y it is a set of all vectors y such that this y is the linear combination of the set of all vectors in S so x is all in this ai's are constants so y is the linear combination of vectors in S and each of the ai's are real numbers x is R in Rn so think of it now I have been given a fixed set of I have been given a fixed set of numbers I have been given a fixed set of vectors which are x1 to x2 so these x's are fixed I have a choice in a's each of the a's are real so for each coefficient there are infinitely many choices there are k such coefficients so there are k way infinity of combinations that one part that is possible the set of all linear combination you put them all together we call the span effect so that is called the set of all linear combinations that affects I will give you a quick example now let e1 be the vector of 1 0 let e2 be the vector of 0 1 span of e1 e2 let us consider the span of e1 e2 so this is the x axis x1 axis this is the y axis so e1 goes like this e1 goes like this e2 goes like this every vector in this space x can be replaced as x1 times e1 plus x2 times e2 we all know that right any vector x here is equal to x1 x2 so what does this mean x1 times e1 plus x2 times e2 so any vector x is the linear combination of e1 and e2 therefore the two dimensional space r2 is the span of e1 and e2 I hope that is very clear to you now so the two unit vectors span the whole space so that is the power of the notion of span so clearly a span is a vector space and it is a subset of rn we say the span of s is a subspace generated by the set of vectors the axis so in summary what is the concept here using the concept of linear combination and by picking a set of k vectors I am able to define a subspace generated by a subset of vectors a subset of k vectors a subset of k vectors in here so that is the notion of a span of a set of vectors the next concept is called the notion of a basis and dimension you can really see I am not proving any of these concepts I am trying to introduce all these concepts because you must be aware of these concepts so you should have a good access to a good book on linear algebra to be able to further explore these concepts but I want to bring all these concepts to the forefront to emphasize these concepts play an intrinsic role in the development of algorithms for data simulation data simulation as a discipline belongs to computational science it is a branch of applied mathematics it is a very deep roots in many of the different sub disciplines in mathematics I am trying to expose such basis one would need to be able to do data simulation thoroughly. So I am now going to do the next concept called basis and dimension let us consider a vector space v let b be a subset of the vector space so b is a subset of the vector space I am not going to be talking about a particular property of the vector sub subset if every vector so what is the basic idea here this is the vector space v b is a small subset of it if every vector x in v can be obtained as a linear combination of those in b that means every vector in v can be expressed as a linear combination vectors in b then b plays a very basic role b is very important because everybody in v depends on b such a subset is called a generator for v the notion of a generator for example the two unit vectors d1 and e2 generate the whole two dimensional space because every vector in a two dimensional space is a linear combination of the two vectors d1 and e2 if the set of vectors in b are linearly independent then b is said to be the basis we already know the notion of linear independence so b is the basis or span of as e i is the unit vector with 1 as 1 in the ith element and 0 elsewhere so that is called the ith unit element bn the set of all unit vectors i 1 to n this must be i i is equal to 1 to n this is the capital i it must be little i the set of all unit vectors is the basis for bfn therefore you can really see the n dimensional space is essentially created by a linear combinations of vectors in the basis the number of elements in b is called the dimension or the span of b so the dimension relates to the number of generators so what does it mean what is the minimal number of element that you need to be able to create the whole space if i had n unit vectors i can define the whole n dimensional space if i have two unit vectors i can define the whole two dimensional space so the notion of base notion of dimension and you can readily see all these things are intimately related to the notion of linear combination linear dependence linear independence and these are fundamental concepts relating to vector spaces now i am going to conclude with a set of problems which i would like you to work and extend your understanding to be able to work some of these problems you may need to consult some of the other books that describe all these methodology lot more in clear detail but i would like to i have hit on major concepts one must be aware of to be able to pursue things to follow so verify the parallelogram law verify the triangle inequality for the two norm one norm and infinity norm that very good mathematical exercises prove that the inner product is equal to the product of the norms if x and y are parallel vectors this essentially comes from the Cauchy-Schwarz inequality using MATLAB plot the contours of fx when x transpose ax in other words fx is x transpose ax so this is the quadratic function i would like you to plot the contours of this using a MATLAB MATLAB is one such example you do not have to use MATLAB you can use Mathematica or any other software system that you are comfortable with but MATLAB has very powerful graphics that makes the job of plotting almost trivial verify that if x1 x2 is 3 or 3 linear independent vectors that x1 plus x2 x2 plus x3 x3 plus x1 are also linearly independent let x be a vector 1 2 3 that means i am now giving a very specific vector with components 1 2 and 3 i would like you to verify the relations between the one norm two norm infinity norm given in slide 12 of module 2.1 the module is essentially 2.1 in this in this particular module i do not have to even say the in this particular module but the module number is 2.1 that is that is that is what i would like to able to emphasize in here so this is 2.1 with that i think we come to the end of the coverage i told you you have to go into other books for for the reading i am giving you 3 references one is a book by gullabhan van laan that is one of my favorite i have a copy of that i have the second one is matrix analysis and applied linear algebra the third one is honan johnson matrix analysis the third one is little bit more advanced second one is quite elementary third as first one is rather intermediary i have all the 3 copies of these books these are extremely useful anybody who wants to do fundamental work in data simulation must have at least one of these 3 my preference is 2 the book by mayor published by sam is an excellent book with primary emphasis on not only on matrix theory but also on computational aspects of matrix theory with this we conclude our overview of the basic principles of vector spaces so what are the basic things we covered vectors norms distances concept of linear dependence concept of linear independence orthogonality conjugacy basis dimension these are the nuts and bolts of linear algebra that you would need to master to be able to proceed further thank you.