 So, what I would like to do is to continue the discussion of the first course on linear algebra to discuss the notion of inner product space, okay. So let me start with the notion of inner product space. See the notion of vector space generalizes the concept of addition of vectors, scalar multiplication that one is familiar with after doing a course on two dimensional or three dimensional geometry. The notion of inner product allows us to generalize the notion of dot product of vectors, say which we have studied for two dimensional, three dimensional vectors. This also allows us to talk about angle between vectors when vectors are orthogonal in a general vector space. The notion of inner product also allows us to talk about the notion of a norm of a vector. Norm of a vector is informally the distance function. So norm, once you have an inner product you define a norm and through the norm you can define the notion of distance between vectors, etc, okay. So first we will discuss the notion of inner products. So let me start with the definition of an inner product. It is convenient to start with complex vector spaces to introduce the notion of an inner product. So I have a complex vector space. So the first notion is inner product and inner product spaces, okay. So I have V as C n, okay the space of vectors with n coordinates each coordinate is a complex number C n over C that is a usual notation V is C n an inner product on V, an inner product on a vector space. An inner product on V is a function, is a function which we will denote using these two brackets dot, comma dot. This is a function from the cross product V cross V to C. So in general it is a complex number. The inner product is a function on the cross product. The inner product on V is a function on the cross product. It takes complex values, satisfies the following. It is a function which satisfies the following conditions. First condition, okay I will call it 1, 2, 3, etc. First condition is inner product of X, X, this must be non-negative for all X and V. The inner product of X with itself must be a non-negative real number. That is the first condition. And as part of the first condition we also have inner product of X, X equal to 0 if and only if X is equal to 0. This is part of the second condition, okay. V cross V is something like a set of all X, Y, X and Y coming from V. This function in particular must satisfy this condition. That is the first condition. Second condition is if you look at inner product of X plus Y, Z, this must be inner product of X, Z plus inner product of Y, Z. It is like additivity with respect to the first argument. This must be true for all X, Y, Z in V. Condition 3, you look at the inner product of lambda X, Y. Look at the image of lambda X, Y under this function. This is, this must be lambda of the image of X, Y. This is true for all lambda in C and for all X, Y in V. That is my third condition. Condition 4 is inner product Y, X. The image of Y, X is the complex conjugate of the image of X, Y. This must be true for all X, Y element of V. These are the four conditions that an inner product on a vector space must satisfy. And for convenience we are taking this vector space to be the complex vector space. Complex vector space C n, okay. There are some names associated with these conditions. The first one is called positive definiteness. First one is positive definiteness. Positive definiteness of the inner product. It is greater than or equal to 0 and it is equal to 0 if and only if X is 0. So first property is positive definiteness. Second and third is linearity with respect to the first variable. 2 and 3 together is linearity of the inner product with respect to the first, we can say argument or the first variable. Linearity with respect to the first argument. We can understand why it is linearity. You have these two notions. Additivity and addition of vectors, how does it behave with respect to that? With respect to the first argument, similarly scalar multiplication, okay. Fourth one is called conjugate symmetry. Property 4 is conjugate symmetry. Property 4 is conjugate symmetry. In general an inner product is not linear with respect to the second argument. I will give an example, okay. But before that let us look at examples of inner products. But even before that let me tell you that a vector space, this notation we will adopt, a vector space V together with an inner product is called an inner product space and the notation for an inner product space will be usually V, this inner product. When the inner product is clear from the context we will not use this. We will simply say V is an inner product space when the inner product is clear, okay. So let us look at examples of inner product spaces. See what I have done is to give an example to start with V SCN, okay. I have started with V SCN. This definition can be given over a general vector space V, okay. But let us look at the first few examples. This will be over C N, R N, etc. So first example, let us take V to be R N. See if you go back to this example, you will see that no property of C N has been used. What I want is a complex vector space, okay. What I want is really a complex vector space. So I can replace this as well by a complex vector space, okay. So I will simply say R V equal to C N or any complex vector space. The property of C N has not been used. What I need is just a complex vector space or any complex vector space which can also include real vector space. So I look at V as R N. So this is a real vector space R N over R. That is what this means. So what is a usual inner product? Usual inner product for 2 or 3 dimensional vectors we will extend it to R N, okay. So let us take X Y in R N and the notation as before will be X equal to X 1, X 2, etc X N and only for convenience we are writing it as a row vector. So X is this, Y is Y 1, Y 2, etc Y N. If these are the coordinates of the vectors X and Y then the inner product on R N, this is called the usual inner product. We will define this to be summation I equals 1 to N X I, Y I, summation I equals 1 to N X I, Y I coordinate Y's product or component Y's product. This is the definition. Then you can see that this satisfies the 4 conditions listed here. First condition is inner product of X with itself will be summation I equal to 1 to N X I square. Each X I is a real number. So this sum is non-negative and each term is non-negative. So the sum can be 0 if and only if each term is 0. Each term 0 means X I is 0 for all I. So first condition satisfied. Second condition satisfied. We have verified. Look at X plus Y Z. The inner product is linear with this inner product is linear with respect to the first argument. In fact with respect to the second argument also, okay but that is not what we want. 2 and 3 are satisfied trivially. You can check 2 and 3. 4 is satisfied without the bar because it is a real space, okay. So this is one example. It can be a real space and I have a real inner product space. So R N with the usual inner product is an inner product space. So let us look at C N and introduce an inner product. Second example, V is C N as before X and Y will be represented by the coordinates X 1, etc. X N, Y 1, etc Y N. This time the inner product is defined as summation I equals 1 to N X I Y I bar X I Y I bar. This is the definition for a complex for C N. Now you will see why you need the bar here. By the way each coordinate here is a complex number. Y I bar means it is a complex conjugate, okay. Why do you need this? See if you just take X I Y I it will not be an inner product, please you take this as an exercise, little exercise. It will not be an inner product. For instance what is inner product X comma X? Inner product X comma X is summation I equals 1 to N X I X I bar, okay. That is summation I equals 1 to N mod X I square. Now this is the reason why you need the complex conjugate here. Now this is a sum of non-negative real numbers. So this must be greater than or equal to 0 and if it is equal to 0 then each since each term is non-negative each term must be 0 if it is equal to 0. If the modulus of a complex number is 0 then the complex number itself must be 0. So each X I will be 0 and so X is 0, okay. That is the reason why you need the complex conjugate here. I think I will leave it for you to verify that it satisfies the other 3 conditions, okay. So this is the usual inner product on C n. This is the usual inner product on R n, okay. Okay maybe I will just write it down. This inner product the above inner product inner product is referred to as the usual inner product or the standard inner product on C n, usual inner product or the standard inner product on C n. We can do a little more general for the third example. We can do a little more general than what we have done for C n. We can take n positive real numbers. Let T 1, okay I will call it W 1, W 2, etc W n be positive real numbers. I have n positive real numbers for X Y in C n. I can define an inner product with respect to these numbers, with respect to these numbers. This is another inner product. So I will call it maybe inner product W, inner product with respect to W. W denotes the set of numbers W n, etc W n each of them being positive. So this inner product is just an extension of the above. It is summation i equals 1 to n. Use these positive real numbers. These are called weights. Use these positive real numbers and then define as before X i Y i bar. These are called weights. The rest is as before. You need them to be positive in order for the positive definiteness of the inner product to be satisfied. In order for condition 1 to be satisfied, you need them to be positive not non-negative. Even if one of them is 0, it will not be an inner product. So you can verify that this is an inner product. This reduces to example 2 when each of the W i's is 1, okay. So this is more general than this. Sometimes we may use this. Let us look at other examples. These are for R n C n. Let us look at C n cross n, the space of matrices. My fourth example will be the space of matrices. V is C n cross n. So C n cross n is the complex vector space of matrices of order n by n. Complex vector space means that the entries of the matrix are complex numbers. So what is the inner product? What is one possible inner product on V? So let us take AB in V. Let us recall the notion of the trace. The trace of the matrix A. Let us say it is denoted by T R A. Trace of A is defined as, see it is a square matrix is defined by, we have seen this definition earlier. Trace of A is summation i equals 1 to n A ii where the notation is A is A ij. If the entries of A are denoted by A ij then I look at the sum A 11 plus A 22 etcetera plus A n n that is the trace of the matrix. This is a linear function for A star. This is A bar transpose. A star will denote A bar transpose. By A bar I mean the complex conjugates of the entries of A and then I take the transpose that will be denoted by A star, okay. What is the inner product? Let us define inner product of two vectors in V. Two complex matrices to be trace of AB star. So can you derive the formula for the trace of AB star in terms of the entries of A and B? If you are given the entries of A and B what is the formula for the trace? Can you make a guess? First, that is what you expect to happen. My question is what is the guess for trace of AB star? The answer is summation i equals 1 to n A ii B ii bar. Let us see whether that is correct. Given two matrices let us first compute the arbitrary element of the product. Let us say I use C and D. C is C ij, D is D ij. Suppose I want to compute the r as the component of the product C D. See I will assume C and D are in V, okay. So matrix multiplication is possible both sides. C D, D C both multiplications are valid. Look at C D. For the product C D let me calculate say r s component. What is the r s component of C D? Summation, C r k D k s, k equals 1 to n, okay. This is the r s component, okay. What is the r s component of AB star now? Summation k equals 1 to n AB star. For A I have C that is A r k B star. See it is the k as the component of B star. S k component of B. B, S k of course it will go with the bar, okay. Now I want to put r equal to s, okay and then I want to sum it over s, isn't it? I must sum it over s. I have put r equal to s. I have not made the appropriate change. I will call this s. On one side I have r and the other side I have s, okay. So this is the formula for the s s component of AB star. This is the diagonal element of AB star. I must sum all the diagonal elements. So what is the formula then? So I will write that here. Trace of AB star is double summation. I have used r and s but I will come back to the usual i and j. i equals 1 to n, j equals 1 to k, sorry, j equals 1 to n, A ij B ij bar, okay. So this is the formula. Please check this after going home. So this is the formula for the trace of AB star and so this is the final formula in terms of the entries of A. So please verify that this is again an inner product on this complex vector space V, okay. Now all these are finite dimensional spaces. Let us look at one example from infinite dimensional spaces, okay. Let us look at one last example coming from infinite dimensional space. One last example of an inner product space. Let us consider V as C 0 1, the space of complex valued continuous functions on the closed interval 0 1, the vector space of all complex valued continuous functions on 0 1, okay. We know that it is a vector space and we also know that it is an infinite dimensional vector space. It does not have a finite basis. For two elements in V, let us define the inner product by, so what is the most natural just integral, these are continuous functions so integrals exist. So simply look at 0 to 1, f of t, since they are complex functions you have to imitate what you did for the discrete case gt bar dt for f and g from V. Then remember the first one is as it is second one goes with a bar that is why this is conjugate symmetric that is it is not linear with respect to the second variable but conjugate line with respect to the second variable, linear with respect to the first variable, linear with respect to the first argument. But it is the first property which is important to verify, first property is inner product f comma f, see that is integral 0 to 1 mod f square dt. It is a non-negative function and we know from Lebesgue integration if you want that it is a non-negative function and it is 0 if and only if the integrand is 0. It is a continuous function, if the integral is continuous non-negative function mod f t square dt continuous non-negative function if the integral is 0 then the integrand must be 0, okay. So it is you need to use techniques from integral calculus in some cases to verify that certain inner products are really inner products. So please verify the other condition this is an inner product space V together with this is an inner product space, okay. So that is a quick review of the definition of an inner product and an inner product space. I want to move on to the notion of a non-linear space, okay but I will be interested in first norms which are induced by inner products, okay so let us look at the following. I have an inner product space let V be an inner product space the norm let us take an arbitrary element in V the norm of X denoted by this is defined by norm X is the positive square root of the inner product of X with itself the norm of a vector in an inner product space. So we are looking at the norm the so called norm induced by an inner product, okay is it well defined we are taking the square root of a number if you look at the inner product space the first condition you know that this is a non-negative number so and I am also taking the positive square root of this, okay. So this is a non-negative real number to begin with the norm of a vector is a non-negative real number, okay. So let me list the properties that this satisfies first property I will just write down. So norm of X is greater than or equal to 0 for all X and V suppose norm X is 0, okay that happens if and only if norm X square is 0 if and only if inner product X with X is 0 which happens we know if and only if X is 0, okay so this norm has this property something similar to the property of an inner product positive definiteness. What are the other properties I will list them in the form of a result let V be an inner product space then the following properties hold. Property 1 is look at norm of alpha times X norm of alpha times X is mod alpha norm X mod alpha norm X this is true for all alpha in C and for X in V condition B look at the inner product of modulus of the inner product of X with Y this is less than or equal to norm X into norm Y this is true for all X Y in V this is called the Cauchy-Schwarz inequality which you may have encountered in some other context this is called Cauchy-Schwarz C H inequality, Cauchy-Schwarz inequality connecting the inner product and the norm it is an inequality. Property 3 is norm of X plus Y is less than or equal to norm X plus norm Y for all X Y in V this is property 3 this is called triangle inequality triangle inequality, okay let us prove this quickly see this is a defining equation for the norm the defining equation for the norm is this and in probably all the cases we will take norm X square as inner product of X with X and prove all these. So let us start with the first one consider norm alpha X the whole square by definition this is inner product of alpha X with itself for the first one alpha comes out for the second one it comes with a conjugate alpha alpha bar mod alpha square norm X square on the left I have square of norm alpha X I can take the positive square root because this is norm is non-negative modulus is non-negative again norm is non-negative I can take the square root to get norm alpha X to be mod alpha norm X that is the first property okay then Cauchy Schwarz inequality consider the following we have for all lambda in C for all lambda in C from the inner product we have 0 less than or equal to X minus lambda Y comma X minus lambda Y first property of the inner product apply to the vector X minus lambda Y for all X Y in B expand this using the norm. So this is equal to inner product X with X inner product X with minus lambda Y that will go with the conjugate minus lambda bar X comma Y this term minus lambda Y X the last one will go with the lambda lambda bar that is mod lambda square mod lambda square norm Y square okay I can rewrite the first one and call it norm X square that is what it is so this is non-negative let me call alpha as the number lambda bar X comma Y alpha is the number complex number lambda bar X comma Y then the second term is alpha bar I mean the second term is alpha the third term is alpha bar okay. So this right hand side let me go to the other board I think I will remove this now okay so what I have is 0 less than or equal to norm X square plus so let me take the term corresponding to Y mod lambda square norm Y square that is the last term and club the other two it is plus 2 times alpha plus alpha bar okay where alpha is lambda bar X Y alpha is a minus 2 times is that okay both go with the minus sign Y 2 there is no need for 2 yes but alpha plus alpha bar that is the real part 2 times the real part so this is equal to norm X square plus mod lambda square norm X square minus 2 time real part of alpha okay now this is true for all lambda I look at a particular value for lambda okay set lambda to be inner product X Y in this order divided by norm Y square okay if norm Y is not 0 that is if Y is not 0 let me rewrite if Y is not 0 Y equal to 0 okay so we need to consider okay we will come to that but that is very easy. So let us look at the case when Y is not 0 when Y is not 0 this number lambda is well defined for this lambda this inequality holds so what happens to this inequality let us just check that so let me call this 1 inequality 1 in this case becomes 0 less than or equal to norm X square plus mod lambda square this is modulus inner product X Y square by norm Y to the 4 and this goes together with norm Y square minus 2 times real part of alpha now what is alpha? Alpha is lambda bar X Y okay from the previous calculations alpha is lambda bar X Y what is lambda bar X Y in this case so let us make that calculation somewhere here what is alpha? Alpha is lambda bar X Y which is lambda bar is again modulus inner product X Y the whole square by norm Y square this is alpha which is a real number in fact a non negative real number so its real part is itself. So this term is minus 2 times so I will remove this minus 2 times modulus inner product X Y the whole square by norm Y square okay for this choice of lambda alpha is a real number non negative real number in fact so I get this inequality Y is not 0 so norm Y is positive I can multiply throughout by norm Y square of course this can be cancelled so this goes with 2 so I will have just these 2 so I am doing this here itself this simplification of cancelling norm Y square in the second term. Now I multiply throughout by norm Y square Y is not 0 norm Y positive inequality will remain so I get 0 less than or equal to I am multiplying throughout by norm Y square so norm X square norm Y square this term has a norm Y square in the denominator that vanishes plus this norm Y square will get cancelled with this I get a plus modulus inner product square minus 2 modulus inner product square that will go with a minus sign minus modulus inner product square. You push this to the left hand side you get and take the square roots which can be done because this is modulus square non negative this is non negative norm this is non negative so you get Cauchy Schwarz inequality okay so what is crucial is this choice of lambda let me prove the last one Cauchy I am sorry triangle inequality and then stop let us consider norm X plus Y the whole square this is by definition inner product of X plus Y with itself again expand this this is first term norm X square the last term norm Y square plus Y with X plus X with Y this is Z plus Z bar Z is a complex number 2 times real part of Z bar so this is norm X square plus norm Y square plus 2 times real part of X, Y but the real part of a complex number what is the relationship it is a real part of a complex number and the modulus Z equal to yeah alpha plus IB then mod Z square is alpha square plus beta square but alpha is less than or equal to modulus alpha is less than or equal to okay there is alpha and beta real numbers alpha less than or equal to okay all that I want to say is this is this clear real part of Z less than or equal to modulus of Z take the square root of this you get this so real part is less than or equal to modulus of that so this one will be less than or equal to maybe I can remove this okay so this real part is less than or equal to norm X square these two terms plus what did I write 2 times modulus of the complex number now apply Cauchy-Schwarz this is less than or equal to norm X square plus norm Y square plus mod X Y that is 2 times norm X norm Y but this is norm X plus norm Y the whole square what is inside the bracket is a non-negative number and this is a norm square so both are non-negative I can take the square root so norm X plus Y is less than or equal to norm X plus norm Y okay so that is the triangle inequality I remember that in Cauchy-Schwarz inequality I did not consider the case Y equal to 0 okay but it is trivially satisfied is not it if Y equal to 0 norm Y is 0 so this part is 0 and we know that X comma 0 is 0 we do not know that X comma 0 is 0 but you can prove that it is 0 so that is that is the exercise in any inner product space V the inner product of a vector with the 0 vector will be 0 inner product of X comma 0 is 0 that is not part of any of the conditions but you can prove it okay so in that case when Y is 0 this is satisfied trivially okay so let me stop here tomorrow we look at see we looked at the inner product that is rather we looked at the norm induced by the inner product we look at a general norm definition of a general norm just for completeness and look at the notion of orthogonal or the notion of orthogonality in a vector space and probably if time permits discuss the Gram-Schmidt authorization process okay so let me stop here.