 So, this is a process by which given a set of linearly independent vectors in a inner product space. So, x is my inner product space, this is my inner product space and I am given some set of linearly independent vectors. So, I have this set S which comprises of which is a subset of x and S corresponds to say x 1, x 2 and so on. I have given given a subset of vectors in the inner product space, this could have finite number of vectors, it could have infinite number of vectors. Right now I am not worried about how many vectors are there in this set, they could be finite, they could be infinite. All that I know is that these vectors are linearly independent, these are linearly independent but this is not an orthogonal set, S is not an orthogonal set. So, S is not, what is an orthogonal set? The vectors are mutually orthogonal, you take any pair of vectors and find a inner product, inner product will be 0. So, that is where you have a set to be called as orthogonal set. So, this is not an orthogonal set and I would like to generate an orthogonal set because orthogonal sets are very very useful when you do modeling, when you do applied mathematics numerical computations. So, how do I do that? I start by defining a unit vector, I start by defining a unit vector. So, E 1, my E 1 is going to be x 1 divided by norm x 1. Well, inner product defines a norm, so my norm x 1 is nothing but 2 norm is nothing but inner product of x 1 with itself rest to half. So, this is my first vector, this is a unit vector, I want to create a set starting from this set, I want to create a set which is not just orthogonal but which is orthonormal. I want to create unit vectors, I want to create unit vectors which are orthogonal to each other. So, this is my first vector, what I do next is, well, orthogonality allows us to split a vector into two components, one along the direction and one orthogonal to the direction. That is the concept which I am going to use in Gram-Schmidt process. So, what is my first thing? First thing is I pick up now this vector x 2 here, I pick up this vector x 2 here and then I create a vector z 2 which is x 2 minus x 2 E 1 inner product of x 2 E 1. So, this gives me component of x 2 along E 1 times E 1, you can very easily check that this vector and so if I define another vector say v 2 which is, I have two components of vector x 2, z 2 and v 2. This is one component, x 2 minus v 2 is another component, this is nothing but, this is what I am calling as v 2, x 2 minus v 2 and v 2, I am decomposing the vector x 2 into two orthogonal components. It is very easy to check that inner product of v 2 and z 2 is equal to 0, v 2 and z 2 inner product is 0. Just substitute and find out, you will get inner product to be 0, not at all difficult, these two are orthogonal components. I am splitting vector x 2, okay. So, x 2 is okay, so now what I do next, well I got two directions which are orthogonal, one is E 1 and other is z 2, see because v 2 is just some scalar times E 1, right, some scalar times E 1. So, one direction is E 1 and z 2, these are orthogonal to each other and then I define E 2 which is z 2 by norm z 2, okay. So, I got two directions, E 1 here starting from the first vector, then I removed the component along E 1 from x 2, from x 2, I created z 2, then I just normalized z 2 to create E 2, okay. So, now there are two directions, E 1 and E 2, both of them are unit magnitude, this is a unit magnitude vector, right, unit magnitude vector and E 1 and E 2 are orthogonal, okay. I just do this by induction, so I take x 3, I remove component along E 1 and E 2, whatever remains I make it unit magnitude, I go to x 4, I remove from E 1, E 2, E 3, just go on doing this. So, this process is called as, so my next step would be, you know, define z 3 which is x 3 minus x 3 inner product E 1, E 1 minus x 3 inner product E 2, x 3 inner product E 2, E 2, so this is my vector z 3 and then using z 3, I can define E 3 which is z 3 by norm z 3 and so on, okay. So, given a set of vectors which are not orthogonal, I can just follow this systematic procedure to split, to create a set which is orthonormal, okay and creation of this orthonormal set from a non-orthonormal set is called as Gram-Schmidt process, okay. So, now let us start today doing some things, let us actually look at some examples and let us create some orthonormal sets starting from some non-orthonormal sets, okay. So, my first example is going to be in R 3, my first example, my inner product space is simply R 3 and the inner product between any two vectors is simply x transpose y, okay, simply x transpose y and I am given 3 vectors x 1 which is 1, 1, 1, x 2. Now, I want you to do this by hand, I want to start doing it, 1 minus 1, 1 and x 3 is 1, 1, minus 1. Are these linearly independent? Are these three directions linearly independent in R 3? These are linearly independent. Are they orthogonal? They are not orthogonal. You take inner product of any two, it will not get 0. So, these are not orthogonal directions. I want to construct an orthogonal set starting from this non-orthogonal set. I want to apply this process. So, just start doing this. What will be e 1? e 1 will be simply 1 by root 3, 1, 1, 1. What will be z 2? Just calculate. So, we have to start with, so what is inner product of x 2 e 1? What is this quantity? x 2 is this vector 1 by root 3. So, what is the second vector? What is z 2? Will be 1 minus 1, 1 minus 1 by root 3 times 1 by root 3, 1, 1, 1. So, what is this vector? Two-third minus four-third and two-third. So, this gives you, so z 2 is, you said two-third minus four-third, two-third transpose. So, this is my z 2. So, what is e 2? You have to help me with this. Three root six, two-third minus four by three, two by three. This is my e 2. Just check whether e 1 and e 2 are orthogonal. What do you get? If you do even transpose e 2, what do you get? You get zero. If you do not get zero, you have made a calculation error. You must get a zero here if you take this inner product. And those who have done this, just go ahead to e 3, compute e 3. Does this turn out to be zero? It does. Just check. If you take inner product of this with e 1, you should get zero vector, not zero vector, zero magnitude. Inner product should be zero. Even transpose e 2 should be zero, perfectly zero. If you are not getting it, there is some error somewhere. Is it zero? I do not hear yes clearly. Yes, it is zero. What about next? What about x 3? What about z 3? What is z 3? 1 1 minus 1, then even what is inner product of x 3 and even? 1 by root 3. 1 by root 3 into 1 by root 3, 1 1 1, then minus. What is the inner product here? So, what is the number that should appear here in a product? e 2 with 2 by minus 2 by root 3 minus 2 by root 6. So, this becomes 2 by root 6. Is that fine? Because there was, did we have a minus? Is this correct? Is this minus correct? This is correct. What about here? This becomes plus. Does it become plus? Is this fine? So, what is this vector finally? Can somebody help me with this? What are these three numbers? Well, after you find this z 3, you have to make it in its magnitude. You have to divide it by its magnitude. But that is a simpler part. What will be, has anyone completed? You tell me, this could be, I am just writing here, I am not doing the calculations. I am just writing here, somebody is prompting me. So, you have to tell me whether this plus is correct or minus is, should be minus here. Plus is correct here. So, what is the total? 1 0 minus 1. See our friend says 1 0 minus 1. Everyone agrees or there are some different calculations or, by the way 1 0 minus 1, is this orthogonal to this and this vector? It seems to be this is this 1 0 minus 1 is orthogonal to this vector. This direction, forget about the multiplying factor. This direction is orthogonal to this. What about here? 1 1 1. Oh yeah, it is. So, these are mutually orthogonal because this and this are orthogonal. This and this are orthogonal, this and this are orthogonal. So, 1 0 minus 1 seems to be, I am not sure about the multiplying factor. Is that correct? 1 by root 2, okay. E 3 is 1 0 minus 1. G 3. So, E 3 becomes, E 3 becomes 1 by root 2 1 0 minus 1. So, I started here. So, I started here with 3 vectors in R 3, okay, which were not which were not orthogonal and then systematically I could construct, I could construct set of 3 vectors which are unit magnitude orthogonal vectors, okay. Unit magnitude orthogonal vectors. So, if you try to, if you try to, you know, you have started with something like this in R 3, 3 vectors which are, you know, linearly independent, okay. Let us say v 1, x 1, x 2 and x 3. Starting from this, what you have done is you have created a set which is, which is orthonormal, okay. You have created a set which is orthonormal starting from a set which was linearly independent but not orthogonal, okay. So, we had a situation like this, we moved to orthogonal set, okay. That is what, that is what we have done. Now, specific vectors that you get here will depend upon how you define the inner product, okay. I just want to do repeat one calculation. Well, subsequently all the calculations will change but if I change my definition here of the inner product, okay, the subsequent calculations will change. The directions may not change in R 3 but magnitude calculations can change and I want to emphasize this one small thing. Is this clear? We started with a non-orthogonal set and we came up with three directions which are orthogonal to each other, okay. So, now let me just do one small change here. My second example is again R 3. My second example is again R 3 but I am going to change the definition of my inner product to x transpose W x, x transpose W y where W is a symmetric positive definite matrix. I am going to pick up one particular matrix here. I am going to pick up this matrix W. Well, there are many ways you can pick up a symmetric positive definite matrix. Take a matrix which is simply diagonal elements which are positive, okay. So, that is one way. I am going to pick this matrix 2 minus 1 1. Is this a symmetric positive definite matrix? It is a symmetric matrix that is for sure. Is it positive definite? Do you know a test for finding out positive definite list? Eigenvalues. Any other test? Principal minors, this is greater than 0. 2 is greater than 0. Is this determinant greater than 0? This determinant is greater than 0. What about this determinant? All three put together. Calculate. Is the determinant greater than 0? The simple algebraic test to find whether a matrix is positive definite or not. Look at these matrices constructed by first element and first 2 cross 2 matrix then 3 cross 3 matrix. Is it positive? Okay. It is a simple test to find whether a matrix is positive definite or not. So, this is a positive definite matrix and X transpose W Y will define an inner product on R 3. This inner product is different from what we defined previously, okay. So, now just remember what is our 2 norm? 2 norm is X inner product X raise to half. So, in this case it will be you know X transpose W X raise to half. All the calculations will have to be done using X transpose W. So, what will be the unit direction now? What is my first vector? What is the very first vector that you what will be E 1? E 1 is X 1 divided by norm X 1. What is norm X 1? So, you have to work with 1 1 1 transpose this matrix. So, norm X 1 square is 2 minus 1 1 minus 1 2 1 minus 1 1 minus 1 2 into 1 1 1. So, what is this quantity? 1 is 6 2. This multiplication comes out to be 2 square is 4. So, this comes out to be 4 and then square root of this is 2, okay. So, what is the direction? First direction? Yeah, the first direction then becomes E 1 becomes half 1 1 1. This is different from what we got earlier, right. Earlier we defined inner product in a different way. So, the norm which was defined through the inner product was different and the unit vector was different. With this definition of inner product which see this definition of inner product here X transpose W Y where W is a symmetric positive definite matrix that induces a norm 2 norm. The 2 norm of 1 1 1 using this definition of inner product turns out to be 2. 2 norm square is 4, right. And unit vector the direction is same but the vector is different, right. The direction is same the vector is different. Earlier we got 1 by root 3 1 1 1, okay. Now I am getting 1 by 2. So, what I want to do is further what I want to stress here is all further calculations will have to be done using this inner product. Don't forget this, okay. So, in the exam if I give you a problem which has matrix W which is see earlier we had a special case W was identity matrix 1 1 1 and okay. If I give you a different matrix W you have to keep using that matrix every time you calculate inner product in that example because R 3 with this inner product is a different space different inner product space than what with W is equal to I. R 3 with W is equal to I and R 3 with W is equal to this matrix are 2 different inner product spaces, okay. Calculations will be different, very very important. Is this clear? I am not doing the further calculations. We will move on to some other example, okay. Well now I want to graduate from finite dimensional spaces to infinite dimensional spaces and then we will see how we start meeting some of the old friends that you have known in your undergraduate curriculum. So now inner product space is any set which satisfies certain axioms, right and we have generalized the concept of inner product space. Now I am going to look at set of continuous functions. My inner product space is going to change my inner product definition is going to change. So my third example and this is where now you have to do some workout, okay and you have to help me on the board and as to how do we come up with vectors which are unit vectors, okay. Then we start doing developing vectors which are orthonormal we start with a non-orthogonal set and develop an orthogonal set same idea, okay. Now my x is set of continuous functions over 0 to 1. My inner product space is set of continuous functions over interval 0 to 1. So I am going to do some workout, okay.