 Welcome back everyone to our lecture series on Litter and Algebra based upon the textbook Litter and Algebra done openly. As usual, I'm your professor, Dr. Andrew Misdine. It's great to have you here today. Today, we're going to talk about section 4.7, which is entitled the Gram-Smith Algorithm. Now, the Gram-Smith Algorithm are sometimes called the Gram-Smith process, is a way of constructing an orthogonal basis. The orthogonal projections and the like that we've talked about previously in the section, like the Fourier coefficients and what have you, typically depended upon having an orthogonal basis. It turns out that using the idea of orthogonal projections, we can actually construct an orthogonal basis given any basis of a subspace. In fact, this doesn't even have to be a basis. If we have a spanning set, we can construct an orthogonal basis using this Gram-Smith process. Then when we're done with constructing an orthogonal basis, if we want an orthonormal basis, we can normalize each of the vectors to make them univectors. Now, some people present the Gram-Smith Algorithm as normalizing along the way. I don't really see a need of doing that initially. We can always normalize when we're done. And the Gram-Smith process, it actually constructs the orthogonal basis in a recursive manner. And so let's explain what we mean by that. So suppose we have already in hand a basis for our vector space W that sits inside of Fn. And so what we're gonna do is recursively create a basis that's orthogonal in the following manner. Well, if that basis is empty, then well, we're already done. We don't need to do anything about that because the empty basis is an orthogonal basis for the trivial subspace. But for a positive dimensional vector space, we're gonna take, we already have the basis X1, X2 up to XP. And so we're not gonna do anything to the first vector. V1, which would be the new basis that we're forming, we'll just take that to be X1, no change whatsoever. But for V2, what we're gonna do is we're gonna take the current X2 and we're gonna subtract from it V1.X2 over V1.V1 times V1. That is we take the orthogonal projection of X2 onto V1 and we subtract it from X2. And this is gonna have the consequence that the new vector V1 will be orthogonal to the previous vector V1. So the current vector V2 will be orthogonal to the V1 using this orthogonal projectionality. And then for V3, we do the same thing. We're gonna take the current X3, we're gonna subtract from it the orthogonal projection of X3 onto V1 and the orthogonal projection of X3 onto V2. And when we do these together in tandem, this is the orthogonal projection into the subspace spanned by V1 and V2 of the vector X3. And so this new vector V3 will be orthogonal to the vectors V1 and V2, excuse me. And so then we just replicate, we repeat this process over and over and over getting recursively. V4 will be X4 subtract the orthogonal projection of X4 onto V1, V2, V3. X5, sorry, V5 will just be X5 subtract the orthogonal projection of X5 onto V1, V2, V3, V4. And we keep on repeating this over and over and over again until we get to the last vector Vp for which we take Xp subtract from it the orthogonal projection of Xp onto V1, the orthogonal projection of Xp onto V2 onto V3 onto V4 all the way down to Vp minus one. All right. And so we do this step by step by step. And so each step along the way, we're replacing an X with a vector V, which will be orthogonal to all the previous Vs. And we're gonna have the property that every time we replace a vector V, sorry, we replace a vector X with a vector V, we don't change the span. So if we go from one up to K, one up to K, we don't change the span of the original set X1 up to XK, but we are replacing with orthogonal counterparts. And the basic idea is the following, X1 and V1 are the same vector. So interchanging them does nothing. If you replace V2, that is, if you replace X2 with V2, well, notice that V2 is just X2 minus some multiple of V1. Since we have a V1, you could replace that away and you get back the original X2. And so when we come to V3, V3 is inside the span of X3, V1 and V2, which by mathematical induction, we know we'll have the same span as the previous ones. So V3 doesn't add anything new, but if we use V3 instead, we can reconstruct X3 using V3 in combinations of V1 and V2. So by induction, this doesn't change the spanning set whatsoever. So let's show you an example of how this process works in practice. So take the following three vectors, X1, which is 1111, X2, which is 0111, and X3, which is 0011. And this will form a subspace of R4, that is W is the span of X1, X2, X3. If we apply the Gram-Smith process to that, what do we get? Well, the Gram-Smith process will reproduce the exact same basis if these vectors are already orthogonal, but a quick check shows that they're not orthogonal. If we take X1 and X2 together, the dot product is three, X1 and X3 gives us a dot product of two, and then X2 and X3 gives us a dot product of two again. So these things are not orthogonal. So step one of Gram-Smith is pretty easy. You just take V1 to equal X1, you don't change anything, so just get 1111 here. Make no changes whatsoever. For V2, what we're going to do is we're gonna take X2, and we subtract from it V1 dot X2 over V1 dot V1, and times that by X1, V1, excuse me. So let's see the details of that. So the original X2 is 0111. And so if we take the dot product of V1 and X2, X2 is the vector you see just drawn right there, and then V1 is still just X1. If we take the dot product of those, we're gonna get a three. And if you take the dot product of V1 with itself, you get a four. And we're gonna times that by 1111, V1 there. And so when we combine those together, you're gonna end up with zero minus three-fourths. So you get three-fourths right there. You're gonna get one minus three-fourths, which is just a one-fourth. And then that actually happens with all the other coefficients as well. So you get three-fourths, one-fourth, one-fourth, and one-fourth. And that should be a negative three-fourths at the top there. Sorry about that. And so that gives us a vector, which is gonna be orthogonal to the original one. Notice now if you take V1 dot this V2, you'll get negative three-fourths, plus a fourth, plus a fourth, plus a fourth. That gives you zero. Now, if you are completely important to all fractions, the ratio phobic amongst us might be panicked here. Be aware that you could notice there's a common factor of one-fourth. If you were to scale that out, the one-fourth, you get negative three-one-one-one, like so. And this vector, negative three-one-one-one, is still orthogonal to V1. We're gonna get still negative three, plus one, plus one, plus one if we take the dot product. And as scalar multiples don't change, they don't change the span whatsoever. We actually could get rid of the one-fourth and just take V2 to be this vector right here, negative one-one-one-one. We're gonna normalize all these vectors in the end. So it doesn't really matter what scale and multiple you do. If you prefer the integers, that's perfectly fine. I do too. I'm not afraid of them, but I think integer arithmetic takes a little bit easier than working with fractions. So we're gonna use negative three-one-one-one for our V2. All right? So how do we do V3 then? So we're gonna take X3, whoops, we're gonna take X3, which remember that was the vector zero, zero-one-one. We're gonna subtract from it V1.X3. So X3 is the vector I just wrote down there. V1 was all ones. So you take the dot product, you get two, this will sit over four, and you times that by one-one-one-one, all right? And then we have to set track from this also V2, which as we mentioned before, V2 is now gonna be negative three-one-one-one, but we have to compute the Fourier coefficients, all right? If you take V2.X3, you end up with a two. If you take V2.V2, you're gonna get a nine plus one, plus one, plus one, that's a 12 sits in the bottom. Now these numbers will be a little bit different if you had used the original V1, V2 we had, this one right here. But the thing is we can scale this because these Fourier coefficients will correct any scalar multiplication we have there. That's why I'm perfectly happy using integers. Let's simplify some of the fractions because after all one, the two-fourths is the same thing as one-half, and the two-twelves, we can make that become a one-sixth. And so adding everything together, for the first component, we are gonna get zero minus one-half, we're then gonna get plus three-sixth, and I'm gonna help us out there. Let's see, three-sixth of course is one-half, zero minus a half plus a half is zero, we like zeros. For the second component, we still get a zero, we're gonna get a negative one-half, and this time we're gonna get a minus one-sixth. And so I would need to write this one-half here. Like I said, we're afraid of fractions. This would be three-sixth. So this will all add up together to be negative four-sixth or negative two-thirds. And then the third component is gonna be one minus a half minus a sixth. So this is gonna take the negative two-thirds we had before, but we're adding one to it. So we get a positive one-third. And then the last component is the exact same thing. So we get another one-third as well. And so this is our vector v3. If you don't like the fractions, again, we could factor out the one-third. So we get zero, negative two, one, and one. And so then we could just take v3 to be that scalar multiple, zero, negative two, one, and one, like so. And if you check with the previous vectors, if you take v3.v1, remember v1 was the all ones vector, so you're gonna get zero, negative two, plus one, plus one, that's a zero. And if you take v3.v2, which is up here, negative three, one, one, one, what you're gonna see happening there is you're gonna get negative two, plus one, plus one. So that's also orthogonal. So these three vectors are now orthogonal. And so this is our new basis, c. We didn't change the first vector at all, all ones vector. The second vector was then negative three, one, one, one. And then the last one we just got was zero, negative two, one, and one. And so this provides now an orthogonal basis, an orthogonal basis for the original vector space, w. We did not change the vector space, we just changed that we now have an orthogonal basis for which then we can do all the cool things with an orthogonal basis for this vector space now. So we can always create an orthogonal basis if we need one. Now this is sort of if we want an orthogonal basis and we also chose this so they're all integer numbers there. If we want an orthonormal basis, then remember the idea is we need to normalize everything. So we could take u one to be the normalization of v one. So one, one, one, one. The length of that vector is gonna be the square root of four. So we get the vector of all one halves. For the second one, we wanna normalize that. So we take the vector, what did we have before, negative three, one, one, one. And we actually calculate the length of this thing earlier, remember, because when we were doing the Fourier coefficients, we had to do v two dot v two, which gave us the 12. Well, now we just take the square root of that, which of course, if you want, you can rewrite the square root of 12 as just two root three. I don't know if that's necessarily simpler. I'm perfectly happy with the square root of 12 and the denominator there. So you two, we're gonna get negative three over the square root of 12, one over the square root of 12, one over the square root of 12, and one over the square root of 12. So that's our second vector there. And then for the third one, just normalizing this thing, u three, the vector we had was zero, negative two, one, one. And so if we calculate its norm, we're gonna get four plus one plus one, which is six, square root of six in the bottom. And again, just leave them. You don't need a rational answer denominator. That's a silly thing that people tell us that it really gives a good reason, negative two over the square root of six, one over the square root of six, and one over the square root of six. So with these vectors in mind, we now have an orthonormal basis. If we take the set u one, u two, and u three, this is now an orthonormal basis for w. So if we have a basis for any subspace, we can construct an orthogonal basis and then an orthonormal basis from that. And like I said, if you just have a spanning set, this process will automatically prune down that spanning set when you form the orthogonal basis. Whenever you have a vector you didn't need, it'll actually produce a zero and just throw it out of the process.