 Given a basis for a vector space, it's often desirable to produce an orthonormal basis whose vectors are orthogonal with magnitude 1. A common approach to producing this orthonormal basis is known as the Gram-Schmidt algorithm. We'll present traditional Gram-Schmidt first, and then we'll present a better approach. Stop, stop, stop! Don't skip ahead to the better approach. See why Gram-Schmidt works in the first place so that you understand why the better approach is better and it still works. So, suppose we have our set of basis vectors. We want to produce an orthonormal basis. Now, since normalizing a vector almost always requires multiplying by a radical and never changes the direction of the vectors, we'll normalize our vectors at the end. Now, since we have to start somewhere, we'll start by letting w1 equal v1. And we can think about this as the first vector in our orthogonal basis. We want a second vector w2 to be orthogonal to w1. And here's the important concept. If v2 is already orthogonal, we can let w2 equal v2 and be done. But if not, we can remove the part of v2 that is parallel to w1. And here's what that geometry might look like. And so we can think of v2 as some scalar multiple of w1 plus an orthogonal vector w2. So we want the projection of v2 onto w1. And if you don't remember what that looks like, that's going to be the dot product of the two vectors divided by the dot product of the vector we're projecting onto pointed in the direction of that vector. So by the geometry, we see that we want w2 to be v2 minus that projection. And we can generalize this. For w3, we'll start with that vector v3, and then we'll remove the parts of the vectors that are parallel to either w1 or w2. And these are the projections onto these vectors. So for w3, we'll take v3, we'll subtract the projection onto w1, then we'll subtract the projection onto w2, and lather, rinse, repeat. Now if I don't show Gram-Schmidt done this way, I'll have my mathematician card revoked. So we'll do one traditional example, but don't worry too much about the details because they do get nightmarish rather quickly. So let's find an orthogonal basis for the vector space spanned by this rather innocuous set of vectors. So again, the first vector in our orthogonal set is going to be the first vector in our original set. For the second vector, we'll take that second vector in our orthogonal set and subtract off the component that is parallel to the first vector in our orthogonal set. And that's just our projection vector, and so we find. If we muddle through the dot products, we get the third vector in our orthogonal set. We'll start with the third vector in our original set and subtract off the projections onto the other two vectors, 1, 3, 4, and the, well, this thing. And so wading our way through the dot product in the projection formulas we get. Now you can memorize the formula for finding the Gram-Schmidt decomposition. But remember, don't memorize formulas, understand concepts. So we have the formula for that second orthogonal vector. But notice that this also means that w2 is a linear combination of the first orthogonal vector w1 and the vector we're replacing, v2. Similarly, we have our formula for w3, but this means that w3 is a linear combination of w1, w2, the first two orthogonal vectors in our new basis, and v3, the vector we're replacing in the original basis. And so the basic concept is that we form an orthogonal basis by linear combinations of the current set of orthogonal vectors and the next vector in our original non-orthogonal basis. Which makes this a linear algebra problem. So again, we let w1 be our first vector. And we want that second orthogonal vector to be a linear combination of our first orthogonal vector and the vector we're replacing from the original set. Since we want w2 to be orthogonal to w1, we know the dot product will be zero. So if we dot both sides with w1 we get. And we can find the dot products, which gives us the equation. And we can find one solution, x2 equals 26, x1 equals negative 11, which gives us w2. Next we want our third orthogonal vector to be a linear combination of the first two orthogonal vectors and the vector in the original set that we're replacing. Again, because we're supposed to have orthogonality, our dot product with w1 gives us. And if we dot this with w2 we get how I have a system of equations. Incidentally, a good check is that since the only non-orthogonal vector is supposed to be this last one, then each equation should only contain this last variable and one of the other variables. It's a nice continuing check on our work. And we can parameterize it. If we let x3 equals, well, how about 91? We obtain integer solutions, which gives us w3. Now it might seem that we obtain two different orthogonal sets. These vectors, which we obtained using Gram-Schmidt in its classical form, and these vectors, which we obtained by using the concept that we want our vectors to be orthogonal, as well as the idea that every problem in linear algebra begins with a system of linear equations. So which set is correct? Well, it turns out the vectors are scalar multiples of each other. And if our next step is to normalize these vectors, these magnitude differences vanish when we normalize. So while there may be a difference between the vectors, this difference makes no difference.