 And so under what circumstances can we predict a unique solution? So if you have a matrix A, which is n by n, then the following three conditions are equivalent to each other. The linear problem AX equals B, I should say the least squares problem has a unique solution. There's a unique least square solution. That's equivalent to the columns of A are being linear independent, and that's the same thing as A transpose A is invertible. One can show that all three of those conditions are the same thing. And like we saw before on the first example, when A transpose A is invertible, so that is we have A transpose AX hat equals A transpose B. Well, if it's invertible, you can multiply on the left by both sides, A transpose A inverse. Oh boy, I'm gonna squeeze it in there. A transpose A inverse. And so then you actually get that the least square solution is X hat there will be A transpose A inverse A transpose B. Now, I wanna caution you here that A transpose A inverse is not necessarily the same thing as A inverse A transpose inverse. The issue here is that A, is it a square? We don't actually know. Now, if A was a square matrix, then yes, we could say that very thing because if A was a square matrix, having linear independent columns would make it invertible. But the thing is A could be any rectangular matrix, so A inverse might not even make sense. But A transpose A can still make sense because even when A is a non-square matrix, A transpose A will always be a square. All right? And so another thing I wanted to comment about here is that the least square, it turns out the least squares problem actually allows us to compute orthogonal projections without using an orthogonal basis. We have learned previously in this chapter that we can compute orthogonal bases using the Graham-Smith process. So we can actually do that, but it turns out with an orthogonal basis, we can compute orthogonal projections, but we don't need an orthogonal basis to do an orthogonal projection. So let's say we have a vector space W and we have a spanning set, preferably a basis. So assume V1 up through VR is linear independent. It needs to be at least to be a spanning set. And for which case we can build a matrix A whose columns are that spanning set. And this will guarantee that the column space of A is equal to the same subspace W. And so with that in mind, let's think about the orthogonal projection of any vector Y onto W called Y hat. Well, since W is equal to the column space, we can make that interjection there. And like we've been talking about early in this lecture, the orthogonal projection onto the column space is none other than Y column. And like we talked about here, Y column is just gonna equal AX hat where X hat is the least square solution to the system AX equals Y. So if you wanna find, if you're trying to find Y hat, to do that, what you're gonna do is find the least square solution to the equation AX equals Y. If you find the least square solution, then Y hat will just equal AX hat. AX hat. So you solve the least squares problem that you multiply that by A and that'll give you the orthogonal projection. And so that's kind of a neat little trick there. Continuing on with that, if we have a matrix A, which is M by N and it has linear independent columns, so there'll be a unique solution to the least squares problem. And if W is the column space of that matrix A, then the orthogonal projection onto W, this will be the map FN to W, it'll have as a standard matrix representation. The following, that is the matrix that represents the orthogonal projection will be A, that's a typo there, I'm sorry. Let me correct that. You get A times A transpose A inverse times A transpose. And that can kind of be a beast to compute, but the alternative is one doesn't have to compute an orthogonal projection, which orthonormal bases, although useful have a lot of square roots and such, this thing might entirely avoid any square root. There might be some fractions because you are taking an inverse, but this might be a more preferable way of computing orthogonal projections as opposed to an orthonormal basis. Also, to make connections with the QR factorization we talked about before, if A has linear independent columns, then it has a QR factorization. Remember, in the QR factorization, the matrix Q will be, its column vectors will be orthonormal. And with R, this will be a unit upper triangular matrix, right? And not unit, but the diagonals we pause, it will be an upper triangular matrix, all right? And so then you have a matrix A, which has a QR factorization. You have any vector B, then it turns out the unique least square solution to the problem AX equals B. I never wrote that down here, but the unique least square solution to AX equals B will be the vector X hat, which can then be factored as the form X hat equals R inverse Q transpose B. Now, be aware that in order to find the QR factorization that typically requires we do the Gram-Smith process. And so what we're basically saying here is if you've done the Gram-Smith process, you have essentially an orthogonal basis, you can find the least square solution. But also, if you have the least square solution, you can find the orthogonal projection. So the least squares problem and the least squares problem and the orthogonalization problem are essentially the same thing. Hand in hand, you can choose one or the other if you prefer. And so to kind of finish up here, let's take the matrix A with the vector B that we had before. Remember, we solved that problem. There was a unique solution, which was the vector one comma two. If we wanna compute the orthogonal projection, we can take the matrix A and we can multiply it by this least square solution, one, two. And if we go through that, we're gonna get four, four, and three, which this right here is the orthogonal projection of the vector B, two, zero, 11, into the column space of A there, into that subspace W. So if you have the least square solution, you can find the orthogonal projection super easy. But that's if you already have the least squares problem solved. If you don't have the least square problem solved already, you can do the orthogonal projection by computing all these things. So we get four, zero, zero, two, 11. And like we did before, the inverse of A transpose A, I'm gonna write the one over 84 above. What was that? We get five, negative one, negative one, 17. And you have A transpose, four, zero, one, zero, two, and one. Like so, you go through this matrix multiplication. There's some calculations to do. If we do the first two matrices together, the three by two times the two by two, you'll end up with, let's see, you get five times four times plus zero, that's a 20, four is zero times one and negative, 17, that's a negative four. Do the next row, you're going to get negative two and positive 34. And then the last row, you will get five minus one, which is a four, and you'll get 17 minus one, which is a 16 times that by then four, zero, one, zero, two, one. If you do this three by two times the two by three, you end up with 80, negative eight, 16 for the first row, you'll get negative eight, 68, 32 for the next row, and then you get 16, 32, and 20 for the last row. Although nothing is nicely divisible by 84, there are some common factors of like two in the such. So we can simplify this thing. We can simplify this as one over 21 times the matrix 20, negative two, four, negative two, 17, eight, four, eight, four, eight, and five. And so this gives us the orthogonal projection, its standard matrix onto W right here, the column space. And so if we were to multiply this matrix by the vector two, zero, 11, vector two, zero, 11, go through the details of that right there. You have this one over 21 sitting up in front. You're going to end up with 40 plus zero plus 44, that is 84. Then you're going to get a negative four plus zero plus 88, that's equal to 84 again. And then lastly, you're going to get eight plus zero plus 55, that adds up to be 63. And then all of those are actually divisible by 21. 84 is four times 21, and 63 is three times 21. And so you get four, four, and three, which if you remember from what we saw above, four, four, three, that was the projection. So this one looked really drawn out in terms of calculations. That's because we computed the projection maps matrix. And then the issue now is that I can continue to multiply this matrix to do all the orthogonal projections. Like this one seemed a little bit longer compared to what we did before. That's because we solved the least square problem on a different page. But also this solution only really works for this vector two, zero, 11. This matrix down here at the bottom does the orthogonal projection for every single vector we could do. So the least square problem helps with the orthogonal projections. And it helps us solve so many problems when there might be inconsistency. So like one last comment I do want to mention here is like take an example in statistics. For example, let's say we have a scatter plot of data, something like this. And we're like, there's not like one function that seems to hold all these points. But it kind of does feel like this data set forms maybe a line. And so we can do something like this. We could be like, is there a line of best fit? Is there a line? Whoops. Is there a line that kind of minimizes the distance between points off the line? Can we do that? Can we find this line of best fit? And it turns out the line of best fit that you might have seen in the stats class is exactly the least square solution to some curve fitting system of linear equations. And although we won't talk about that this time, maybe we'll talk about it another time. This least square solution has immense applications to so many things because so often our problems are inconsistent linear systems but we need an approximate solution like this line of best fit, this best fit line. And that then ends chapter four in this discussion about the least squares, the least squares problem. If you liked this video, please feel free to like it, comment positively, subscribe. If you wanna see more videos like that, subscribe and you can get updates as I upload more videos in the future. If you have any questions about the lecture today, feel free to post a comment and I'll be happy to answer that sometime in the future. Continue on with your linear algebra endeavors and I will see you later. Bye everyone.