 For example, take the linear problem Ax equals b and let's find the least square solution. This system is inconsistent if we use the matrix 400211 and the vector 2011. Now in order to find the least squares solution, you have to first compute the normal equations A transpose A equals A transpose b. So we have the first calculate A transpose A, so you're going to get 401021. Remember transpose turns rows to columns and then you get 40211. And so when you do that calculation, you get a 4 times 4 which is 16 plus 0 plus 1. For the next bit, you're going to get 0 plus 0 plus 1. You'll get again 0 plus 0 plus 1. And then finally you end up with 0 plus 4 plus 1. And so A transpose A simplifies to be the matrix 17, 1, 1, and 5. One thing you'll notice here is that this matrix A transpose A is symmetric. That is if you reflect it across the diagonal, you get the exact same thing. This happens in general. We've talked about this earlier in this chapter. A transpose A is always a symmetric matrix. In the case you're having complex matrices, you'll take A star A, in which case A star A will always be a Hermitian matrix, the complex equivalent of a symmetric matrix. If you take A transpose B, 401021, and you times it by the vector 2011, if you multiply that through, you end up with 8 plus 0 plus 11. And then you'll get, let's see, 0 plus 0 plus 11. So you get the vector 19 and 11, like so. So this gives us the normal equations. So if we write this down, we have 17, 1, 1, 5 times it by x. Then this should equal 1911. And you can solve this in a lot of different ways. Like you've solved so many systems equations. You're going to try this Gauss elimination, Gauss-Jordan elimination. You can augment the matrix and row reduce it. That is an acceptable way. In this situation, A transpose A is actually an invertible matrix. So we could actually compute the inverse of A transpose A. It is just a 2 by 2 matrix after all. It's not so bad. To do that, you first calculate the determinant. So you get 17 times 5, which is 85 minus 1 times 1, which is 1. And then to calculate this adjugate here, you swap the diagonal entries, so 5 and 17. You negate the off diagonal entries here. And so you get 1 over 84 times the matrix. 5, negative 1, negative 1, and 17. And then you can multiply B, or I should say A transpose E by this number. And so A transpose A inverse times A transpose B, this is going to equal x hat. And so we have 1 over 84 times 5, negative 1, negative 1, negative 17 times that by 19 and 11. So if we multiply that thing through using usual matrix multiplication, you're going to get 5 times 19, which gives you 95. But you have to subtract 11 from that, which is going to give you 84. Don't forget the scalar. 84 and 84, that's kind of convenient. Almost if I plan that. So you take negative 1 times 19, and then you'll subtract that from 17 times 11. 17 times 11 is 187. So if we subtract from 187, 19, we end up with 168, which 168 is just 84 times 2. So the least square solution we get is 1 and 2. So this right here gives us the closest vector, the best approximation of a solution. It's not an actual solution, but it will simulate a solution the best way that one can. As another example, let's consider this one. We have a 6 by 4 matrix A, and we want AXT equal the vector B there. If you calculate A transpose A, you're going to have to, again, take the transpose multi-together. This will give you a 4 by 4 matrix, since you have four columns to begin with. I'm not going to go through the complete details this time. If you want to, you might want to pause the video and work it out yourself. But A transpose A will look like 6, 2, 2, 2, 2, 0, 0. We get 2, 0, 2, 0. And lastly, we get 2, 0, 0, 2, right there. And like I said before, this will always be a symmetric matrix, as you see in front of you. If we take A transpose B, look at all those possible products. This is going to give you a vector inside of F4. When you go through the products, you end up with 4, negative 4, 2, 6. Again, I kind of skipped over the details there. Feel free to pause the video and calculate yourself. Nice thing about multiplying a matrix, multiply by the transpose matrix, like A transpose A, all this is is just you take all the possible combination of dot products of columns of A together. And A transpose B is just all the possible dot products of columns of A with the vector B. And so to solve this system, we have to augment the matrix. So here we have A transpose A, augment A transpose B. And row reduce this, like some things you could do as you notice, every number is even. You could divide everything by 2. You can move a 1 to the first row if you want a nice pivot. But I won't go through all the details of this. But if you row reduce this thing, you'll end up with its R-R-E-F, which is 1, 0, 0, 1, 3. That's the first row. Second row will be 0, 1, 0, negative 1, negative 5. You get 0, 0, 1, negative 1, negative 2. And then for the last row, you get 0, 0, 0, 0, and 0. So in terms of pivots, we get a pivot in the first column, in the second column, and in the third column. No pivot in the third row or fourth row or fourth column there. So in terms of my solutions, you get a free variable in the fourth column. We get a particular solution. So x hat, as a particular solution, we'll take the particular solution with x4 equals 0. So the first one is 3. The next one is negative 5. The next one is negative 2. The last one is 0. And then we have one free variable, which coincides with x4. And that free variable, if we just look at the negation of the fourth column, we get negative 1, 1, 1, and 1. So this is the general solution. So this is the general least square solution. So like the previous example, we can get that the least square solution is unique. But in this situation, we get a whole line of least square solutions. And that is each point on this line is closest to the solution set. And so one can have multiple solutions. One can have a unique solution, just like the linear problem.