 Welcome back to our lecture series linear algebra done openly. As usual, I'll be your professor today, Dr. Andrew Misseldine. In the previous section 3.3, we discussed the idea of matrix inverses. When can we cancel out multiplication by matrix? We saw a formula for two by two matrices had to compute the inverse, but for a general formula, we don't yet have. The primary goal of section 3.4 about elementary matrices is to accomplish exactly that, an algorithm for computing the inverse of a non-singular matrix. We're going to do this, develop the inversion algorithm using the technique of elementary matrices, which we're going to define right now. An elementary matrix is a matrix obtained by performing a single row operation to the identity matrix. Remember the identity matrix enforces the matrix with ones along the diagonal zeros everywhere else. As there are three different types of elementary row operations, the replacement, the interchange, and the scaling operations, elementary matrices will come in one of three types in correspondence to those operations. We're going to see that each and every one of the types is actually itself a non-singular matrix. The one we use the most often is replacements. Let's begin with that. A typical replacement operation looks like the following. We're going to replace row i with row i plus c times row j, where c is some specific scalar in the field right here. We're going to add to row i c times row j. What does this do to the identity matrix? Let's think of that for example. If we were to take, let's use a specific example. Let's say we're going to replace row 3 with row 3 plus, let's say, now let's do minus 2 times row 1. Now, if we're going to take the standard, we'll take a 3 by 3 matrix here. Think about the standard identity matrix, ones along the diagonal, zeros everywhere else. If we took row 3 and we replaced it with row 3 minus 2 times row 1, this would change the number in the 3, 1 position, and that would actually make it a negative 2, that we would see right there. Let's call this elementary matrix E1 for later reference here. This is what happens in general, that if you have a replacement operation you're going to do, you're going to take the order we have right here, so the 3, 1, this comes and affects the 3, 1 position, and in the 3, 1 position, you're going to put this number here, the negative 2. The negative 2 is going to go inside the 3, 1 position of the identity matrix. This gives us a typical replacement matrix. We were in the i, j position, we put the number C where i and j represent the row that we're adding together in that order. The order does matter here. Now, I claimed that this matrix is a non-singular matrix, and the inverse of this matrix is also going to be a row replacement. You're going to get ones along the diagonals, zeros everywhere else, except for the one number that was non-zero, which in this case was a negative 2. You're going to replace it with its negative. Negative 2 would become a positive 2, or positive 2 would become negative. Just switch the sign, and I claim these are going to be inverse operations of each other. Let's see that real quickly. I'm going to scroll up a little bit so we have some blank spot here. But if we take 1, 0, 0, 0, 1, 0, negative 2, 0, 1, and you multiply that by its proposed inverse, 1, 0, 0, 0, 1, 0, 2, 1, 0, notice what happens here. When you take the first row, times it by the first column, you're going to hit a 1 and then a bunch of zeros. You're going to ignore everything down here in the second row, in fact. So you're just going to get a 1. Likewise, you're going to get a 0 with the second column and a 0 with the third column. So you end up getting 1, 0, 0, like so. When you do the second row here, you're going to, the only thing that, the only thing that's a chance of not being zero is whatever is in the second row of the second matrix there. And so you're going to grab a 0, 1, 0, and notice you're just going to copy the matrix, its first row and second row, because that's what the identity matrix does. And then lastly, if you take the first row, or the third row times the first column, this time, you're going to end up with, well, you're going to get a negative 2 plus 2, which in that situation, you're going to end up with a zero, negative 2 plus 2. If you take the third row times the second column, you end up with just a zero. Oh, why is there a 1 in that spot? That's alarming. Try that again. This should be zero, one. So now when I do that calculation for real this time, you do get a zero, right? You're going to get zero plus zero plus zero, and when you do the third column, you're going to get a one right there. So in fact, this does give us the identity matrix when we multiply these things together. So these things are inverses of each other like we proposed. So when you have a replacement matrix, you're going to put the off diagonal entry that corresponds to the rows that you're combining. The row position is going to be the row you're adding to. The column position is going to be the row you're adding from, all right? When you want to find the inverse, you just have to switch the sign of the number. Negative 2 becomes positive 2, and positive 2 becomes negative 2. How about interchange? If we were doing an interchange row operation, that means we would be interchanging row i with row j. Now if we did that to the identity matrix, let's again take a three by three identity as an example here. If we took the standard identity matrix and let's interchange two rows, let's say we wanted to interchange rows. We're going to interchange. And we'll do rows two and three, rows two and three. If we did that, so we want to swap rows two and three, what's going to happen is the second row is now going to look like the first row, the third row, excuse me. That's what it was, zero, zero, one. And then the third row is going to turn into what the second row used to be, so zero, one, zero. We're going to call this elementary matrix two for a moment. And so this is what we would see happening when you interchange rows of the identity matrix. You're just going to swap the rows and question. Everyone else says the same, but those who got swapped will switch positions. So the location of ones might look a little bit different. Now what's curious, these interchange matrices are also non-singular. And what you get here is that the inverse of the interchange matrix is actually itself. And think about that, how do you undo interchanging rows? You just put them back the way you were. If two and three got swapped, then if you swap two and three again, they'll go back to the way they were. With replacements, how do you undo a replacement? If I took row three and I subtracted from it two times row one, I can undo that process by adding back two times row one. So inverse operations there. And let's try to convince ourselves for these interchange matrices. If I take the matrix and I square it, I claim I'm going to give back the original matrix. Well, that is the identity matrix is what I meant to say. And so consider that if you take the first row, first column, you're going to get one, second column you're going to get zero, third column you're going to get a zero. That looks like the identity so far. If you take the second column, sorry, second row first column, you're going to get a zero. With the second column, you're going to get a one. And with the third column, you're going to get a zero. So you end up getting one, zero, zero. And then if you take the third row, first column, you're going to get a zero. The second column, you're going to get a zero. And the third column, you're going to get a one. Thus recapturing the identity matrix, like we said it would, interchange matrices are their own inverses. Pretty cool that way. Now the third row operation is scaling. So we're going to take a single row I and times it by some scalar C. And so let's start off with the identity matrix. Again, we'll just take three by three for an example here. Zero, one, zero, zero, one, zero, and zero, zero, one. And let's say we want to scale, let's scale row two by the number seven. So if you scale the second row by seven, what this does is, well, when you times something by zero, it'll still be zero, but one times seven is going to be seven. And so then what we see here for this third elementary matrix type is that if you're going to be doing, if you're scaling row two, then you're going to replace the two, two position with the seven, with the scalar that's in play right here. And that's how we get a scaling matrix. Just remove the corresponding row, the one, replace it with whatever you scaled by. The inverse of this, because this is also announcing the matrix is easy enough, you're going to take one, zero, zero, zero, one, seventh, zero, zero, zero, one. That is you're going to take the reciprocal in the position that you're scaling by. And so to see that these are in fact inverses of each other, take one, zero, zero, zero, seven, zero, zero, zero, one. Times that by the matrix one, zero, zero, zero, one, seventh, zero, zero, zero, one. And go through the multiplication first row, first column that gives you a one, second column gives you a zero, third column gives you a zero. So we get one, zero, zero. Take the second row, this is going to be an interesting one, times it by the first column, you're going to get a zero. With the second column, you're actually going to get seven sevenths, which is a one. And then with the third column, you're going to get a zero. And then the last one, you take the third column, first row, you'll get a zero, second column is a zero, third column will give you a one. Thus giving us the identity matrix again. So we've now talked about the three different types of elementary row operations. Why do we care about elementary row operations? Excuse me, elementary matrices. They correspond to elementary row operations. Well, it turns out that multiplying by an elementary matrix has an interesting effect to matrices. So let's just take a generic three by three matrix. We'll call it matrix A. Its entries are going to be A, B, C, D, E, F, G, H, I. And let's multiply this generic three by three matrix by the three elementary matrices we had considered. So for example, let's take the matrix E one, which was one zero zero zero one zero negative two zero one. And remember, this is the matrix that correspond to the operation where we're going to replace row three with row three minus two times row one. Remember that was the operation I got this thing started. If we go through the multiplication here, we're going to take the first row times the first column, that'll grab us an A, the second column, that'll give us a B, the third column that'll give us a C. Let's record this down. So we're going to get A, B, C. Notice we just copied down the first row of the matrix A, which might not be too surprising because the first row of E one is the same first row as the identity matrix. The identity matrix when you multiply it, you're going to get back the original matrix. So we copy down the first column of the first row of A, excuse me. Well, let's see that same thing happen when we do the second row of E one. When you get the first column of A, you're going to get the letter D. The second column's going to give you the letter E and the third column's going to give you the letter F. And so here again, we're just going to copy down the second row unaffected because after all the second row right here is just identical to the second row of the identity matrix. The third row is when things get interesting. So in this situation, you're going to get a negative two A plus zero D plus G. And so if we summarize what we had there, we're going to have a G minus two times A. I'm going to add a line here to keep some space clear. If we do the third row times the second column, you're going to get a negative two B plus zero E plus one H, which will look like H minus two B. And then finally, if you take the third row times the third column, you're going to get negative two C plus zero H plus one I. And so in the end, you end up with an I minus two times C. And so examining that third row, you can see exactly what's going on right here. The third row looks just like the original row three, but we subtract it from the two times row one. Multiplying by the matrix E one, performed the elementary row operation where we replaced row three with negative two times row one. Multiplying by the elementary matrix performed the associated elementary row operation. Well, that's kind of interesting. What happens if we do the same thing for E two, where E two was this interchange matrix? If we multiply by E two, let's see what happens here. You take the first row times the first column, you get an A, second column gives you a B, third column gives you a C. Which again, the first row of E two, just like an E one, it was identical to the first row of the identity matrix, so you shouldn't expect it to do anything. Well, what happens when we take the second row times A? With the second row, you're gonna get zero zero G. For the second column, you're gonna get zero zero H. And for the third column, you're gonna get zero zero I. So we end up getting the third row G, H, I. The second row kind of disappeared there from A. If you multiply by the third row of E two, you're gonna get a zero D zero for the first column. The second column gives you zero E zero. And then the third column is gonna give you zero F zero, which we'll notice is actually the second column, second row of E, excuse me. So we ended up swapping the rows two and three inside of A. And remember E two corresponded to the interchange operation where we said we were gonna interchange rows two and three, was it? And so multiplying by this interchange matrix had that effect that it performed the associated row operation on the matrix A. Now let's do the scaling here. We might be able to guess what's gonna happen. Remember that E three was the elementary matrix associated with row two is gonna be scaled by a factor of seven, right? So we scaled row two by seven. Well, what happens here? If you take the first row, you're gonna get back an A, a B, and a C as you go through all the columns. So you get back the original row one, A, B, C. And if I skip ahead and I do the third row, you're gonna see the same thing. The first column will give you a G, the second column will give you an H, and the third column will give you an I. So we get G, H, I, nothing happened there. But when you take the second row, right, this is the thing that's different from the identity, you're gonna get zero A plus seven D plus zero G. That's gonna give you a seven D. If we do the second row, third column, a second column, excuse me, you're gonna get zero B plus seven E plus zero H, the third column. You'll get zero C plus seven F plus zero I. And so you see here that the product of these two matrices ends up just multiplying the second row by seven, which is exactly what this scaling matrix was supposed to do. And so then if we summarize what we saw in this previous example, we have the following. If an elementary row operation is performed on an M by N matrix A, the resulting matrix can be written as E times A. So there's a factorization associated to this, well, I mean, there's a matrix product I should say. The resulting matrix E times A where E is the M by M elementary matrix associated to this matrix. This is the same thing. So I guess what I'm saying is that when you row reduce a matrix by a single row operation, that's the same thing as multiplying by that elementary matrix. So multiplying by an elementary matrix does the exact same thing as row reducing a matrix. Performing a row operation. And so what we're gonna see here is that if A is row equivalent to a matrix, there's gonna be an elementary matrix that actually gets you there. And so this factorization, that is we're connecting row operations with matrix multiplication. And that's gonna be the key to finding the inverse of a matrix, which we'll see in the next video.