 In this video we're going to talk about the LU factorization of a matrix. Now suppose we have a matrix A which is M by N, it does not necessarily have to be a square matrix, any size of matrix is appropriate here. So let A be an M by N matrix which has an echelon form which can be obtained solely by replacement row operations. So we can get from A to its echelon form which we're going to call the echelon form U, it'll have the same dimensions that A does of course, and if we can get there with only using replacement row operations, then there's going to exist another matrix L which is going to be M by N, this one is a square, such that A will factor as L times U where what conditions can we expect for the matrices L and U? Will U will be an echelon form of A that we obtained from these row replacements? And also L is going to be a lower unit triangular matrix. So remember that what that means, lower triangular means that everything below the main diagonal will be, well everything above the main diagonal will be zero I should say. So unit triangular means we'll have ones along the diagonals, we can have anything else we want below the diagonal, whatever, but we'll have zeros above the diagonal. So we get this unit lower triangular matrix and in particular the numbers that are going to be below the one diagonal there is going to come from the replacement operations we do. So whenever you do a row replacement where row I is replaced with row I plus C row J, you're actually going to put a negative C inside of the IJth position. So the numbers below the diagonal here come from these row replacements. And the argument on why this factorization exists and this is called the LU factorization, not a very clever name but it's what's commonly referred here, the proof behind the LU factorization really comes from the elementary factorization we had done in the previous lecture. So when computing an echelon form of a matrix the scaling operation I just want to mention is never actually needed. When you look at Gauss-Jordan elimination, scaling is not required until the backwards phase. The forwards phase does not require scaling whatsoever. Sometimes interchange might be necessary in the forward phase but like we said for this for this theorem here we're making the assumption that interchange is not necessary and I'm just making the point that scaling matrices is never necessary to get into echelon form. So suppose that A reduces to an echelon form U without any interchange whatsoever. Then there's going to be some sequence of elementary row operations. So they're going to be replacement operations that transforms A into U and let's say there's P many of those. Now each and every one of those row operations will correspond to an elementary matrix. So let's say that the first elementary row replacement corresponds to the matrix E1, the second one corresponds to E2, the last one corresponds to Ep until eventually we see that we have this factorization that Ep times Ep minus 1 all the way down to E2 E1 times A is going to equal U. Now if we multiply by the inverse matrix on the left side we can move this product of elementary matrices to the right hand side and we end up with the statement that A equals the inverse of the product of all those elementary matrices times U. Then of course if you take the inverse of a product the shoe sock principle comes into play. You put your socks on then your shoes but then you take your shoes off then your socks. That will reverse the order of each of the elementary matrices so now E1 shows up first then E2 and then Ep's at the end but we take inverses of each and every one of these things. Now the inverse of a elementary matrix is an elementary matrix and in particular if you have an elementary matrix of replacement type its inverse will be a replacement type and that's why we have an inverse C that shows up here because we perform the operation we added C times row J well because of the shoe sock principle you can actually be taking the inverse of C and so then we're going to define L to be the product of all of these elementary matrices. Now if you have in the forward phase if you are doing an elementary operation which is replacement this is going to put a number below the main diagonal because you're taking a row from above and adding a multiple of that to a row below so each and every one of these elementary matrices is going to be a forward phase replacement matrix that these are all going to be unit lower triangular matrices and as we talked about previously the product of a unit lower triangular matrix will likewise be unit lower triangular and I shall also mention the inverse of the unit lower triangular matrix is a unit lower triangular matrix therefore the product of of these elementary matrices L is going to also be a unit lower triangular matrix and the coefficients the scalars inside of that matrix are going to be determined exactly by these elementary row operations we talked about here so the reason why it's so often important to go through proofs of mathematical statements is that the proofs illuminate on why it's true and it often gives us clues on how it is true that is it gives us the proof here actually gives us an algorithm for computing the LU factorization of a matrix so the first thing to do is to compute you know to start with your matrix A and you're going to row reduce it to find your echelon form using only uh not backwards phase what why is that saying that we need to be doing forward phase uh row replacement operations uh no interchange no scaling whatsoever so row reduce A into U once and then key track the row operations you did and while what when you get to U then you go back and you can look through the row replacements you did and so for each time you do the replacement row i transforms into row i plus c row j you're going to put a negative c in the ij position and honestly i do this at the same time and i want to show you in an example how we can do this so consider the following four by five matrix A we're going to work over mod 11 in this example uh switching the field from the real numbers to the complex numbers to the finite field zp or any other field it doesn't really make any difference on how this calculation is going to go so because the matrix A has four rows the uh the matrix L is going to be four by four uh and then the matrix U will also be four by five the echelon form will have the same shape that A does L though will just be the number of rows because we're on we're multiplying on the left of U so it needs to be four by four so what i'm going to do over here is you're going to see to the left the sequence uh we're going to see the sequence of reduction from A to U now U just has to be any echelon form and so we're going to grab grab U to be the first echelon form um that as we broke reduce A right here L is going to be this four by four matrix and what we know about L is the following it's unit lower triangular so we have ones along the diagonals it's lower triangular so everything above the diagonals is going to be zero the stuff that's below the diagonals we don't know what those are yet so we're going to determine that through our row reduction here so this right here is just our original matrix A unaffected so the first thing we do to row reduce it again no interchange nothing like that's possible here we're going to first put a pivot position in the one one spot and then we have to get rid of all the numbers below so how are we going to get rid of the seven that's down here well to get rid of uh to get rid of the seven what we would have to do is we have to take seven divided by two we're going to subtract row one from row two in that regard but what is seven over two working mod 11 uh we could say add 11 to seven that would give us 18 over two which reduced to nine mod 11 so we're going to take row two minus nine times row one that's going to take care of the seven that's below uh the next one how do you get rid of the two right here that's pretty easy you're just going to take two minus two so we'll take row three minus row one and then for the for the five right here together the five we have to take row four minus five divided by two row one but what is five halves just like the seven i'm going to replace the five with five plus 11 which is 16 over two that reduces to eight and so we're going to subtract eight times row one to cancel those things out there and these this this observation here is critical because we have row two minus nine row one we're going to add a nine in the two one position we're taking the opposite of the negative nine we have right here uh because we're taking row three minus row one we're going to put a positive one in the three one position because again we take the the additive inverse of this coefficient and lastly because we have row four minus eight times row one we're going to put a positive eight right here if you always think of your replacements row i minus c times row j if you think of that way then you're always going to put a c inside of your matrix the opposite of the sign you have right there so now we actually have to go through these calculations right here uh so notice that if we go through this negative nine when you're working mod 11 is the same thing as plus two row two plus two which is a little bit easier in terms of multiplication so i'm going to take row one and times it by two um so two times two is four notice that seven plus four is 11 that's what's going to cancel out right here uh we're going to take four times two which is eight six plus eight is 14 uh subtract 11 you're going to get a three then we're going to plus 20 right there two times 10 is 20 23 um if you take away 22 you're going to end up with a one uh five times two is 10 plus three plus three is 13 so track 11 would be two and then lastly for the second row you're going to get two times nine which is 18 plus one is 19 if you track 11 from that you get eight all right so that's how that row simplifies uh doing the next one here we're just going to subtract row three from a row we're going to subtract from row three row one so we get a minus two sets of zero uh we're going to get six minus four which is at two we're going to get seven minus 10 which is negative three which is the same thing as eight mod 11 we're then going to get one minus five which is negative four which is the same thing as seven and then eight minus nine that gives us a negative one which is the same thing as 10 mod 11 and then for this one right here uh row four minus row eight again multiply by eighth a little bit bigger than i want to do right now so we're going to do row four um we can replace negative eight of course with plus three three row one and so let's take two times three which is six notice five plus six is 11 aka zero uh you're going to take four times three which is 12 plus zero which is 12 you can subtract 11 from that get a one uh 10 times three is 30 37 if you subtract 33 from it you're left with a four four times sorry five times three is a 15 plus eight uh that's going to give us 23 reduces to one mod 11 and then the last one nine times three is 27 plus one is 28 but subtract 22 from that and that leaves us six behind so that's the first level of row reduction so then we're going to move our pivot to the uh to the 22 position right there and so now we have to get rid of the numbers below it again no scaling no interchange going on right here so if we just focus on the the replacements we got rid of the two and the one that's below this thing right here so to do that we got to get rid of the two so we're going to take row three minus two over three row one but then we think of how do you write two thirds as a fraction mod 11 well two over three you could add 11 to the top uh that's going to give you 13 over three that's not divisible by three so let's add 11 again so we get 24 over three that's an eight and so we're going to take row three minus eight times row two what that tells us in terms of l is that if we take row three minus eight times row two that means in the two three spot excuse me in the three two spot you're going to put a positive eight and then the next thing right we have to do we have to get rid of this one right here how do we get rid of the one well we have to take row four minus one over three row two but one third how we write that in mod 11 is one third we could write as 12 thirds added 11 to the numerator and that becomes a four so we're going to take row four minus four times row two whoops and so that we're therefore we're going to put a positive four right here because we're subtracting four now in terms of multiplication if you want to make life a little bit easier like we saw before eight we could replace with a three row three plus three times row two like so so we're going to take the third row times everything by three so if you take three times three it's a nine two plus nine is 11 so that should cancel out you're going to take eight plus three which is also 11 so you're going to get another zero right there two times three is six seven plus six is 13 which reduces to two mod 11 and then eight times three is 24 plus 10 is 34 which if you subtract 33 you get a one right there um four row four minus four row two i actually think four is a small enough number to do multiplication by my head simplify i'm just kind of keep things easy here um so we're going to take three times negative four which is going to be negative 12 if you add one to negative 12 you get negative 11 which will give you 11 will zero right mod 11 and then you're going to take four minus four notice that also cancels out to be zero uh you're going to take two times negative four which is negative eight plus one is a negative seven which if you add 11 to that you get plus four and then eight times negative four is negative 32 we add six to that that should give us well and if you don't like negative 33 of course or negative 32 be aware that if you added 33 to that that just gives you a one six plus one is seven so we get the reduced form right there and so then the last step to do right here is we then move our pivots to the 33 position we got to get rid of the four below it so we're going to take row four minus two times row three uh notice in this situation you have to take four divided by two and as usual integers that's a two right there no big deal whatsoever so to take to take row four minus two times row three we're going to put a plus two in the four two position right we're taking the opposite of the number we see right here now notice we have constructed our matrix l we filled in the lower diagonal the lower triangular region by using the coefficients from the row replacements now to finish this thing off we're going to oops we're going to take two times four two times two which is negative four excuse me then we're going to get negative two right there so that ends up giving us a five in that position a zero right here and so let's them record what we have if we copy down the echelon form that we've built because this matrix now is the matrix U we're going to copy that down and we copy down exactly as we have it so the first row is identical two four ten five nine the second row zero three one two eight the third row zero zero zero two one and then the last row which we just modified zero zero zero zero five so you just bring down the echelon form exactly how it is that's U and then the matrix L which we were building up here we bring it down on the left L for left there I mean it's actually for lower triangular but that's fine and so it's going to be unit lower triangular so you can see the lower triangular region right here and the coefficients are exactly what we wrote down nine one eight eight four two just like we saw right here and this gives us the this gives us the the L U factorization of a matrix so without any without any scaling or interchange you rover doosh your matrix A into echelon form that gives you your U and then keeping track of the row replacements you do put their inverses inside of the columns going one by one by one through it and that'll construct your matrix L which we see right here and we have the L U factorization