 So let's build on yesterday's lecture on elementary matrices by doing what is called LU decomposition. So we have a matrix A. In this instance it's a square matrix. It doesn't really have to be square, but when we are talking LU, and if we want to be pure about it and the definitions of what it is, we really are going to stick with square matrices. They don't have to be invertible. And we put that out right in front. There are certain circumstances which you won't end up with. Now I'm telling you all sorts of things without any context and that is a terrible thing to do. So first of all, what is LU decomposition? It means we're going to take a matrix and we're going to decompose it into two separate matrices. So if we multiply those two matrices they give us back A. And the L stands for lower triangular and the U stands for upper triangular. So what is a lower triangular and what is an upper triangular? Let's start off with what an upper triangular matrix is. Upper triangular. That just means that I'm going to have a matrix. I'm going to say A sub one one, row one two, row one three. Let's make it row two one, row two two, row two three, row three column one, row three column two, row three column three. It's a three by three matrix. In a three by three matrix, four by four matrix, square matrix, we have a main diagonal. And those are all these values here on the main diagonal, on this main diagonal. And an upper triangular means everything below that is a zero. So A two one is zero, A three one is zero, A three two is zero. These along the diagonal, they can be anything inclusive of zero. And the values on top can also be anything as long as the values below the main diagonal are all zero. That's going to be zero, that's going to be zero. And in a bigger matrix, more of them have got to be zero. Then it's called an upper triangular matrix. As opposed to that, we get the lower triangular matrix. It's just to find these terms so we know lower triangular matrix. And with a lower triangular matrix, we want all of these to be ones on the main diagonal, ones, and everything with different values here and everything above it has to be zero. So that's a zero, that's a zero, and that's a zero. So you see the subtle difference. Listen to it again. Watch the video again with the difference between upper triangular and lower triangular matrix. You've seen upper triangular matrices before. When we do elementary row operations, we try to get this in row echelon form where all of these values, these are some values, they are values and everything below that because that's a pivot and everything below it must be zeros. That's a pivot, everything below it must be a zero. So we've gone to a stage if you do elementary row operations where you do get to an upper triangular form. And the thing about these upper triangular matrices, if you multiply them with each other, they stay upper triangular. If you multiply lower triangular ones with each other, they stay lower triangular. There's something special about these. So why on earth would we want to decompose this? Well, it has many applications. With yesterday's elementary matrices, we talked about the fact that it can help with solving systems of linear equations specifically numerically. If you get into computer science, it can help with calculating the determinant and we'll get to those as well. So now we can decompose. There's other decompositions which are also very important and we'll get to those. They're more complex. This is the easiest form of decomposition of a matrix into two other matrices that if I multiply them in this order, remember, we're talking about associativity. We're talking here the fact that they don't commute. So it's not u times l, it's l times u. And if you do that other way around, remember, matrix multiplication does not commute. So there's your definition of upper and lower and we're going to do that decomposition. And we're going to do it, as you can imagine, if we get to that row echelon form, all we did was a series of multiplications by elementary matrices. That's exactly how we're going to find this first one we're looking for, upper, because as we carry on with Gauss elimination, we get to upper triangular form in the end. So why not do that? So what do we have to do? So first of all, this is a 3 by 3 matrix. So we are going to do something to the 3 by 3 identity matrix. And remember, that's 1, 0, 0, 0, 1, 0, 0, 0, 0, 1. So what do we have to do to this? To eliminate, there's our 1. We want to eliminate the leading one. We want to eliminate everything below it. So we've got to eliminate that. So we're going to take minus 2 times row 1 and we're going to add that to row 2. So what is minus 2? So the one that we are trying to change is row 2. So to make an elementary matrix, we only want to do, we're changing one row. So this one is going to stay the same. And this one here is going to stay the same. We're trying to change row 2. So minus 2 times row 1, so that is actually minus 2 there, 0, 0. I'm just multiplying this row out by minus 2. And I'm adding that to row 1. And remember row 1 was 0. I hope you can see here, 1, 0. So I'm going to add these two. Now negative 2 plus 0 is negative 2. 0 plus 1 is 1. 0 plus 0 is 0. You've got to make these little in-between things for yourself. So that is an elementary matrix. We'll call this elementary matrix 1. And I want to multiply this with A. So I'm going to have E sub 1 times A. Let's do that. So it's 1, 0, 0. And then the negative 2, 1, 0, 0, 0, 1. And I'm going to multiply by A, which is 1, 3, 1. And I have 2, 5, 1. And I have 2, 1, 2. And I'm going to end up with that. So remember it is E1, elementary 1, times A. Not the other way around to make that mistake. Because I'm multiplying like this, the first row stays exactly the same. Minus 2 plus 2 plus 0 is 0. Minus 6 plus 5 plus 0 is minus 1. Minus 2 plus 1 is minus 1 plus 0. That stays minus 1. And this one is going to stay exactly 2, 1, 2. Now you'll see shortcuts. Your lecturer might show you shortcuts. Your textbook might show you shortcuts where you can combine all of these, 3 into 1, but please in the beginning do it separately like this. So now we left with... Now we left with... Let's clean the board. Now we left with this matrix. So now we have E1 times A. And that is 1, 3, 1. We have a 0 minus 1 minus 1. And we have 2, 1, 2. That's what we have. So what do we have to do? We've got to get rid of this. We've got a leading one there. We've got rid of that. We've got to make this. And that again will be minus 2 times rho 1 added to rho 3. So that's going to stay the same. That's going to stay the same. We need to change rho 3. And it's minus 2 times rho 1. Remember this rho is 0, 0, 1. Let's put that very lightly in there. Minus 2 times rho 1. So that's minus 2, 0, 0. And we add that minus 2 and the 0 is 0, 0 and 0. If you just see, am I doing that right? 0 plus 0 is 0. And let's redo that. Let's redo that. So I need to do minus 2 times rho 1 plus rho 3. So this is negative 2 and a 0 and a 0. And I need to add that to 0, 0 and a 1. So that gives me minus 2. That gives me 0 and that gives me 1. There we go. There we go. So that is going to be our E2. Let's do that multiplication. Let's take E2, 0, 1, 0, 0, 0, 1, 0 and then minus 2, 0, 1. And we're going to multiply it by that. 1, 3, 1, 0, negative 1, negative 1 and 2 and 2 to get this. So that's going to stay the same. 1, 3, 1. This is going to stay the same. 0 minus 1 minus 1. And now we have negative 2, 0, 2. 0 as required. We have negative 6, 0 and 1. That's negative 5. And we have negative 2, 0, 2. That's a 0. So here we have E2.E1.A and it is now 1, 3, 1, 0, negative 1, negative 1 and 0, negative 5, 0. That's what we have now. So we go on to the next leading non-zero number and that is this one. Now you can change that. You can do an elementary row operation by getting rid of this and making it a positive 1. But remember I said upper triangular. This main diagonal can be anything as long as everything below that is 0. So leave that negative 1. It can be a million. Just leave it. That is not required for upper triangular. All we have to get rid of is this. So we're going to have 5 times, negative 5 times row 2, add it to row 3. So if that is multiplied by negative 5, becomes positive 5 and if I add that, this is going to turn into 0. So again, it's 1, 0, 0. And then row 2, which is 0, 1, 0, now needs to be multiplied by negative 5. So that's going to be a 0, a minus 5 and a 0. And I've got to add that to 0, 0, 1. 0 and 0, 0, minus 5. And 0 is minus 5. 0 and 1 is 1. So this is my E3. This is my E3. Let's do that. 1, 0, 0, 0, 1, 0. And I have 0, minus 5, and 1. And I'm going to multiply that by that product, which is 1, 3, 1, 0, negative 1, negative 1, 0, negative 5, 0. So that is 1, 3, 1. That is going to be 0, negative 1, negative 1. And now we have 0. And we are with 0, 0, 0, and 0. That's a 0. We have 0. We have plus 5. We have negative 5, which is 0. And we have 0. We have 5 and 0. So we have a 5 there. Is that right? Let me just check. Let me just check, yes. So there we have E3.E2.E1. And together with A. And that gives us this upper triangular form. 1, 3, 1, 0, negative 1, negative 1, 0, 0, 5. Upper triangular, because everything, the main diagonal, everything below that is a 0. That's a 0. That's a 0. That's a 0. So we now have the fact that we have E3 times E2 times E1 times A. Gives us, and we're going to call this what we have here, U, for upper triangular. And that is going to be an upper triangular form. So how do we get the L? Well, look at this, look at it this way. If I have E3 inverse on this side, E3, E2, E1, A. What I do on this side, I must do on that side. So it's E3 inverse U. Let's cancel these two out. Now if I do E2 inverse, E2, E1, you see where this is going. This is E2 inverse E3 inverse U. That's gone because that gives me the identity matrix. And then lastly, as you can see, I'm going to do E1 inverse. So A equals E1 inverse E2 inverse E3 inverse U. The multiplication of these we call L. And it will be lower triangular. So A equals L U. And you can do that multiplication, do it in this order. And if it was a larger, you know, you would have E4 inverse, E5 inverse, etc. And you have done your very own LUD composition of a matrix using this idea of elementary matrices. So I'm going to go to Mathematica and just redo this so you can see how it works. And if you can do it in Mathematica, you can obviously check your work that you do on paper as you exercise to be able to do this for your exam. You can check your work on Mathematica, see that you can do it right. And later on after the exams, if you do some real work, just use Mathematica. So here we are in Mathematica. Let's look at LUD composition. So there is my matrix. I want to just pause and copy all of this down because I have my matrix A that we had on the board. And then the one, two, three, all three elementary matrices I do have here. So you can pause and just copy that. I'll carry on. So shift enter that gives me A. Let's just have a look A in matrix form. And we see it is the matrix that we had on the board. There is E1 and all I want to do is E1 dot A. So that period there, full stop, that means the matrix times a matrix. And if we have a look at that, that was our first elementary matrix times A. And then if we go on and we do E2 times this product of E1 and A, that is where we were left. And then that is where we were left after E3 times E2 times E1 times A. And we have this upper triangular matrix. Everything below the main diagonal is zero. Not everything on the main diagonal has to be one. If they were all ones, that would be sometimes referred to as a strictly upper triangular matrix. But we don't need that. We only need that in this form that we have here. So let's create U. Remember U was then E3 dot E2 dot E1 dot A. Semicolon to suppress that. And L we're going to make the inverse of E1 and the inverse of E2 and the inverse of E3. There we go. We have both of those. And we can ask is A equal to L dot U and we see indeed that it is true. So Mathematica has a function called LU decomposition. It gives you back three answers. So you have to store it in three separate computer variables. So I'm going to create three. And because it's a list of three, it's got to go inside of a list. I'm going to call the first one LU. The second one P and that is in the last one. I'm going to call C just by convention. It doesn't matter really what I call it. But I'm going to say LU decomposition of A. And let's see what that looks like. So I see these three elements that I get. The first one is a 3 by 3 matrix. The second one and this third one. The first one LU, let's look at LU in matrix form. And I see it's a 3 by 3 matrix. What that is is a combination of L and U. Not the multiplication of it. Because that will just give me back A. This is not what we're looking at. I'm just looking at a combination of them. I have to extract L and U from that. The second one is this 1, 2, 3. That's the rows that we use. Remember this is not unique. You can come up with different elementary matrices to get to LU decomposition. And last is a numerical, four numerical analysis. Just the accuracy. So we're not concerned about that. L and U from this matrix form. The first thing I want to just make you aware of is this thing called the sparse array. Now a sparse matrix is a massive, if I have a massive matrix, and most of the values are 0. And yeah, and there's a value that's not 0. That'll take up a lot of space in memory. So we have these sparse arrays where we just store in memory the values that are not 1 and where they are, their own column value. And that's much smaller than storing these needless zeros. So let me just show you the sparse array. I'm going to create that. And I'm going to put two placeholders for row and column. So I with an underscore and J with an underscore to indicate that these are placeholders. And then I'm going to use this shorthand notation, forward slash semicolon. So what I want here is if J is larger than or equal to, so I'm hitting escape, larger than to get that i. So I'm saying if the column value, if the column number is larger than the row number and then I'm going to say a little arrow, remember that's minus forward slash 1, give it a 1. Everywhere else, give it a 0. And then lastly, I've just got to tell it how large I want this. Now have a look at this and I'm going to print this out in matrix form. Have a look at that. See what I did? That is if the column value, if the column number was larger than the row number, it gave it a 1 otherwise, or equal to, otherwise it gave it a 0. And you can work that out. That will give me that exactly. So I'm just going to create this that is upper triangular. It's upper triangular because everything below the main diagonal is 0. It is strictly because all are ones. And I'm going to create this u, my upper one from this is l u times this little thing that I created here. Copy that and paste that. Times I just left a space. So that's u. If I want to print out u in matrix form, there is a matrix form and that was indeed our upper triangular matrix. It's not strictly upper because it's not all ones, but it is upper triangular because everything below the main diagonal is 0's. So that's how we got our how Mathematica got its upper and it seems as if it followed the same steps of elementary matrix as we did because it got the same upper upper triangular matrix. So let's do the following. Let's do sparseray again. But this time I'm going to say when the column number is smaller than when it's smaller than the i. And let's print that out in matrix form just to show you. Then I'm going to get this, what I actually want is for all of these to be ones as well. So it's strictly lower triangular. So all I'm going to do is add to that the identity matrix. Identity that is 3 by 3 and I get this strictly lower triangular because everything above the main diagonal above it is 0's and it's strictly because on the main diagonal I only have ones. And this is how I'm going to do l. l equals l u space which means multiplication and this little thing that I've just created here. That I've just created here. And that gives me l and if I put l in matrix form you see that it was our l as well. The lower triangular matrix strictly lower because on the main diagonal it's all ones. So that's how you extract. Just make sure that when it's 4 by 4 you put a 4 there if it's 2 by 2 you put a 2 there and a 2 there and a 2 there and a 2 there and a 2 there etc. But that is how you extract from this l u that came out of the l u decomposition it's a combination of l and u and you have to extract l and u from it and this is the way that you extract l u from it. And we still get to l u decomposition with a lower and upper triangular matrix. There's one more thing I want to add to this and you would have noticed that we only dealt with two elementary row operations we actually only did one but you can multiply throughout by a constant or you can take a constant multiple of one known added to another row. What we did not do is a row interchanging rows. When we do interchange rows as an elementary matrix we are not going to end up with lower and upper. We are going to end up with two matrices that we can multiply with each other with to get back to A but they will not be upper and lower triangular. They will not be there are other words for these psychological upper and lower pseudo upper and lower there's different terms from this. So if you do an interchange of rows then your elementary matrix is going to lead you not to have upper and lower triangular matrices for LU decomposition. So you can still do it in computer science that is done for a variety of reasons but you won't end up with strictly. So stick to those multiplying by constant you usually don't have to do that because we don't have to leave it in strictly upper triangular form so that all the main diagonals are one for the upper triangular matrix but this constant multiple of one plus another one plus another row that's the row operation that is used very frequently. You would also notice the following that in the vast majority of cases I do these row operations so that I can add two rows and many textbooks and many lecturers will do the opposite of that they will leave it so that you subtract one row from another row. Exactly the same thing happens in the end you're still going to get to the same solution. Obviously your elementary matrices are going to look different. So I stick to the fact that I want to keep it simple so I do addition of two rows to give me that zero under the previous leading value. I do addition so just note that difference.