 In our previous video, we learned how to construct the LU factorization of a matrix. But now I want to talk about why you would be interested in doing something in the first place. Consider the following problem. Suppose we have a system of equations where each equation is a matrix equation, ax equals b. So we have some ax equals b1, ax equals b2, all the way down to maybe say like some ax equals bp. So what if we have a system of linear matrix equations? Each equation itself is a linear system that would have to be solved. What if we have some sequence of this, right? Now, this one, this is like a sequence of equations where you have to solve this one, solve this one, solve this one. We could solve all of these ones in isolation right here. They don't necessarily interact with each other. But be aware that the process of solving the first ones, you would row reduce a, right? You could either find like the inverse of a, or we row reduce a as a matrix would have you. That would give us b. And then we row reduce the next one to get b2. And then we do all the way through here. Now, it's going to be the exact same row operations involved in solving each and every one of them. So computationally, it might be more efficient to record the necessary row operations once and for all. And this is actually the case with our, now we've done problems like this before. Like when we did the change of basis matrix, we basically would connect b and c together. Because we have all of these problems together. So we just kind of put c as one. Or when we do row reduction to find the inverse of a matrix, the idea is we can put a lot of solutions right here. We can do that. And so you could think of this like as a augment all of the b's together. But from a computational point of view, since you have to do all the same row operations, you can simplify it using the LU factorization, for which this could be a cost savings benefit. If you have a lot of these systems you have to solve, which could very well be a possibility. So let's talk about how LU factorization can potentially be a cost savings device right here. So we have seen the convenience of solving an augmented matrix in echelon form. Or when that matrix U is actually a square matrix, we would call this the upper triangular form, right? It's an upper triangular matrix. And so when you have an echelon form or an upper triangular matrix, it turns out that back substitution saves the day. It's very easy to solve a system of equation that looks like z equals one, y, say y plus z equals zero, x plus y minus z equals two or something like that. This would be a very simple system of equation to solve, right? We know automatically z equals one. That would apply that y equals negative one. And then putting negative one minus one right there, you're gonna get x minus two. You move to the side, x equals four. Very simply, we can solve this system equation because it's in this echelon form. That's why we like echelon form so much. Back substitution saves the day. Now, what if we were sort of the other way around? What if we were upside down echelon form, this chandelier form or something? If we had like x equals four and then we have x minus y equals seven and we have x plus y plus two z equals zero. What if you had something like this? Well, the fact that it's upside down echelon form doesn't make it any harder. It would be the exact same ease, right? It has all the benefits of echelon form except you're zeroes on the wrong spot. So if your matrix was lower triangular instead of upper triangular, it would still be a very easy system to solve. So now enter the LU factorization. Suppose we have an LU factorization of A. A equals L times U, where L is a unit lower triangular matrix and U is an echelon form, which is kind of like an upper triangular matrix if it were a square. Well, if we have to solve the equation AX equals B, then what we could first do is think of the following. Well, we'll let y equal the vector U times X where X is a solution vector to the equation AX equals B. So if X is a solution vector, U times X will equal Y. And so solving the equation AX equals B is equivalent to solving the equation LUX equals B. But by matrix associativity, this would be LUX equals B, but UX is equal to Y. So what we end up seeing here is that if you wanna solve the equation AX equals B and you have an LU factorization, what we can first do is we can solve first, we've solved first the equation LI equals B, we solve for Y, and then second, we solve the equation UX equals Y. So what we can do is to solve the equation AX equals B, we can solve two linear systems, LI equals B and UX equals Y, which the benefit of doing two is that this one right here is an echelon form, this one is lower triangular, which is basically just as good as echelon form. And so we can solve two easy linear systems for the price of solving one, harder one perhaps if we have this LU factorization. So let's demonstrate this. In the previous video, we computed an LU factorization for this matrix A, which is four by five. We found L and we found U right here. You can take a look at that video to see the details of that if you haven't seen it already and be aware we were working mod 11 in this example. So let's solve the equation AX equals B when B is the vector of one, two, three, four. It's like my favorite vector. So the first thing we have to do is we're gonna solve the equation LI equals B. So we're gonna take the lower triangular matrix on the left and we're gonna augment it with B right here. We wanna solve for this vector Y. Y is not gonna be the solution to AX equals B, it's an intermediate vector. So we solve this one and I wanna go through the details of this to show you how simple this calculation's gonna be even when you're working like mod 11 right here. So because this is a lower triangular matrix, it's unit lower triangular, right? I have ones along the diagonals. In terms of solving it, we have to get rid of the numbers below. So we're gonna take row two minus nine times row one. Now as we're working mod 11 minus nine is the same thing as plus two, right? So you're gonna get a two right here and you're gonna get two right there. So this next row turns into zero one zero zero four. Then the next row, we wanna get rid of the one right here. So we're gonna take row three minus one row one. That's pretty easy. Minus one minus one. You're gonna get a zero, the eight one zero is left alone and then you get three minus one which is two. And then for the last one to get at the eight, you're gonna take row four minus eight times row one. If you want a smaller number though, you can think of it as actually plus three row one. So you add three, you add three and so we're gonna get zero four two one states the same and you get a seven. It's pretty simple. So in terms of replacement sets, a pretty nice step. Go to the next matrix right here. So now we have a one right here. We gotta get the eight and the four below it. So we're gonna take row three minus eight row two or like I said, since we're working mod 11, we can do plus three row two. So we add a three, we're gonna add a 12. So the eight plus three is 11 which will give us a zero right here. You don't have to do anything in the next two columns, the last column, you get, well, adding 12 is adding one mod 11. So you're gonna get a three, two plus one. And then to give her the four right here, we're gonna take row four minus four times row two. So you get a minus four and then you're gonna get a minus 16 which again, working mod 11, subtracting 11, excuse me, subtracting 16 is the same thing as just subtracting a five. So seven minus five is a two. There you go. And so then we have to do one more replacement here given of the two that's below the one. So we're gonna take row four minus two times row three. You get a minus two minus six. That gives you a negative four which is the same thing as seven. And so with very little effort where I'll solve this system of linear equations because it's in this lower triangular form. Notice what we've discovered here is that y equals the vector one, four, three and seven. All right, so we found this intermediate solution. We then are gonna solve the system ux equals y. So we're gonna augment the vector y we found just a moment ago, one, four, three, seven. And now we have to solve it with this echelon form matrix. This will be a little bit more difficult because our diagonal entries are not necessarily one. Here's our pivots. So we're gonna have to do some scaling as well to the row replacement. So the first thing to do is to divide the fourth row by five. So five divided by five is one, that's pretty easy. Seven divided by five is a little bit more tricky. But let's see, if we add 11 to seven, we end up with 18, that's not quite good enough. If we add 11 again, we'll get 29, not quite there. If we go one more time, we add 11 again, that's gonna give us 40. 40 divided by five is gonna be eight. So we get an eight right here. So then we have to do row replacements. We gotta get rid of all these numbers right here. So we're gonna take row one, subtract from it nine times row four. Again, for simplicity, I'm gonna add two times row four. So we're gonna get plus two, plus 16. But if you take away 11 from that, that's just really just adding five. So we're gonna end up with a zero and a six right there. So for the next one, we're gonna do row two, subtract eight, which is the same thing as adding three times row four. So we're gonna add three, we're gonna add 24. Now 24, if you reduce that, mod 11 is actually just a two. You can take 22 away from that. Four plus two is equal to six. So you're gonna get zero and six right there. And then for the next one, we're gonna just take row three, subtract row four. So you get a minus one, minus eight, from which you'll give you a negative five, which is plus six. So you get a zero and a six right there. The row replacements are very minimal right here. And then we move our pivot to the next one here in the three, where is that, the three, four position. So we are gonna get the numbers above it. Well, you can take row two minus row three, that's easy enough, minus two, minus six. Oh, I guess I should scale it first. Let's scale it. I mean, that's okay what we were doing there, but let's scale it. So let's take one half times row three. You're gonna take two divided by two, which is a one. And you're gonna take six divided by two, which is a three. All right, that's just regular integers. Now if we take row two, subtract two times row three, which is exactly what we had before, you're gonna see that you got two minus two and then six minus six. So this is just gonna be a bunch of zeros right there. To get rid of this five, we would take row one minus five times row three, minus five, and then you're gonna take three times five is 15, so negative 15. If you reduce that model 11, that's actually a negative four. So six minus four is a two. So I like to reduce whenever I can to make the arithmetic simpler. I like small numbers. And so then the next pivot position you're gonna come over here, gonna grab the three, divide this whole row by three. So one third row two. Three divided by three is a one. All these are zeros, so that's gonna just give you zero here. So we have to take one divided by four. Excuse me, one divided by three. Got a little ahead of myself there. But if you take one and you add 11 to that, that's 12 divided by three, which is, that's a four. So we put that number in right there. So to get rid of this four right here, this is our last step. We're gonna take row one, subtract four times row two, for which, because these are zeros, we don't have to worry about anything there. We have a negative four. We're gonna have a negative 16 right there, which again, as we're working, well, we'll just add it together. 10 minus 16 is a negative six, and negative six of you add 11 to that as a five. So the first row becomes two, zero, five, zero, zero, two. And then you have to divide the first row by two, which two divided by two is a one. Two divided by two is a one, that's pretty easy. Then we have to deal with this five right here. Five divided by two. Again, I cheated there and saw my answer's gonna be eight. But how do we get that? Five divided by two, let's add 11 to five that gives us a 16 divided by two. That's where the eight came from. So you see an eight right here and let's see the whole thing. So we're gonna get 108010140001003 and 00018. So pulling this apart, the general solution to this system of linear equations would look like x, where you're gonna have x one is gonna look like one minus eight x three. Then the next one's gonna look like zero minus four x three. Then the next one would be x three. Your free variable, right? Is this non-pivot column that you see right here. Then we get x four equals three and then x five. x four is three, x five is an eight. And so if you pull that apart, you're gonna have this constant vector 10038. This is a particular solution to the system. Then you're gonna get t times just the x three part. You're gonna get 00 down there. A negative eight is the same thing as three. So if you prefer a positive, you can do that. A negative four, you could write that as a seven and then you get a one right there. So this gives you your general solution as is written right here. And this is the solution, the system of equations that we solved using the LU factorization. Now, why is the LU factorization so useful here? Now, if we go through and calculate the number of flops that we did with the LU factorization flops here being our sort of fundamental complexity measurement here. If you think about like how much effort would a computer have to put into solving this problem? If you count the number of flops with respect to the LU factorization, you turned out to get, I think if I counted correctly, you had 27 flops along the way. If you took the regular method or original method where you didn't have the LU factorization, that turned out to be, I think it was like 46 flops. Just counting things like multiplication and division, these basic arithmetic operations. And so you can see that the LU factorization takes almost about half of the effort to solve the system of equation than the original one. If you think about all the replacements that you might have to do, all the flops you have to do. So LU factorization is a lot easier to solve in general. But, let's see, it's on the previous slide still. But there is a cost to computing the LU factorization. That's where we return back to this issue right here. That if you're just trying to solve one system of equation, the effort of computing LU, the LU factorization, and then solving it is gonna be more than just solving the problem itself. But the thing is when you start stacking this on, you have this sequence of linear systems you have to solve. Then it starts to become a cost opportunity that if we invest a little bit on the first problem to find the LU factorization and then solve each subsequent problem using the LU factorization, then at the end of this problem, we might have actually saved some time. So from a computational efficiency point of view, the LU factorization could help us simplify our calculations. And that's one of the reasons why we care about the LU factorization.