 So, the last time we looked at LU decomposition via Gaussian elimination and we outlined the procedure and I also showed you an example. And towards the end of the last class, I told you about some numerical issues that arise in the LU decomposition, namely that if there is a very small number that appears as a pivot element, then inverting that when you compute it using a finite precision machine can lead to incorrect answers and so that motivates us to look at LU decomposition with pivoting. So, pivoting is a process by which you try to stabilize the LU decomposition process and the way we do that is through the use of these permutation matrices. So, just very briefly a permutation matrix, there are many types of permutation matrices, but for the purposes of this discussion, we will discuss about permutation matrices where a pair of rows or a pair of columns are getting exchanged. So, we have an notation p and pi, p is the identity matrix with two rows, say rho i and j being permitted or exchanged and pi is the identity matrix with columns i dash and j dash being permitted or exchanged, then if you define p and pi this way, then if you consider pA for any matrix A, it exchanges the i and j th rows of A and a pi similarly exchanges the i dash and j dash columns of A. So, I mean permutation matrices have many properties including these and also, so I will just say other properties of permutation matrices. So, for example, p transpose equals p inverse. So, it is a real orthogonal matrix and the product of permutation matrices is another permutation matrix and these permutations that I discussed here are called exchange permutations, which basically exchanges exactly two rows or two columns of the matrix. It involves swapping two rows or columns of the n cross n identity matrix and such matrices have the property that p squared equals p equals identity matrix. So, basically if you exchange two columns and then you exchange them back, you get back the original matrix. So, if you apply p twice with an exchange permutation, you will get the original matrix back. So, p squared is the identity matrix or p equals p transpose because p, so this means p is its own inverse but p inverse equals p transpose, p transpose for any permutation matrix. So, p equals p transpose for such matrices. So, in other words, if I take the n cross n identity matrix and exchange any two rows or any two columns, I will still get a symmetric matrix and that matrix is its own inverse. That is the special property of exchange permutations. So, now to connect this to the Gaussian elimination based LU decomposition procedure. So, like the previous development, suppose k minus 1 stages of the Gaussian elimination have been done and we will describe how to proceed with the kth stage and from you start with k equal to 1, 2 up to n minus 1, then you get the entire LU decomposition. So, suppose k minus 1 stages of Gaussian elimination are done. That is to say we have got this matrix a k minus 1 which is equal to m k minus 1. Earlier I had m k minus 1 all the way down to m 1 times a is what I had as a k minus 1. It is a pre-multiplication by these Gauss transforms but now I will have a potential permutation that was done at the k minus 1 stage, p k minus 1 all the way up to m 1 p 1 a pi 1 and then at the second stage I may have to do another column exchange that is pi 2 up to pi k minus 1. So, these are the exchanges that I have done so far. So, I have been told you how to do these exchanges but it will become clear. As soon as I tell you what exchange we will do with the a k minus 1 matrix to get the a kth matrix. Now, this matrix by construction has the form it has a 1 1 k minus 1 at the top left and this is upper triangular and I have a 1 2 k minus 1 and then 0 here and a 2 2 k minus 1 and this is k minus 1 rows and this is n minus k plus 1 rows and similarly this is k minus 1 columns and n minus k plus 1 columns and now in order to describe what I want to do next I will consider the entries of a 2 2 k minus 1. So, let us call them it starts with index k a k k of k minus 1 a k n because this is the first row of a 2 2 k minus 1 is the kth row of this matrix a k minus 1. So, this k minus 1 and a n k of k minus 1 a n n of k minus 1. So, this is what we will call the entries of this matrix. So, basically when I apply mk what should happen is this entry will remain as it is and everything else below this will become 0 but we want to do this in a stable way meaning that among all these entries here we want to get the largest magnitude entry and place it as the top left entry because we are going to be dividing all these entries by that entry and then doing row operations to make zeros appear in the below the the below the diagonal below this entry. So, so to to annihilate first column below the first row of a 2 2 k minus 1 in a stable way we call it p tilde k and pi tilde k of size n minus k plus 1 cross n minus k plus 1. So, basically the point is that suppose some entry over here is the largest entry then what you have to do is you have to exchange these two rows so that this entry goes up here and then you should exchange these two columns so that that entry ends up over here right. So, if I do one row exchange and one column exchange I can take whatever entry I find to be the biggest and make it appear as the top left entry. So, you do this such that the top left entry of pk tilde a 2 2 k minus 1 pi k tilde has the largest absolute value among all elements of a 2 2 k minus 1 then we find mk as before okay. So, we saw the last time that we will find mk by it will be a matrix with once along the diagonal zeros above the diagonal and minus lk plus 1 comma k up to minus ln k and below the main diagonal of the kth column with lk chosen to be equal to aik of k minus 1 divided by akk of k minus 1. So, all this we discussed the previous time. So, then we will have ak is equal to mk pk ak minus 1 times pi k which will be of the form a11k which is now k cross k a12 and upper triangular 0 and a22k and of course pk is related to this pk tilde just by slapping on an identity matrix of size k minus 1 like this pk is equal to i k minus 1 0 0 p tilde k and similarly pi k is i k minus 1 0 0 pi tilde k this just ensures that the first k minus 1 rows and columns of ak minus 1 remain untouched okay. So, keep in mind that the exchanges are happening over entries columns and rows of ak so this when I multiply this pk times ak minus 1 it is going to exchange rows of ak minus 1 but it will exchange rows starting from the kth row up to the nth row it will not touch the first k minus 1 rows of ak minus 1 and similarly right multiplying by pi k will exchange columns starting from the kth column to the nth column it will not touch the first k minus 1 columns of ak minus 1 okay. So, we will come back to that point later so I will put a star over here so this is this is basically fine. So, we can execute this and at the end what will happen is that since ak is of this upper triangular form if I do n minus 1 steps of this kind of thing I will get an upper triangular matrix out here so but then I need to still show that this matrix which is the pre-multiplying matrix is of the form l which is a lower triangular matrix so that when I do when I pre-multiply by l inverse then I get a is equal to l u where l is a lower triangular matrix and this u that I have over here is an upper triangular matrix so we still need to discuss how that matrix will end up becoming lower triangular. So, in the kth stage what we have is ak is equal to I am just copying from here mk pk but ak minus 1 itself is mk minus 1 pk minus 1 ak minus 2 okay which is mk minus 2 pk minus 2 etc. So, this keeps going so I have mk pk mk minus 1 pk minus 1 all the way down to m1 p1 a and then there is a pi k but when I substitute for ak minus 1 I would have got mk minus 1 pk minus 1 all the way down to m1 p1 a pi 1 pi 2 all those things will follow so this pi 1 pi 2 up to pi k so this is the structure of ak this is how it is obtained now because these pi's are exchange matrices we have that pi squared is equal to the identity matrix so using this we have ak is equal to I will still keep mk mk pk and then mk minus 1 what I will do is before I write pk minus 1 I will write pk pk pk minus 1 so pk pk is the identity matrix then I have mk minus 2 and so on all the way down to so okay maybe just to illustrate this thing I will just write one more term here so this will be clear so there is pk pk times so this is the identity matrix okay so this is the identity matrix pk minus 1 and then I have mk minus 1 and then instead of writing m pk minus 2 I will write pk minus 1 pk then I will multiply again by pk pk minus 1 and then I will write pk minus 2 and so on okay so what I am doing here is notice that this has the structure pk minus 1 mk minus 1 pk minus 1 and then I have pk sorry if I combine this together as well then I get this form pk pk minus 1 mk minus 1 pk minus 1 pk here I have mk pk minus 1 pk and then the next term will be pk minus pk pk minus 1 pk minus 2 mk minus 3 and then there will be a pk minus 2 pk minus 1 pk so all of these things becomes like a symmetric product and so this will go all the way down to just before the last term I will have like a p2 m1 then instead of writing m1 p1a I will write it as p2 p3 up to pk times pk pk minus 1 p2 p1a okay and then I still have my pi 1 pi 2 up to pi k so I will just rewrite this so that it is clear how I am splitting this product so I will write this as mk times these three times together pk mk minus 1 pk times pk pk minus 1 mk minus 2 pk minus 1 pk times just one more term for the sake of completeness pk pk minus 1 pk minus 2 mk minus 3 pk minus 2 pk minus 1 pk and so on and down all the way down to the last term will be p2 m1 times p2 p3 up to pk times pk pk minus 1 down to p1a and I still have pi 1 pi 2 pi k okay and so now what I will do is I will let mi dash be equal to sir yeah sir in the first line that you wrote ak after using pi square is equal to i and it means repeat how pk times pk is coming because at that place pk minus 1 is there so it may be pk minus 1 times pk minus 1 that is no see I can insert a pk times pk wherever I want it's the identity matrix so I'm inserting it here okay got it okay so pk pk minus 1 up to pi plus 1 times mi pi plus 1 up to pk so I'll define this to be mi dash then after n minus 1 stages what we have is that a n minus 1 which is an upper triangle matrix is equal to m n minus 1 dash m n minus 2 dash all the way down to m 1 dash times so I have all this together till here that will give me m1 dash and this product which will be pn minus 1 down to p1 I'll call that p and then this is a and then this product pi 1 through pi n minus 1 I'll call that pi and this is an upper triangular matrix u p is equal to pn minus 1 pn minus 2 all the way down to p1 and pi equals pi 1 yes sir in a n minus 1 shouldn't an a n minus 1 be multiplied because in ak there was mk being multiplied can you say that again what is the question okay in ak sir the term is mk pk mk minus 1 pk so in a n minus 1 there should be an a m n minus 1 there right there will be an m n minus 1 correct so you are right it will be m n minus 1 and that's why m n minus 1 dash is defined to be so okay I think you can I understand your confusion so if I want so in the so notice that the first so this thing is true for i being less than k okay at i equals k these matrices won't be there I can't go to the so if I take oops yeah so let's let's maybe clarify this point oops okay so if I take if I take k equal to n minus 1 because I want a n minus 1 okay so then I'll write over here k equal to n minus 1 then I have mi dash is equal to p n minus 1 all the way up to pi plus 1 times mi times pi plus 1 all the way up to p n minus 1 okay and so notice that if I take i equal to n minus 1 then I will get m n minus 1 dash and I have m n minus 1 here but this should be p n minus 1 plus 1 which is p n but there is no matrix p n that we are using in this process so these matrices will not be there for the m n minus 1 dash so m n minus 1 dash is actually equal to m n minus 1 there is no matrices multiplying these but if I take m n minus 2 dash that is going to be equal to p n minus 1 m n minus 2 p m n minus 1 and so on does that clarify your question yes sir thank you sir okay so basically what we have then is so we have this product of all these matrices okay that times p a pi is an upper triangle matrix so for a moment let us assume m n minus 1 dash n minus 2 dash m 1 dash is equal to l inverse and which is a lower triangular matrix okay just for a moment imagine that this is true I will show you why this is true in a minute but assume that this is lower triangular then what we have is if I multiply by l inverse or if I multiply by l on both sides then I will have l times u is equal to p times a times pi so it is an l u decomposition not of a but it is the l u decomposition the product of l and u gives you a row and column permuted version of a okay but I still need to show that this product is going to be lower triangular now the point is that if each of these matrices were lower triangular then of course their product would all in fact they are all they are all matrices which we defined the original mks that we defined had were unit lower triangular so we will end up showing that these matrices are also unit lower triangular so it is sufficient show that mi dash is unit lower triangular okay and the but then this is simple because these permutation matrices like I mentioned at that star the permutation matrices only touch the rows and columns corresponding to so in the k minus 1th stage they will only exchange rows and columns corresponding to the a 22 k part that is they are not touching the top k minus 1 cross k minus 1 entry so in other so for example if you take the the original mi if you remember the original mi was of this form was of this form where I had once along the diagonal but in the k so mi right so i th column so in the i th column I had a 1 here and then I had some let me just go back here in my notes um yeah it was minus l k plus 1 or li plus 1 i like that right i plus 1 comma it's too difficult to write so I'll just say here there's some star here and some star here whatever these entries were non-zero only below this thing everywhere else it was zeros but only these entries were non-zero okay and now what I'm doing in the kth stage is that I am applying an exchange matrix which is exchanging rows and columns corresponding to this part of the matrix so if I exchange rows and columns corresponding to this part then um let's see so what happens is that um pre matrix so let's see so let me put it this way suppose I take this matrix with ones along the diagonal and then non-zero is only here okay so okay maybe it's even easier if I take a slightly more concrete thing so let me write it as 1 a b c and then zero one zero zero zero zero one zero and zero zero zero one and suppose I had exchanged some two rows let's say these two rows and then um say so I need to bring the largest element to the top left so what yeah so it will be it won't be like this so I have to look at a 22 of k minus one and I'll be trying to bring the largest element over here so the exchange will be of the form where I would exchange maybe these two rows and then um I have to exchange some pair of columns but of course I'm trying to bring it bring the largest entry over here so maybe it is an exchange of these two columns now if I exchange these two rows what will happen is I'll get b okay let's write that b zero one zero out here and then a one zero zero here and then one zero zero zero here and c zero zero one here and now let's say I exchange these two uh columns um then I'll end up with yeah so I'll I'll I'll put this here one if I exchange these two columns I'll end up with no this is not how you go one second this is not correct let me let me write it in a different way so the p matrix it exchanges um rows of mi with okay I think I understood um the point is that mi dash is yeah so yeah okay my example was okay what I was doing was not correct so maybe okay we'll come back to that um with rows greater than i and okay so basically let me let me put it like this suppose I start with this matrix zero zero zero a one zero zero b zero one zero and c zero zero one then remember that the mi dash is obtained by doing p a p mi times pi so if you go back here notice that the the core structure is something like this pi plus one mi pi plus one that kind of thing okay and pi plus one is going to exchange rows in mi corresponding to um uh some pair of rows which are greater than i so basically what you do is here uh in this structure what it'll do is you pick two rows um the it's the same matrix pi and pi plus one that's going on the left and right so suppose you exchange these two columns then you will also be exchanging the first and third column you will be exchanging the first and third row if I do this let's see what happens when I exchange the first and third row I'll get b zero one zero a one zero zero and one zero zero zero and c zero zero one here and now if I exchange these two columns then this column appears here I'll have so I still have the same just a second what am I doing wrong here I have to correct myself again anyway let's see so um the exchange is going to be among rows of mi with indices greater than i so this this part is not going to get touched it's going to be a pair of rows and columns among these so if I take let's say these two rows and I exchange them then let's see what I get I'll get a matrix one zero zero zero c zero zero one and then b zero one zero and a one zero zero and now I have to exchange the same columns because it's pi plus one mi pi plus one so I have to exchange the same second and fourth column of this matrix if I exchange these two then what I will get is the matrix um one c b a zero one zero zero zero zero one zero and zero zero zero one exactly so this is the idea so all that happened by this pi i plus pi plus one mi times pi is that the entries a and c got exchanged otherwise the structure is exactly retained so that is the basic idea here so for example suppose pi plus one exchanges rows l m which are both greater than i then pi plus one mi times pi plus one is the same as mi but with elements l k l k l so this is a bad notation because I had written l ij to be the entries of this matrix mi but anyway it's l k l so let me say this is l dash then k l dash and l k m exchanged or in other words this matrix pi plus one mi pi plus one is unit lower triangular and that means that all mi dash are unit lower triangular and this means that l inverse is unit lower triangular which implies that l is also unit lower triangular okay so that completes this description to say that this this process even with pivoting gives you a an l u d composition not of a but of p a pi where p is a permutation p and pi are permutation matrices