 Today's lecture we will discuss the proof of the fact that any rectangular matrix can be made row equivalent to a row reduced echelon matrix. This is one of the important results in linear equations. So we will prove this result and look at other consequences. Let me first recall the definition of a row reduced echelon matrix. So let us say R a rectangular matrix with real entries m rows and n columns is called row reduced echelon matrix is called a row reduced echelon matrix if it satisfies the following conditions. The first condition is that the leading non-zero entry of each non-zero row of R, the leading non-zero entry of each non-zero row of R is 1 that is the first non-zero entry in each non-zero row of R is 1 this is the first condition. Second condition look at each column, at each column corresponding to the leading nonzero entry of a nonzero row all the other entries along this column must be 0 this is the second condition. So the statement is each column corresponding to the leading nonzero corresponding to the leading nonzero entry which is 1 okay so look at each column corresponding to the leading nonzero entry of each nonzero row of R has all its other entries 0 has all its other entries 0 this is the second condition okay so look at the columns which have the leading nonzero entry of a nonzero row of R the entry all the other entries except this leading nonzero entry must be 0 this is second condition we also have 3 more conditions so let me go to the other conditions. Each 0 row of R occurs below every nonzero row of R and I said that this is like stacking the 0 rows of R at the bottom condition 4 if i equals 1, 2, 3, etc R are the nonzero rows of R and if the leading nonzero entry occurs in column leading nonzero entry I think I need to make a slight change the leading nonzero entry of row I okay that occurs in column C I I equals 1, 2, 3, etc R then we require that C 1 less than C 2 etc less than C R so these are the conditions that a row reduced echelon matrix must satisfy okay okay this we had seen yesterday what I would like to do today is to discuss how a matrix how every matrix can be reduced to a row reduced echelon matrix see this is important how every matrix can be reduced to a row reduced echelon matrix that is what I am going to discuss today and also discuss some of the properties of row reduced echelon matrix okay but before proceeding let us look at 1 or 2 examples I have given a few yesterday let me give you a couple of examples today identity matrix of any order okay of any order this is a row reduced echelon matrix trivially let me give you one more example which we will also use a little later the matrix I will call it R R will here after stand for row reduced echelon matrix let us say I have something like this 3 rows 5 columns this is another example of a row reduced echelon matrix okay there is also another fact which we will need a little later you think over this question I will post this as a problem suppose that R is a square row reduced echelon matrix with property that the number of nonzero rows okay the property that the number of nonzero rows R equals the order of the matrix I have a square row reduced echelon matrix with the property that the number of nonzero rows is equal to the order of the matrix okay what can you say about R just think it over okay we will come back and answer this question a little later okay okay as I mentioned it is important to note that every matrix can be reduced to a row reduced echelon matrix okay let me prove that now okay so what I will discuss next is the following theorem and its proof let me just change this I will simply say every is row equal now that is a statement of a theorem so I would like to put it this way any matrix is row equivalent to a row reduced echelon matrix okay by the way the reason why we need to reduce a matrix to a row reduced echelon form is that row reduced echelon form is a much simpler matrix to handle than the original matrix A okay you will see that little later if you have not already seen it remember that we are looking at the system A X equal to B that system 1 we have reduced it to system of the form C X equal to D and we are looking at a particular structure for C okay it is really this structure that we are seeking we are seeking C to be a row reduced echelon matrix so we must know that every matrix can be reduced to a row reduced echelon matrix that is what we are going to prove now okay so proof let me write down the matrix A with its entries A i j as follows A 1 1 A 1 2 A 1 3 etc A 1 n m rows n columns A 2 1 A 2 2 A 2 3 etc A m 1 A m 2 etc A m n okay I want to do elementary row operations on this matrix and reduce it to a row reduced echelon matrix okay we will handle this matrix row by row let us look at the first row if all the entries of the first row are 0 then I will go to the second row if all the entries of the second row are 0 I will go to the third row and so on okay so one possibility is that A is 0 that is trivially a row reduced echelon matrix okay okay suppose A is not 0 and suppose the first row is not the 0 row okay there is some non-zero entry let us take the first one the first non-zero entry when you go from left to right the first non-zero entry when you go from left to right in the first row so I will call that as A 1 k let A 1 k be non-zero and what is the property of k? k is the least k being the least among the non-zero entries in the first row among the non-zero entries I am looking at A i k A 1 k k is the least A 1 k is not 0 okay which means to the left of A 1 k all entries are 0 okay so how does the matrix A look like with this information the matrix A looks as follows the first k minus 1 entries are 0 this entry is non-zero that is A 1 k the other entries are there as it is the other rows I would not disturb them now so they are left as they are I know that A 1 k is not 0 so I will do this first elementary row operation see what I want is that the leading non-zero entry of each row is 1 I know that this A 1 k is not 0 so I will do this first operation obviously look at 1 by A 1 k of the first row okay that is the first operation upon this operation this is an elementary row operation okay then A is row equivalent to this matrix all the entries to the left of A 1 k okay they are all 0 the entry for A 1 k that has been replaced by 1 all other entries I will use a star to denote them the other entries are left as they are right now okay okay that is the first step next step remember the condition that must be satisfied by a row reduced echelon matrix the second condition each column that has a leading non-zero entry has corresponding to some row has all other entries 0 which means I must make these entries 0 I must make the entries of the kth column of A below this 1 to be 0 but I know how to do it so that is that tells me what the next set of operations must be look at row 2 the new row 2 will be see this entry is A 2 k okay etc this entry is A m k entries in the kth column so I do these operations simultaneously row 2 has been replaced by minus A 2 k row 1 plus row 2 etc row m has been replaced by minus A m k row 1 plus row n okay do this one after the other then the matrix A reduces to the following form A is row equivalent to this matrix 0 0 etc 0 this entry A 1 k originally that was 1 leave the other entries of the first row as they are now these entries I do not know what they are okay this particular entry is 0 that much I know these entries also they may be 0 they may be non-zero but right now I am not concerned let me fill up the first the column corresponding to k that is the kth column of the original matrix A now if you look at the entries in this kth column they are 1 0 0 0 okay the first entry is 1 all the other entries are 0 and again these entries may be 0 non-zero I am not concerned presently okay so the first two conditions of a row reduced to echelon matrix are satisfied for the first row at this stage okay the first the leading answer entry of the first row is 1 each column containing the first column the column that contains the leading non-zero entry of the first row that is the kth column has all its other entries 0 okay now we are in two cases when I look at the second row let me write down the matrix if the if all the entries of the second row are 0 I do nothing to the second row okay presently if all the entries of the second row are 0 I do nothing to the second row I move to the third row if all the entries of the third row are 0 I do nothing I proceed like that if there is some entry of the second row which is non-zero then there is one thing for sure this cannot occur in the kth column if there is a non-zero entry in the second row that non-zero entry cannot occur in the kth column because kth column has been taken care of let me use this to denote the kth column kth column all the entries except the first row entry are 0 so the non-zero entry in the second row cannot be in the kth column okay so it is either to the left of the kth column or to the right of the kth column I will consider these two cases separately okay so let me introduce two arrows to denote the two possibilities one happens somewhere here okay this corresponds to the left of column k okay I will call that p column p this happens in column p less than k okay this is one possibility the other possibility is to the right that is the leading non-zero entry of the second row that is what I am looking at that either happens to the left of the kth column or to the right of the kth column it cannot be the kth column that is clear now so let us say this entry corresponds to the leading non-zero entry so I will call it to the right of column k I will call that q column q greater than k p or q cannot be equal to k that is clear so let us take these two cases separately and then see how this can be handled let us take the case on the right okay so let me write down this matrix once again A is row equivalent to this matrix the other entries I leave them as they are I know that these are 0 below 1 so this is the case we are in this is non-zero okay that is this corresponds to A2 I am calling that as q A2 q is non-zero okay leading non-zero entry q is the least among the second subscripts leading non-zero entry of row 2 q is the least among the second subscripts q is the least among the second subscripts occurring in the second row second row is we started with A21 A22 etc A2 and they have they have not changed except A2 k okay so among these I take the one which is the first non-zero entry so I am call that is A2 q I know that is not 0 so what is the next operation I will divide this second row by this number so the second row is replaced by 1 by A2 q times row 2 okay so that this entry becomes 1 remember I am looking at the leading non-zero entry of the second row I am looking at the leading non-zero entry of the second row okay so this gives me the following matrix 0 0 etc 1 the entries along this column are 0 except the first one these remain the same the second row now changes I do not okay the second row changes I know for certain that these entries are 0 this is also 0 let me use the same picture here this entry is 1 all the entries other entries I am not concerned but I know for sure that these entries to the left of 1 are 0 because of the definition of A2 q A2 q is the first non-zero entry in the second row so first non-zero entry in second row which means all the other entries before that entry in the second row are 0 okay then the rest of them are kept as they are presently then we do we need to do elementary row operations now using the second row okay we need to do elementary row operations using the second row so what are these operations okay that is clear we need to change the first row also because I must make this entry 0 this entry must also be made 0 okay only then the condition of the second condition corresponding to row reduced echelon matrix will be satisfied corresponding to this leading non-zero entry of the second row okay so I do these operations row 1 is see this entry is 1 and see this entry is as it is the original entry so this is A1 q right so I look at 1 I am sorry I look at minus A1 q times row 2 plus row 1 I do a similar thing for all the other rows row 2 remains the same row 3 is minus this entry A3 q row 3 plus row 1 etc row m minus A m q row m plus row 1 yes A1 q divided by A1 q yes you are right A1 q divided by A1 k yes this is A1 q divided by A1 k alright yes we are doing the first operation is yes that is correct first row has been replaced by the row divided by A1 k the first non-zero entry of yes that is correct so this is A1 q by A1 k that is not 0 I want to make that entry is 0 so I do these operations okay I do these operations then A reduces to the following matrix all these entries are 0 this entry is 0 now okay corresponding to this so let us say I put a star and then put a 0 leave the other entries I am not concerned about the other entries rather then all these entries are 0 one occurs here this must be 0 other entries I am not concerned what I also know is that these entries now become 0 so first this column next this column not concerned about the other rows presently yes this is the pivot row is the second row yes this must be R3 this must be Rm yes the pivot rows are 2 I am keeping R2 as the row with respect to which I do the other with respect to which I change the entries of the matrix yes so this is what happens when the leading non-zero entry of the second row is to the right of column k that is it happens in column q q is greater than k okay I need to look at the other case and then we will proceed the by induction what is the other case let me start all over from here the second case corresponds to the leading non-zero entry being present here okay this is A2p okay that is A2p according to my notation p is not k and p is the least p is the least among the second subscripts of the second row entries of the second row p is the least among the second subscripts of the entries of the second row so now it is to the left of k so I again do the same operation a similar operation row 2 has been replaced will be replaced now by 1 by A2p row 2 okay that will give me a 1 that will give me a 1 here so A is equivalent to okay this is column k these entries I am not I will not be concerned now this is the first non-zero entry so these two entries are 0 I have 1 here these are certain other entries these are not changed is that okay so now I do now this is to the left of the kth column so I will not disturb the first row unlike what I did earlier earlier it was to the right of the kth column and so I need to change this particular entry to make it 0 okay but now it is to the left of this column the one the leading non-zero entry of the second row is to the left of the column k and so these entries I know are already 0 so I do not touch row 1 now okay I do the elementary row operations corresponding to row 3 etc row m using this second row to get the following matrix okay so I am looking at row 2 has been row 1 has been fixed I do not change that now row 2 has been fixed row 3 etc row 3 for instance I must look at this entry that is minus A3q I am sorry A3p minus A3p times row 2 plus row 3 that is row 3 etc row m is minus Amp row 2 row 2 has been used as so called pivot row Amp row 2 plus row m to get the following matrix as a row reduced rather the following matrix as a row equivalent matrix row equivalent to A first row entries they are 0 1 here these entries are 0 first row entries are not disturbed I have a 1 occurring here 0 0 1 this entry is 0 this is already 0 let us note that this is already 0 these entries have been made 0 now because of these operations these entries have been made 0 so this is now corresponding this corresponding to the second row the leading non-zero entry appears in column P and I observe that the second condition is satisfied all the other entries are 0 okay now what is to be observed is the following when the leading non-zero entry occurs to the right of the column K okay what we have observed is that the entries corresponding to the first row before the leading non-zero entry they are not changed any case we are is that okay if it if the leading non-zero entry of the second row occurs to the right of column K or to the left of column K the entries of the first row before one including one those entries are not changed nor are the entries corresponding to the Kth column under the entry 1 they are not changed in any case okay so you are to the right of K or to the left of K these entries do not change so by induction one can show that these entries do not change when you do these elementary row operations okay we proceed by induction go to the third row the third row if it is full of zeros all zeros then go to the fourth row etc if third row has a leading non-zero entry then do the same operation again and since there are N operation since there are M rows this procedure stops after you are done with the last row M third row okay so this is a finite procedure in the last step you get a row reduced echelon matrix so I will simply say by induction A is a row equivalent to a row reduced echelon matrix yes I have not mentioned that but those okay then the proof is not complete let me just include the following I will tell you orally it is still not row equivalent to a row reduced echelon matrix I need to look at these C1 C2 etc right and I also need to stack them in such a way that the zero rows are at the bottom okay but that can be done by row interchanges okay so that does not need much explanation at all that is what is the third condition third condition is every zero row must happen must occur below every non-zero row that can be done by row interchanges okay what is the last condition I have the first R non-zero rows I have the first R non-zero rows and I have the leading non-zero entries occurring in column C1 C2 etc C R these must be arranged in such a way that C1 less than C2 etc less than C R okay but this can again be done by row interchanges this can again be done by row interchanges and so I get at the in the final step I get the row reduced echelon matrix R after doing these row interchanges if necessary okay so let us let us consolidate by looking at a numerical example I will take the row reduced echelon matrix that I had given at the beginning of the lecture I want to look at all the solutions of the system R X equal to 0 where R is given by this matrix please tell me if entries are all right 0 1 0 4 this entries 2 what is this entry 4 0 half 0 0 0 1 5 and the last row is 0 this is a row reduced echelon matrix the leading non-zero entry of each row is 1 each column containing the leading non-zero entry of any row has all its other entries 0 non-zero rows are stacked above the 0 rows so this is a row reduced echelon matrix I want to look at the set of all solutions of the system R X equal to 0 homogeneous presently R X equal to 0 is the same as if you expand you get the following equation this is X 2 X 2 plus 4 X 3 plus 1 by 2 X 5 that is 0 first equation second equation is this is X 4 X 4 plus 5 X 5 is 0 the last row is 0 so it does not impose any condition on the unknowns X 1 X 2 etc X 5 okay so what I do now is what is clear is that X 1 is arbitrary X 1 is arbitrary X 5 can also be chosen to be arbitrary all the other unknowns can be determined there are 5 variables essentially 2 equations I need to fix 3 so X 1 is arbitrary I will call it alpha X 5 is arbitrary but before that I will also observe that X 3 or X 3 can be chosen to be arbitrary either X 3 or X 2 let us say I choose X 2 I will call that beta X 2 is also arbitrary finally X 5 is arbitrary X 5 can be chosen as any real number so I will call that gamma then with these choices for X 1 X 2 X 5 I can determine X 2 and X 4 uniquely for this choice I can determine X 2 and X 4 uniquely so X 2 turns out to be what minus 1 by 2 X 5 minus 1 by 2 gamma minus 4 X 3 yes X 2 is beta I want X 3 X 3 is 1 by 4 times okay let me write this okay I am looking at the first equation 1 by 4 times minus 1 by 2 gamma that is this term minus X 2 minus beta that is X 3 for me okay please check the calculations and X 4 X 4 is minus 5 X 5 minus 5 gamma okay you substitute X 1 X 2 X 3 X 4 X 5 into the 2 equations verify that they are satisfied so I will write down the set of all solutions of the system R X equal to 0 in this example I will call that S it is the set of all X 1 X 2 X 3 X 4 X 5 X 1 is alpha X 2 is beta X 3 is 1 by 4 so I will write it as it okay I can take minus minus 1 by 4 minus 1 by 4 minus 1 sorry half gamma plus beta that is a third component the fourth component is X 4 minus 5 gamma the fifth component has been fixed to be gamma alpha beta gamma are arbitrary real numbers so this tells us immediately that there are infinitely many solutions I can rewrite it but I leave it as this I must close this this is the set of all solutions okay for this homogeneous system okay let us now look at some of the other properties of a row reduced echelon matrix okay but before that using this numerical example as a motivating example let us also look at the general case okay which is R is a row reduced echelon matrix I want to look at the system R X equal to 0 okay to formalize what I have done for this numerical example R is a row reduced echelon matrix I want to look at the set of all solutions of R X equal to 0 okay let us the original variables are the original unknowns are X 1 X 2 etc X n I need to relabel them okay let us say that I equal to 1 2 3 etc R are the non-zero rows of R R is a row reduced echelon matrix the non-zero rows are at the top I am assuming that they are R in number I also have these numbers C 1 C 2 C 3 etc okay etc C R are the columns corresponding to the leading non-zero entries of the rows 1 2 3 etc R okay I am just confirming that this is a notation we have adopted we also know that C 1 less than C 2 etc less than C R let me now relabel the unknowns as those that correspond to the non-zero rows and those that do not relabel the unknowns as a rows that correspond to the non-zero rows those that correspond to the non-zero rows I will denote them naturally by X C 1 X C 2 etc X C R I am relabeling the unknowns that we started with X 1 etc X n X C 1 X C 2 etc X C R naturally correspond to the unknowns with respect to the non-zero first R non-zero rows of R capital R the remaining will be named the remaining variables or unknowns will be denoted by V 1 V 2 etc V n minus R the remaining variables will be denoted V 1 V 2 etc V n minus R so there are R corresponding to the non-zero rows and there are n minus R that correspond to the other rows say I am looking at the number of variables then can you see immediately that R X equal to 0 is precisely the following let me write that here R X equal to 0 written in full gives me the following X C 1 plus summation J equals 1 to n minus R let me say I have something like alpha 1 J V J equal to 0 that is the first row second row X C 2 summation J equals 1 to n minus R second row so I will call them alpha 2 J V J equal to 0 etc X C R plus summation J equal to 1 to n minus R alpha m J there are m equations this is R J I am sorry alpha R J V J equal to 0 these are the equations corresponding to the first non-zero first R non-zero rows of capital R the other equation do not appear they do not impose any condition on the unknowns because they correspond to 0 rows so what this says is that you can fix these variables assigning arbitrary values to V 1 V 2 etc V n minus R the values of X C 1 X C 2 etc X C R can be obtained you assign arbitrary values to the so called free variables these are called the free variables V 1 V 2 etc V n minus R n minus R of them you can determine the other R variables X C 1 X C 2 etc X C R using these equations by assigning arbitrary values to these n minus R variables okay that is what we have done in the previous problem in particular we have the following in particular if R is less than n in particular of the number of non-zero rows of capital R is less than the number of unknowns okay what this means is that there is at least one of these variables V 1 etc V n minus R okay n minus R is positive that means I have at least V 1 at least one free variable okay let me say I assign one of them the value 1 in particular of R is less than 1 by assigning one of V 1 V 2 etc V n minus R the value 1 I pick any of these n minus R variables I pick one of them give the value 1 to that variable and I can determine assign arbitrary values to the other variables and arbitrary values to the other n minus R minus 1 variables by doing this we get a non-zero it is called a non-trivial solution we get a non-zero solution for the system R X equal to 0 okay this is an important observation if the number of non-zero rows of the row reduced echolon matrix capital R is strictly less than the number of unknowns appearing in the problem then the homogeneous system R X equal to 0 has at least one non-zero solution do you agree okay I have given the proof here pick one of the free variables assign the value 1 you know that there is at least one free variable because R is less than n okay now this leads to a fundamental theorem that if you have a rectangular system of homogeneous equations with the number of equations being strictly less than the number of unknowns then it always has a non-trivial solution okay that I will prove in the next class and also discuss other properties of row reduced echolon matrices