 See the next objective is to introduce a systematic scheme by which we can formalize elementary row operations, okay towards that I will discuss the notion of elementary matrices and some of their properties. You will see that in the next lecture this will culminate in a complete theoretical understanding of the solutions of a system of linear equations homogenous or otherwise. But before that I will make a quick review of matrix multiplication. I suppose all of you know what matrix multiplication is, how it is to be done. I will make a very quick review and then move on to the topic of today's discussion, okay. Discussing elementary the so called elementary matrices and where do they come in this discussion on elementary row operations, okay. So matrix multiplication is what I would like to discuss first very quickly. Let us say I have two real matrices A and B. I have two matrices let me give these orders like this A is R m cross n, B is R l cross p, okay these are given to us then the product is defined. The product A B is defined only if n equal to l, okay and in this case if you call the product as C, C equal to A B this matrix is of order m cross p, okay this all of us are aware of I am sure. This is a rather peculiar way of multiplying two objects, okay. What I would like to do is to write down the formula for the ijth entry of the product that is what we will need today. If C ij is ijth entry of C then what is the formula for C ij? Is there a volunteer formula for C ij? See I am asking you not the determinant, I am asking you the ijth entry of the product. Summation K equals 1 to what? Okay this is how the product is defined that is if you write down the matrices fully then you will know that the ijth entry of the product is the product dot product of two vectors, dot product of the ith row of A and the jth column of B, dot product of the ith row of A and the jth column of B that is C ij, okay. Now this is you can take this as a definition of the product then what can be verified I am going to leave this as an exercise for you. What can be verified is that the product is associative whenever it is defined, okay that is if AB and BC are defined then AB into C is A into BC, product is associative, matrix product is associative, it is also distributive over addition that is if you have A into B plus C if this is defined then this will be equal to AB plus EC, okay product is distributive over addition. The product is peculiar I mentioned let me give you one or two instances of it is peculiarity, it is quite possible that the product AB is defined but BA is not defined, you can construct simple examples show that the product AB is defined but BA is not defined even if AB and BA are defined they could be of different orders, they could be of different orders for example you take A2 by 3, B3 by 2 then AB is 2 by 2, BA is 3 by 3 they could be of different orders even if the orders are the same AB is in general different from BA, AB is in general different from BA even if the orders are the same, okay you can give an example for each of these for each of these illustrations. Finally it could also happen that AB is the 0 matrix it could happen that the product is 0 without either of the factors being 0, okay it could happen that the product of 2 matrices is 0 without any of the factors being equal to 0, we must have studied in algebra 0 divisors, okay there are 0 divisors if you look at the binary operation as the product of matrices then there are 0 divisors that is it there are matrices A and B such that A not equal to 0, B not equal to 0 but the product AB is 0, okay so I have listed some of the peculiarities of matrix multiplication I think this is enough for me I will move on to the topic of today's discussion that is the concept of elementary matrices. Using the concept of elementary matrices I am going to further formalize the elementary row operations and then bring about a systematic scheme of analyzing systems of linear equations, okay what is an elementary matrix? It is a square matrix an M by M real matrix is called an elementary matrix if it can be obtained, okay it is a square matrix if it can be obtained from the M by M identity matrix, okay I start with the M by M identity matrix and then do a single elementary row operation if it can be obtained from the M by M identity matrix upon a single elementary row operation take the M by M identity matrix do a single elementary row operation on it the resulting matrix will be called an elementary matrix, okay. Let me list all the 2 by 2 elementary matrices okay you will see that there are only 5 possibilities see remember this is upon a single elementary row operation not a sequence, okay example the set of all the 2 by 2 elementary matrices I want to write down all these there are 5 in number, okay you will see that these are the only 5 0 1 1 0 interchange of the 2 rows of the identity matrix take the 1st row that is 1 0 alpha 0 0 1 multiplying a row by a non-zero constant, multiplying the 1st row by a non-zero constant similarly multiplying the 2nd row by a non-zero constant, okay 3 possibilities replace a row by constant times another row plus the row that I started with yes I have replaced the 1st row by 1st row plus alpha times a 2nd row a similar operation for the 2nd row the 1st row remains as it is 2nd row will be alpha 1, okay so these are all the elementary matrices of order to precisely 5 of them, okay in number, okay elementary matrices have a special property that they are invertible, okay so before I discuss this particular property of an elementary matrix let me go to the notion of inverse of a matrix, okay maybe I should do something else before that what is the effect of elementary matrix has been obtained by a single elementary row operation on the identity matrix, okay how is this related to an elementary row operation being performed on a single matrix A let me first discuss that I want to discuss the following this is the importance of introducing an elementary matrix take an elementary row operation perform this elementary row operation on the matrix A, okay I am given a matrix A and an elementary row operation I perform this elementary row operation on the matrix A the effect of this the resultant matrix which is row equivalent to A is the same as pre-multiplying the elementary matrix corresponding to this elementary operation by A, is that clear? E capital E equals small E of y means that I am doing this elementary row operation on the identity matrix of order M I get an elementary matrix I am calling that as E pre-multiply this matrix E with A then I get another matrix this matrix is the same as performing the particular elementary row operation E on the matrix A, okay so performing elementary row operations is effectively pre-multiplying the matrix A by elementary matrices this is an important observation, okay let us first prove this and then look at the consequences again out of the three operations I will discuss only one operation the other two are simple so I will discuss the case when you replace a row by a row plus a constant times another row and show that this formula holds in that case the other two you could treat them as exercises. So I need to prove this, okay is that clear what this theorem says pre-multiplying for example this could be used in a program matrix multiplication is easily done by computers so one could think of a sequence of elementary row operations as multiplying the matrix pre-multiplying the matrix A by a sequence of elementary matrices, okay. So let us see how the proof goes I will move in here for the proof, okay so I have two matrices one on the left one on the right I need to show that these two coincide I will show that the corresponding entries coincide two matrices are equal if their corresponding entries are the same I will show that so this is the claim then claim to show that EA IJ IJ centre of EA is the same as small E of A IJ centre then it would follow that these two matrices are the same, okay. For this I need the formula for the product and also what capital E is what is capital E capital E is do say for me E is for me E is the operation replace row S by row S plus alpha times row T this is my operation, okay E corresponds to this operation I will prove this result for this case the other two are similar simpler so I want to write down the matrix E I will write E ik because that is what I will need here I want the formula for E ik just tell me if this formula is correct for I not equal to S say I am doing this particular elementary row operation on the identity matrix so take the identity matrix look at the S throw replace S throw by S throw plus alpha times T throw all other entries are left as they are so if I is not equal to S it is the same as the entries of the identity matrix for the identity matrix there is a special notation the Kronecker delta notation delta IJ delta IJ this is equal to 1 if I is equal to J 0 if I is not equal to J I am looking at the entries other than the S throw so can you see that this is E ik so that S delta ik when I is not equal to S it is the same entry as the identity matrix that I started with that entry is delta ik if I is equal to S what is the formula S throw delta sk plus alpha times delta Tk is that no there is yeah this is fine E ik is this, okay so I will use this definition of E ik and then use the definition of matrix multiplication and then see what this left hand side is it will turn out to be what you have on the right hand side so let us do matrix multiplication to verify this equality so I want the IJ the entry of Ea, okay that is I am writing down the formula summation k equals 1 to m see capital E has is of order m cross m remember this must be a square matrix and in order to pre-multiply with A the order of E must be m by m A is m cross m k equal to 1 to m E ik Ak J this is the formula for the product the IJ entry of the product now since E ik has the split formula I will write like this summation k equals 1 to m for I not sorry for I equals S summation k equal to 1 to m I will have to substitute this formula delta sk plus alpha times delta Tk into Ak J if I is equal to S it is summation k equals 1 to m delta ik Ak J if I is not equal to S I just substitute the formula for capital E ik the ik entry of capital E expand and simplify this is summation k equals 1 to m delta sk Ak J plus alpha delta Tk Ak J if I is equal to S and look at this now this is delta ik Ak J k is a running index k is a running index I and J have been fixed I throw jth column entry is what I am trying to calculate I and J are fixed k is a running index k takes the value 1 to m I varies between 1 to between 1 and m so when k takes the value I this will be 1 all other entries are 0 okay so remember that k is a summation running index I and J are fixed so when k equals I so this has only one term that corresponds to k equals I when k equals I it is delta I I A I J I not equal to S let us complete the bracket here okay a similar thing we need to apply for the first part so let me give that here for the first part it is summation k equals 1 to m I will right away simplify delta sk Ak J k is a running index when k takes a value S it is 1 all the other terms are 0 corresponding to the first term so when k takes the value S it is delta SS so the first term here reduces to delta SS ASJ plus the second term when k takes a value T it is 1 all the other terms are 0 alpha times k takes a value T delta T T A T J this is when I is equal to S and delta I I is 1 so this is A I J if I is not equal to S I hope this is clear this simplifies to delta SS is 1 ASJ plus delta T T T T is 1 alpha A T J this when I is equal to S it is A I J when I is not equal to S but you see that this is precisely the definition of E of A where E is this operation the S throw is replaced by S throw plus alpha times a T throw all the other entries are the same so this is the I J th entry of E of A okay so these two matrices are the same so performing an elementary row operation on a matrix has the same effect as pre multiplying the given matrix by a particular elementary matrix this elementary matrix is obtained from the identity matrix by applying this particular elementary row operation on the identity matrix single elementary row operation okay so what does this tell you about row equivalent matrices okay that is what we will see next what we will see is the following given two matrices A and B what we would like to show is that then B is row equivalent to A you will see that this is the consequence of the previous theorem B is row equivalent to A if and only if B equals P times A where this time P is not a single matrix it is a sequence okay so this P is a product by which I mean a finite product where P is a product of elementary matrices P is a product of elementary matrices if I do a single elementary row operation and get B from A then this P will be an elementary matrix otherwise it is a product of a finite product of elementary matrices okay. Now you will see that this will be useful later also there are certain things that are preserved by elementary row operations in the case of a square matrix for instance we will look at certain numbers the determinant the rank the inverse etc one could calculate the inverse we will show by using elementary row operations we will show that certain numbers like the determinant the rank will remain the same if A and B are row equivalent okay in proving those results we will make use of this that B is a B can be written as P times A where P is a product of elementary matrices okay now there are two parts okay for this result so let us take the first one suppose B is P times A I must show that B is row equivalent to A okay suppose B is P times A where P is what I am given is that P is a product of elementary matrices so let me write like this E S E S minus 1 etc E 2 E 1 B is equal to P times A where P is a product of elementary matrices we would like to show that B is row equivalent to A okay okay. What do we have B is P times A use the formula for P E S E S minus 1 etc E 2 E 1 A so let me introduce a bracket here matrix multiplication is associative so I can write this as E S E S minus 1 etc E 2 times E 1 A matrix multiplication associative that is what I have used here now look at E 1 A E 1 A by virtue of the previous theorem is row equivalent to A because E 1 is an elementary matrix and so it is an elementary row operation being performed on the single elementary operation being performed on the identity matrix by the previous theorem E 1 A is row equivalent to A okay E 1 is row equivalent to A then the next step you push E 2 again consider B equals P times A E S E S minus 1 etc E 3 E 2 E 1 A E 2 E 1 A is row equivalent to E 1 A by the previous theorem row equivalence is an equivalence relation in particular transitive it is row equivalent to A so I have E 2 E 1 A row equivalent to A okay being row equivalent to A this is the first step this is the second step proceed by induction proceeding similarly what follows is that E S E S minus 1 etc E 2 E 1 A is row equivalent to A taking one elementary matrix at a time but this left hand side E S E S minus 1 etc that is precisely B so B is row equivalent to A if B is P times A where P is a product of elementary matrices then we have shown that B is row equivalent to A I hope it is clear we must we must prove the converse it is the first part clear conversely suppose that B is row equivalent to A then B is obtained from A by a sequence of elementary row operations B is obtained from A by a sequence of elementary row operations I will call it E 1 E 2 etc E L minus 1 E L that is I am doing first E 1 on A then E 2 on E 1 A etc that is that is the first operation E 2 on E 1 A etc E L E L minus 1 here again I am making use of the fact that every elementary row operation corresponds to pre multiplying the matrix A by an elementary matrix an elementary row operation on A has the same effect as pre multiplying the matrix A by an elementary matrix that is why I get this sequence E 1 E 2 etc E L okay so first operation is E 1 I do E 1 on A so that is pre multiplying A by E 1 second is E 2 pre multiplying E 2 pre multiplying E 1 A by E 2 etc so this sequence is E L E L minus 1 E 2 E 1 A I can write this as this whole thing I will call it as P or Q so I have B equals Q A and Q is a product of elementary matrix Q is E L E L minus 1 etc E 2 E 1 which is a product of elementary matrices okay that is the second part okay so the important step to observe is that an elementary row operation on A is equivalent to pre multiplying the matrix A by an elementary matrix and I am taking this sequence of elementary matrices okay okay let us now move on to the invertibility properties of elementary matrices so I need the notion of the inverse of a matrix okay let me give the definition some of you might be aware of it I have an n cross n square matrix with real entries then this matrix A is said to have a right inverse said to have a right inverse if there exists in other matrix B of the same order as A such that right inverse I have A B equals I identity is the identity matrix of order n I on the right is the identity matrix of order n okay if there is a B that satisfies this equation then A is said to have a right inverse a left inverse is defined similarly if there exists a matrix C such that C A equals I then A is said to have a left inverse okay together if A is said to be invertible if A has a right inverse and a left inverse A is said to be invertible if A has a right inverse as well as a left inverse what we will first show is that if A has a right inverse and a left inverse then these two coincide okay if A has a left inverse and a right inverse then they must be the same so let us prove this this is easy let us write down the definition there exists B such that A B equals I that is I am considering B since A has a right inverse there exists a B that must satisfy this equation there exists C such that C A equals I so now start with B B can be written as identity times B identity is the identity matrix is the identity operation of matrix multiplication for square matrices identity times B now for this I will borrow this equation this is C A times B this is matrix multiplication is associated C times A B but A B is identity from that equation so this is C times identity that is C okay so I have made use of both the equations to conclude that if it has a right inverse and a left inverse then they must coincide. Let me also prove two more properties of invertible matrices if two matrices are invertible then their product is also invertible and a formula for the product can be given immediately the inverse of the product is the product of the inverse is taken in the reverse order okay I am going to leave this as an exercise it is easy to verify just look at the defining equations for the inverse of a matrix. For a single matrix what must be true is that you do the inverse operation twice you get back the original matrix A inverse inverse is A okay if A is invertible okay and the first property can be extended the first property I can also say that a finite product of invertible matrices is invertible finite product of invertible matrices is invertible these are easy exercises okay what is important more to what we are discussing is that every elementary matrix is invertible okay every elementary matrix is invertible okay that is what we will prove do you have a choice for the inverse of an elementary matrix E I want you to make a guess of E prime E prime will be the inverse of E yeah what is okay we are using prime for that okay. So let us define capital E prime as E prime of I E prime is a notation that I used for the inverse of E that so let me complete where E prime is the inverse of the operation E E prime is the inverse of E so when I am given E I know E prime explicitly in fact I have written down those three formulas for E prime so I know E prime explicitly so the claim is that this E prime satisfies E into E prime equals I E prime into E equals I so it is both a left inverse and the right inverse and so capital E is invertible that is every elementary matrix is invertible the proof uses composition of functions and the fact that E prime is the inverse of E so proof is already there I have to just verify this so consider E times E prime I want to show that this is equal to identity E into E prime is what this is pre multiplying the matrix E prime by the matrix E E is an elementary matrix so this corresponds to an elementary row operation so this is E of E prime do you agree pre multiplying the matrix E prime by E is performing the elementary row operation little E on capital E prime this is E of E prime definition is E prime of I small E prime of I this is composition of functions this is E circle E prime operating on I okay but E prime is the inverse of E so E circle E prime is the identity function I will call it I I of I identity function on I leaves it as it is so I have E into E prime is the identity matrix identity operation on I I get back I E prime E can be worked out similarly okay let me do that also quickly consider E prime E this is pre multiplying the matrix capital E by E prime that is performing the elementary row operation small E prime on the matrix E which is performing E prime on E the definition is E of I okay this is again composition this is E prime circle E operating on identity as before this is the identity matrix okay so what we have shown is that E prime is a right inverse as well as a left inverse so capital E is invertible that is each elementary matrix is invertible a little exercise for you at the stage find the inverses of all the 5 elementary matrices of order 2 I had written down the elementary matrices of order 2 all the 5 elementary matrices I had written down find the inverses of all these operations you only need to find the inverses of the elementary row operations okay what we would like to do next is in the next lecture is to discuss properties of elementary matrices invertibility connection to systems of equations okay I will just give the result that I would like to start with from the next on the next class. Next class I want to prove this theorem for a square matrix A for A in R n cross n the following conditions are equivalent okay this gives a connection between row equivalents elementary matrices invertible matrices condition A A is invertible condition B A is row equivalent to identity that is condition 2 A is row equivalent to the identity matrix of order n condition C A is a product of elementary matrices okay that is A is invertible if and only if A is row equivalent to I if and only if A is a product of elementary matrices okay in proving this I would make use of a little result which I will ask you now the question I will ask you you try to come up with an answer tomorrow suppose I have a row reduced echelon matrix a square row reduced echelon matrix which is invertible I have a square row reduced echelon matrix which is invertible what can you say about that matrix capital R is a row reduced echelon matrix it is square it is invertible what is R? R is equal to identity try to prove this we will make use of this result next time I will stop here.