 In the last lecture we were discussing the notion of elementary operations, there were three elementary operations that were defined, the formulas for these elementary operations were also given, today we will continue with our discussion on these elementary row operations and how they help in reducing a system of equations A X equal to B into a system of equations of the form C X equals D where reduction here means that the matrix C has a simpler structure than A, now what exactly is this structure that will be made precise a little later, so we will discuss this problem of reducing a system to a simpler system, reducing a given system to a simpler system. What we would also like to do is, see there is an intermediate notion of two systems being equivalent, so we will discuss this notion before we get into proving, before getting into the problem of elementary row operations and what it does to a system of linear equations, so what I want to discuss now is given two systems of linear equations, when are they called equivalent, so we will discuss the notion of equivalence of two linear systems, okay, so let us consider the following, so as before we have the system A X equal to B which we will call as 1 where A is a rectangular matrix M rows and N columns and the right hand side requirement vector is has M coordinates, so it is a vector in R M, let us write this equation in full, so we have something like the following, A 11 X 1 plus A 12 X 2 plus etc A 1 N X N equals B 1, the second equation A 21 X 1 plus A 22 X 2 plus etc plus A 2 N X N equals B 2 and let me just write down the last equation A M 1 X 1 plus A M 2 X 2 plus etc A M N X N equals B M, okay, so this is the expanded version of this equation A X equal to B, okay. Now suppose I choose certain scalars, let us say I have alpha 1, alpha 2 etc alpha, alpha k, I choose k scalars, k less than M be chosen, suppose that I multiply the first equation by alpha 1, second equation by alpha 2 etc the kth equation by alpha k, then I get the following equation, okay, I multiply the first equation by alpha 1, second equation by alpha 2 etc the kth equation by alpha k and then form a new equation, this equation is called a linear combination of the first k equations of this system, okay, so I get the new equation which is, so alpha 1 into A 11 X 1 plus A 12 X 2 etc A 1 N X N plus alpha 2 into A 21 etc A 2 N X N etc plus alpha k into A k 1 X 1 etc A k N X N, I form this left hand side is linear combination of the left hand side of the first k equations, the right hand side also I take a similar linear combination alpha 1 B 1 plus alpha 2 B 2 etc plus alpha k B k, I can rewrite this as that is, that is this can be written as alpha 1 into, okay, I will write this as alpha 1 A 11 plus alpha 2 A 21 etc plus alpha k A k 1 into X 1 plus, so all that I am trying to do is to collect the coefficients of X 1, collect the coefficients of H 2 etc and then see what that equation is. So the next equation is alpha 1 A 12 plus alpha 2 A 22 etc plus alpha k A k 2 into X 2 plus etc alpha k A k 1 alpha 1 A 11 plus alpha 2 A 21 etc alpha 1 A 12 plus alpha 2 A 22 etc the kth equation is alpha k A 1 k plus alpha 2 A 2 k plus etc plus alpha sorry this is alpha 1 alpha 2 A 2 k etc plus alpha k A k k into X k equals alpha 1 B 1 etc alpha k B k, okay. So all that I have done is to consider a new equation apart from these M equations, this is a new equation some constant times X 1 constant times X 2 etc constant times X k right hand side is of this form, the same constant right hand side is of this form, okay. Now the point that I am trying to make is the following, you take any solution X 1 etc X n that satisfies these M equations, you take any solution X 1 etc X n that satisfies these equations then that solution will satisfy this equation also, any solution in fact the way we have taken any solution satisfying the first k equations will be a solution for this single equation, will be a solution for this single equation that means what? So this is called the thing that we have written down here is a linear combination of the equations of the first k equations of the system 1, the single equation is a linear combination of the first k equations of the system. So any solution of system 1 will also be a solution of an equation obtained by taking linear combinations of the equations of system 1, okay. So let us consider this new system, consider the new system, let us call it by something else may be C 11 X 1 plus C 12 X 2 plus etc plus C 1 n X n equals D 1, C 22 X 2 etc sorry C 21 X 1 plus C 22 X 2 plus etc C 2 n X n equals D 2, I am just going to consider k equations for the moment C k 1 X 1 plus C k 2 X 2 plus etc plus C k n X n equals D k. So I am considering a new system which has the property that each equation of this new system is obtained by system 1 by considering linear combinations that is take the first equation that is obtained by a certain linear combination of the M equations of system 1, similarly equation 2 is certain linear combination of the M equations of system 1 etc. So each equation here is obtained as a linear combination of a certain linear combination of the M equations of system 1, then what follows is that any solution of system 1 will be a solution of system 2, I will call this 2, I will call this 2, so this is the point any solution of system 1 will be a solution of system 2, will be a solution of system 2 but the converse is in general false, however suppose that these 2 systems are related by the fact that each equation of system 2 is a linear combination of the equations of system 1 as well as the fact that each equation of system 1 is a linear combination of equations of system 2, if that happens then from whatever we have discussed till now it follows that these 2 equations have the same solution, okay. When we go from system 1 to system 2 what we are trying to do is to obtain each is to observe that each equation of system 1 has been obtained as a linear combination of the equations of system 1 and so any solution of system 1 must be a solution of system 2, all the time trying to do is to go the other way round and I want to conclude that these 2 systems have the same set of solutions, this can be done if I am able to verify, so this is true if it holds that every solution of system, every equation of system 1 is a linear combination of the equations of system 2, okay. There is a name for 2 systems that are related by this condition that is called equivalent system, so let me just give a definition 2 systems are said to be equivalent, if each equation of each system is a linear combination of the equations of the other system, so I have 2 system they are related by this condition, such systems will be called equivalent systems. So the argument that I have given now can be formalized in the following statement, theorem equivalent linear systems have the same solution set, equivalent linear systems have the same solution set, now this is a fundamental lemma that will be useful to us, let me just give a numerical example to kind of illustrate what is going on, let us consider the following equations, so first equation is x1 plus 2x2 plus 5x3 equals 0, the second equation is x1 plus 3x2 plus 8x3 equal to 0, the third equation is minus x1 plus x2 plus 4x3 equals 0, so this is the third equation, now this is system 1 let us say, let me also write system 2, so system 2 for me is, so let me write it on this side x2 plus 2x3 equals 0, so this is one equation x1 minus x3 equals 0 and then x1 plus x3 equals 0, I will call this system 2 and make the following observation, make the following observation which is, let me make the following observation which is that let us look at equation 2 is can be solved quite easily, okay let us do that quickly, from the second equation I have x1 equals, let us say x3 equals x1, I substitute into this, then the third equation gives me 2x1 equal to 0 that is x1 equal to 0, go back to the second equation I get x3 equal to 0, from this equation it follows that x2 is 0, so system 2 has 0, x1 equal to x2 equal to x3 equal to 0 as the only solution, this is the only solution for system 2, has 0 as the only solution, okay, on the other hand let us look at system 1, I will try and see whether each equation here is obtained as a linear combination of the equations of system 1, okay, so my objective is to solve equation 1, it might turn out that these 2 systems are equivalent, I do not know we need to verify that, okay but let us look at the first equation x2 plus 2x3 equals 0, I will leave it for you to verify that the first equation and the second equation are certain linear combinations of the 3 equations of system 1, okay, I will leave this as an exercise for you to verify, you can choose easy constants and then multiply the equation by the constant this by another constant add, subtract and do these operations, you will be able to show that the first 2 equations are linear combinations of certain linear combinations of the 3 equations of system 1, so any solution of system 1 will satisfy the first 2 equations but the third equation I am claiming that is not a linear combination of the 3 equations of system 1, the third equation here in system 2 is not a linear combination of the 3 equations of system 1. Now this is difficult to prove but what we can do is to make the following observation okay this is in general difficult to prove but what we can do is to observe that so okay what I will try and do is to give a solution of equation system 1 which does not satisfy the third equation it will then follow that this is not a linear combination of the three equations of system 1 okay. So let us consider the case I just give one set of values let us say x1 equal to x3 equal to 1 and x2 equals minus 3 okay x1 equal to x3 equal to 1 x2 is minus 3 then let us look at the first equation x1 plus 2 x2 is 6 minus 6 plus 6 that is 0 the second equation is x1 plus 3 x2 that is 1 minus 9 plus 8 that is 0 the third equation is minus x1 minus 1 minus 3 minus 4 plus 4 is 0. So this solves 1 this is a solution of system 1 you can now observe that this is obviously not a solution of system 2 the third equation is not satisfied because x1 equal to x3 equal to 1 and so this is not satisfied and so now it follows okay it now follows that see if the third equation here where a linear combination of the three equations of system 1 then any solution of system 1 would have to be a solution of the third equation now that is not that does not happen in this example and so system 2 is not equivalent to system 1 okay. Let us also observe that the general solution set of this system is of this form I will call S as the solution set that is of the form alpha times 1 minus 3 1 where alpha is an arbitrary real number okay it can be proved by the usual method of eliminating the variables that this is the most general this is the solution set any solution will be of this time let us observe that 0 is contained in this for the choice alpha equal to 0 okay which means that any equation here is a linear combination of the equations of 2 but not the other way round okay so these two systems are not equivalent okay so this is just to illustrate given example of two systems which are not equivalent so these two systems are not equivalent when they are equivalent the solutions sets will be the same okay we were discussing the concept of elementary row operations in the last class and then there is a pause we are looking at something else systems two systems being equivalent okay the thing is that these notions are related considering linear combinations of equations of a system is amounting to doing a certain elementary row operation or a sequence of elementary row operations okay we will make this precise we will make this notion we will make this idea precise but before that let us go back and recall the definition of the elementary row operations okay so what were the operations we call these as E 1 E 2 E 3 so each elementary row operation takes a matrix of order m cross n to another matrix of order m cross n the definition is the first elementary row operation is multiplying a row by a non-zero scalar so let us say that I multiply the sth row by a non-zero scalar alpha then that row changes to alpha ASJ all the other rows remain the same when I is not equal to s this is the first operation the second operation is to replace the sth row by sth row plus alpha times the rth row so when I is equal to s I am replacing the sth row so this corresponds to the first index so I am actually writing down the ijth entry of E 1 ijth entry of E 2 etc so the first index corresponds to the row second one corresponds to the column so when I is equal to s it is replacing the sth row by sth row plus so that is ASJ plus alpha times the tth row that is what I remember having used earlier and then the other rows are kept as they are if I is not equal to s then it is just aij this is the second elementary row operation that we discussed last time the third one is just interchange of rows let us say we interchange sth row and the rth row for instance then the ijth entry of this matrix is if I is equal to r I is equal to s I is not equal to r, s so I am simply interchanging the rows r and the interchanging the rows r and s so when I is equal to r it is ASJ when I is equal to s it is ARJ all the other entries remain the same this is aij these are three elementary row operations I made the statement last time that these functions each of these functions has an inverse and the inverse is also of the same type so let us see that quickly so that is going to be my next result. Each elementary row operation is has an inverse let me say has an inverse and the inverse is also an elementary row operation the inverse is also an elementary row operation what is more important elementary row operation what is more important is that it is an elementary row operation of the same type it is also an elementary row operation of the same type okay let us prove this let us dispose of the easy cases let us look at the first elementary row operation where we multiply a row by a non-zero scalar, okay. So all that I will do is explicitly write down the inverse functions. So let me use primes to denote the inverse functions. So I am saying that the inverse of the first elementary row operation E1A will be denoted by E1 prime of A, I will simply give the formula for that, then we can probably verify one or two of them. So E1 prime is, I must see the inverse operation must give me the matrix A that I started with. So given a matrix A I do the elementary row operation E1 on that and then I need to do another elementary row operation which will nullify the effect. So we should give me back the matrix Aij, so obviously since alpha is not zero it will be just this definition. So look at 1 by alpha Asj, so I am replacing, I am multiplying the sth row by the constant times 1 by alpha. This operation is allowed, this is multiplying a row by a constant, that is elementary row operation of the first type. Of course the other entries are left as they are, so this is my E1 prime. Let us quickly verify that E1 into E1 prime gives me the identity function, okay. So let us look at, let us call E1 prime of A as B, then okay, then I will write down the entries of B. So Bij by definition is, so E1 prime is the new matrix B, okay, Bij by definition is just this, 1 by alpha Asj if i is equal to s, it is Aij if i is not equal to s. So I am just calling this as a new matrix B. Now for this matrix B I will do the operation E1, that is I am now asking you to consider E1 of, see this is a circle E1 E1 prime of A. I want to show that this is equal to A, it would then follow that this composition is the identity mapping. So this is E1 of, so it is a composition function that is E1 of, so I am calling this as the matrix B, E1 prime of A as B and so now I will do this elementary row operation on B. Now how is this defined? So just look at E1 of B, the ijth entry, so you need to go back to this definition, multiply the sth row by alpha, leave all the other rows as they are. So E1 Bij, multiply the sth row that is Bsj by alpha, this is when i is equal to s, all the other entries are left as they are, that is Bij. So I am now doing an elementary operation 1 on B, so I am preferring to use Bij, so when i is not equal to s. Now go back to this and then see what these entries are, so this is equal to alpha times Bsj, what is Bsj when i is equal to s that gives me Bsj that corresponds to this, so alpha into 1 by alpha into Asj, so this is when i is equal to s, when i is not equal to s, when i is not equal to s it is just aij, this is equal to aij when i is not equal to s. Now you can see that this is precisely the matrix A, so what we have shown is that, so what we have shown is that E1 of E1 prime of A we have shown that it is equal to A and so it follows that E1 circle E1 prime equals the identity function, we can show in a similar manner that E1 prime circle E11 is also the identity function, I leave that part, I leave that part that is very similar to this, okay. So you must observe here that second part, it is an elementary row, it is invertible not only that it is an elementary row operation of the same type, type here means multiplying the S row by a non-zero scalar, okay. We can similarly verify that the other two operations are also invertible and that the inverses are of the same type, I will leave this part as an exercise without getting into the details, all that I will do is to write down the inverse formula for the inverse, E2 prime of A, now E2 E2 is this operation replacing the S row by S row plus constant non-zero constant times Tj, now E2 prime the inverse of that can be shown to satisfy this, so when i is equal to s it is again the S row that is ASj, this time it will be minus alpha times Atj, the other row elements are kept as they are i is not equal to s, now this is an elementary, we have to show that this is an inverse, this is the inverse of E2 but let us say we have shown that then it follows that the inverse is also an elementary row operation of the same type that is it replaces the S row by S row plus a constant times the Tth row, okay. So this is also an elementary row operation of the same type as E2, finally E3 prime is straightforward is again this is the ijth entry, E3 prime is straightforward, you interchange two rows you do the same interchange once again you will get the matrix A, so E3 prime of A this time I will write it is just E3 of A, this is such a thing is called a self-invertible map, 3 is its own inverse, okay so that is again if E3 is interchanging row R and row S then E3 prime the inverse is interchanging again row R and row S, so it is again an elementary row operation of the same type as E3, okay so that proves this theorem. Now this gives us this leads to another definition called row equivalent matrices, so let me write the definition here the definition of row equivalent matrices R m cross n for me will be the set of all matrices having m rows and n columns, so this will be my notation, this notation will be considered again when we discuss the notion of vector spaces, okay. Suppose I have two matrices then we say that A is said to be row equivalent, A is said to be row equivalent to B, remember one equivalence that we considered earlier is that of systems being equivalent, okay we are now considering equivalence of two matrices of the same order, however we will show that these two notions are related to each other, okay A is said to be row equivalent to B if B can be obtained if the matrix B can be obtained from the matrix A what is the condition by a sequence of by a finite sequence always by a finite sequence of elementary row operations by a finite sequence of elementary row operations on the matrix A, B is said to be row equivalent A is said to be row equivalent to B if B can be obtained from A by a finite sequence of elementary row operations on the matrix A, so starting with A I arrive at the matrix B then I say that A is row equivalent to B, okay then one would like to ask the question as to whether one can go the other way round if A is row equivalent to B can it happen that B is row equivalent to A, okay, obviously A is row equivalent to itself you do not do any operation you leave it as it is then A is row equivalent to A a finite sequence of operations has been performed so A is row equivalent to itself the question is if A is row equivalent to B then can we go the other way round that is whether B is row equivalent to A we will answer this question a little later in the next couple of lectures by observing that the process of going from A to B can be reversed in fact this can be done by means of the elementary row operations that we have discussed just now. We will look at what is called as the notion of elementary matrices so each elementary row operation gives rise to an elementary matrix using the elementary matrices we will show that if A is row equivalent to B then B is row equivalent to A we will also show that if A is row equivalent to B and B is row equivalent to C then A is row equivalent to C, okay. Now this is intuitively obvious but it can again be a formal proof can again be given for this that is if on the matrix A you perform a finite sequence of elementary row operations to get the matrix B and then further down you perform a finite sequence of elementary row operations on B to get the matrix C then obviously you do the same the sequence one after the other sequence of elementary row operations one after the other on the matrix A you will end up with the matrix C. So this transitive relationship is obvious from the definition alright so this operation this relation equivalence on the set of all m cross n matrices is really an equivalence relation right a reflexive anti symmetric transitive relation is called an equivalence relation so this is actually an equivalence relation that justifies the name equivalence the only thing is there is an adjective here because you are doing elementary row operations so row equivalence is actually an equivalence relation, okay. Now maybe I can give a simple example of obtaining a matrix B from a matrix A so let us look at one simple numerical example and then in the next lecture I will consider the problem of how the notion of equivalent systems of linear equations related to row equivalence of two specific matrices, okay this link this relationship is an important relationship where we will be able to consolidate this two seemingly unrelated notions of row equivalent matrices and equivalent systems of linear equations, okay. So let me conclude today's lecture by considering a simple example of constructing a matrix B that is row equivalent to a matrix A for instance so let us take A to be let us say 110, 111, 101 and let me take B to be the matrix 100, 010, 001, okay the entries are all very simple they are either 0 or 1, note that B is the identity matrix of order 3, okay. Now let us do so all that I need to show so the claim is that A is row equivalent to B that is I will show that upon a sequence of finite sequence of elementary row operations on A I will get the matrix B, okay let us see how this is done. So let me write A once again A is 110, 011, 101. Now I will do the following elementary row operation on the matrix A that is I will replace row 3 by minus 1 times row 1 plus row 3 and leave the second row as it is then A is equivalent to so I will use this symbol then A is equivalent to the matrix the first row entry is the same the second row is also the same the third row is minus this row plus this row so that gives me 0 minus 1, 1 then I do the following operation row 3 is row 2, 1 times row 2 plus row 3, okay and I should also do the following operation on row 1, row 1 will be minus 1 times row 2 plus row 1. So I am keeping row 2 fixed when I do this then I get the following matrix which is row equivalent to the matrix A so the second row is kept as it is so I will write that as it is 010 the third row becomes row 2 plus row so I have to just add these 2 rows that gives me 001 and row 1 is replaced by minus row 2 plus row 1 so this is 100, this is precisely the matrix B and so A is row equivalent to B, okay now in this example you can verify by doing the inverse operations say I have done elementary row operations I know that each elementary row operation is invertible and that the inverse is also of the same type so starting from B I can do the inverse elementary row operations and end up with the matrix A, okay so it can be verified in this numerical example that A is row equivalent to B and that B is row equivalent to A, okay. So let me stop today's lecture with this example the relationship between equivalent systems of linear equations and row equivalent matrices I have 2 systems A x equal to B and C x equal to D I will actually show that if A x equal to B and C x equal to D are equivalent then the matrix A comma B and the matrix C comma D are actually row equivalent, okay so this will help us to formalize Gaussian elimination that will be done in the next few lectures so let me stop here.