 So, in the previous lecture we have seen this good result that if you have a matrix A does not matter whether it is square or rectangular we can always get it to a row reduced echelon form right through a series of elementary row operations in which case we say that any matrix is row equivalent to a row reduced echelon form. So, now then it is only befitting that we show you what is the merit of this row reduced echelon form or why is this so desirable. So, the first thing I will do today is I will take up an example relatively simple example which I can hopefully walk out on the board but I also request you to work out with me because this involves a bit of calculations and I might go wrong on some of those steps. So, do correct me at any step when you find that my calculations are going wrong. So, let us take an example A x is equal to 0 and you might immediately say hang on you said you will be solving for A x is equal to b but now suddenly you are changing the rules of the game and saying I am going to solve for A x is equal to 0 why is this right. So, I just like you to hold on to that thought for a moment and just show you the analogy for instance when you solved differential equations. How did we go about it? We first found the complementary solution by considering that the right hand side of the differential equation is 0 and then we had this particular integral which told us what the particular solution for that particular differential equation when you have a certain kind of a forcing function would look like right. So, we are going to do this in this pretty much the same spirit and therefore this is like finding that complementary solution for the ODE. So, with that sort of intuition in place let us see what we can do with this. The point is that we have already seen that any matrix can be taken to a row reduced echelon form without altering the solution set. So, let us do that to this fellow too. The advantage is if you have a 0 on the right hand side it does not matter what sort of operations you do on this the same operation on the right hand side will still lead to a 0. The structure of the right hand side does not change through those row operations because it is already a 0. You pre-multiply this or hit it with any of those elementary row operations it is still a 0. So, let us say we get from here to the row reduced echelon form which is this. Let me just for the sake of completeness mention that this A is m cross n. So, we are going to try and see an example of this how you get to this just a bit of working out. Let me take this example let us say 2, 4, 6, 8, 1, 7, 4, 2 I hope I am not digging my own grave by choosing a rather complicated one. Let us see 4, 18, 14, 12. Suppose this is my A matrix and I want to get it to the row reduced echelon form. So, please check my calculations as I said earlier also. So, the first thing we do is we scan from the first row onwards. The first non-zero entry in the first row happens to be the first entry itself right. So, what do we do? We need to get a leading one here. So, I am not going to cook up those explicit matrices I am going to try and do a bit of mental calculation and show you what the next step would be. And in fact, I am going to try and combine multiple steps at one go which is why I might go wrong. So, from this let us just divide this by 2 throughout sorry yeah 2, 3, 4. But now the next goal is to generate 0's below the leading one that I have just obtained here for which what do I need to do? I need to just subtract this row from this right. So, the first one will just lead to 0 and 7 minus 2 is 5, 4 minus 3 is 1 and 2 minus 4 is minus 2. By the same token I need to subtract 4 times this row or 2 times this whichever way because we have already reduced it to the leading one. Let us consider this row because I am combining you see how many steps one operation of the first kind and 2 operations of the second kind in one step itself. So, that I am over and done with the first leading one and the entries below it in one go right. So, a lot of mental calculations may be but I hope you are following the bits. So, what is this? This minus 4 times this. So, already 0 here 18 minus 8 right. So, that is a 10 here 14 minus 12 that is 2 here no 14 minus 6 right 12 okay. So, that is a 2 I was right and 12 minus. So, this is 16 right. So, minus 4 yeah of course I cooked it up that way on the go. Now the next step is to look for the second leading one in the second row which happens to be this fellow right. So, I have to make this a 1 now. So, let us try and do that. So, notice that no matter what I do to this and scale it and add it to this, this leading one is unaffected because I have already generated 0s below it right. So, that is the reason why you must do one operation of the first kind followed by m minus 1 operations of the second kind because you have already gotten 0. So, any operation here after does not affect this leading one. So, what do you do? Divide this by 5 of course. So, let me write this second one first 1 1 5th and minus 2 5th right and then I have to subtract 2 times this row from the first. So, that I get a 0 here what happens 3 minus 2 5th right. So, what is it yeah 3 minus 2 5th is it not. So, 15 by 5 minus 2 by 5 that is 13 by 5 I hope again I am not erring and this is 4 minus plus because there is a minus sign already. So, 4 plus 4 by 5 right. So, 20 by 5 plus 4 by 5 that is 24 by 5 same thing I have to do here, but because I have cleverly chosen this I know what the answer is going to be in this case no chance of a calculation mistake and I urge you to check that you will already get 0's here just check right it is just twice this that is the way I had cooked it up right. So, is this in row reduced echelon form does it meet the condition it is row reduced of course and also this is k 1 is 1 k 2 is 2 the position of the first leading 1 is 1 the position of the second leading 1 is 2 and there are no other questions of any leading other other leading ones because there are only 2 non-zero rows. Now, by my claim here these 2 are equivalent systems. So, if I want to solve for A x is equal to 0 I might as well solve the same for R x is equal to 0. So, you might now say what is the benefit of this let us look at it. So, now I have to solve the following 1 0 13 by 5 24 by 5 0 1 1 5th minus 2 5th 0 0 0 and 0 and you have x 1 x 2 x 3 x 4 is equal to 0 what I am next going to do is the following you see these leading ones that I have generated it is not for nothing that I have focused my attention on them. The columns that contain the leading ones are going to be considered as what we call as the pivot ok. So, these 2 in this case are the pivots what does that translate to what do those 2 columns correspond to? The first column corresponds to x 1 and the second column corresponds to x 2. So, I immediately say that the columns that contain the leading ones are the pivot columns and so are the corresponding variables known as pivot variables. How many equations do you essentially have in this equivalent system now? The third one is just rendered meaningless it is 0 it is trivially true yeah. So, there are essentially 2 constraints just hold on to that thought again for a while. So, what does this first one say? I am going to write it in this manner where the pivot is the subject of the formula and the non pivot variables which we shall shortly assign as the free variables will be on the right hand side of this formula. So, what does that mean? This means it is minus 13 by 5 x 3 minus 24 by 5 x 4 and x 2 is going to be equal to minus 1 by 5 x 3 plus 2 by 5 x 4 right. So, where are we getting at with all this? Let me just erase these steps here ok the pivot variables and the non pivot variables. Let me name those non pivot variables by something else. Let us just say let x 3 is equal to u 1 x 4 is equal to u 2. Now, what does this whole vector x look like? Therefore, let us say that x 3 is equal to u 2 x 4 is equal to u 2. Now, what does this whole vector x look like? Therefore, x 1 x 2 x 3 x 4 is going to be equal to and this is how I am going to write this as u 1 plus u 2. So, can you tell me what this is going to be like? I have this as u 1 and this as u 2. So, what is x 1? See the pivot variables do not depend on each other. There is a decoupling already by virtue of the fact that I have gotten this all other entries below or above the pivot variables corresponding column to 0. Everything below the leading one and above the leading one is always a 0. So, they are decoupled. So, quite expectedly the expressions for the pivot variables will not involve each other. They will just depend on these so called non pivot variables or the free variables which are renamed it is just a relabeling really right. So, what is this x 1? It is minus 13 by 5 I presume minus 13 by 5 u 1 minus 24 by 5 u 2. About x 2 it is minus one-fifth and plus two-fifth minus one-fifth and two-fifth by the way these are just real numbers any arbitrary real number would suffice. What about x 3? By my very naming convention the definition I have given x 3 is what? Look at the non pivot variables. What do you notice so special about these non pivot variables? They have exactly one variable. They depend on exactly one variable and all the others will be 0. So, for example x 3 is representable just in terms of u 1 and it does not require any dependence on u 2. So, it is just this one here and a 0 here and this is 0 here and a 1 here. This structure is going to be very important. The fact that the pivot variables and these are the free variables look at this structure. What does this say? If I now try to generalize this for instance what can I say? Already you can observe something about this. Let us just point that out explicitly. You are solving for A x is equal to 0, but now if you give any arbitrary real value to u 1 and u 2 it happens to be a solution of A x is equal to 0. So, x need not be equal to 0. x is equal to 0 is of course one solution the trivial case when you put u 1 and u 2 both to be equal to 0, but even if u 1 and u 2 are not 0, but any arbitrary real number you will still be able to satisfy this. So, there are non-zero vectors that satisfy the equation A x is equal to 0, right. What is it that guarantees this? What sort of a situation guarantees this? That is what we are going to try and understand, but let us look at it in a more generic fashion now and try to understand what is going on. So, I will erase this particular example and I will write the expression. If you have any queries please feel free to stop me at any juncture. So, what I am going to write? Suppose you have A which is m cross n with m strictly less than n, yeah. This is what we call an under-determined system of equations, okay. When you are looking at A x is equal to 0 with m strictly less than n, it is an under-determined system of equation. In fact, A x is equal to b is also an under-determined system. Whenever m the number of rows is strictly less than the number of columns, it is under-determined. And a matrix that results is a fat matrix, okay. Or you could call it a wide matrix if you do not want to body shame it, yeah. So, it is an under-determined system of equations encapsulated by a fat matrix A. What happens then? What can you say about the row reduced echelon form of a fat matrix, yeah. So, look at the row reduced, I am going to use the MATLAB notation, the row reduced echelon form of A. What can we say about this? Yes? Why? I am talking about the row reduced echelon form. What can you say about this immediately? Let us not get to linearly independent vectors now because we have not yet defined it, but we are just talking about the, we will talk about it in the language that we understand as yet. I understand where you are getting at, but let us just use the terms we have seen so far, you know, like terms like leading ones, echelon forms and so on. So, what can we say about this? How many rows are there? m. So, at most how many leading ones can you have? Because each row will have one leading one. So, total number of non-zero rows actually will have leading ones. So, at most m of those rows in the row reduced echelon form can be non-zero. So, the maximum number of leading ones is m. And what are we seeing the number of leading ones to correspond to? The pivot variables. Therefore, the maximum number of pivot variables is equal to n. What can we say about the number of free variables then? n minus m, which is strictly greater than 0. So, therefore, there exist free variables. And what can we say when we know that there exist free variables? These free variables can be freely assigned any arbitrary value, not just 0. Therefore, the moment I have free variables ready to be chosen at my beck and call, I can choose them to be non-zero. And therefore, I am always assured to have a non-zero solution to the equation A x is equal to 0. Make sense? Right? So, therefore, A x is equal to 0 must have a non-zero or sometimes we call it non-trivial solution. That is another way of seeing this in a slightly more sophisticated manner. If I want to write this down a little more formally not very, let us say you take the set 1, 2, 3 like this till n. What is this? The set of indices of all the variables, right? And there is this notation which is the subtraction of elements from a set to another. Let us take this to be k 1, k 2 till k r. What are these? These are exactly the positions of the pivot variables that is the leading ones in rows 1 through r which are the non-zero rows. This r can at most be m. So, therefore, this set let us not call it s, let us call it f. This set is non-empty. I mean I am just making the same argument as before only slightly more formally. Do you see the connection? It is the same argument, a little more technical presentation of the same argument. I am just saying these are the indices of all the variables. These are the indices of the pivot variables. If you take away the indices of the pivot variable from all the variables, from that of all the variables what you left with is the indices of the free variables. If the indices of the free variables is a non-empty set it means there exist free variables exactly this, right? Either you go about this or you go about it in this technical fashion it is the same thing. In fact, now that we have defined this set f already we are now in a position to describe the solution, the complete solution of this in a rather formal term. So, let us say this is it. So, I am going to write this x as a complete solution here and what is it going to look like? You will have certain pivot variables and certain free variables. So, suppose you have let us say this is our 10 tuple, okay. So, suppose a is say 3 cross 10. So, suppose a is say 3 cross 10 which means there are 10 variables and 3 equations and let us say you have 3 pivot variables so that there is no 0 of all 0s, right? What can you say? Suppose x1 the first position here x5 is the first position the fifth position and let us say the seventh position. So, x1 x5 and x3 are the pivot variables. What can we say about this? The first position it will be summation over ci times xi where i belongs to what set should it come from? You see the pivot variables we have seen in that example that we worked out the toy example they must be representable as combinations of the free variables alone. So, this must be for all the i that comes from this set f, agreed? Yeah, same thing can be made about or same sort of comment. So, let us say this is ci this is say bi whatever xi i belongs to free variables xi i belongs to free variables and this is say ai xi i belongs to free variables. What about the remaining entries? They will all correspond to some free variable or the other, right? So, you will have just maybe x2, x3, x4, x6, x8, x9, x10. Yeah, but we want to write this up in a better fashion still. So, can you guess what is coming up next? How am I going to write this down? Let us put all the free variables like so x3, x4, how many are there? Well, I should write something times x6 plus x6 plus x6 plus x6 ok then x8, 9, 10. So, x10, yeah x8 of course, I just missed x9 anyway you can just fill out and what can we say about the overall structure of these? See x2 will just have a 1 in the second entry. Will the second entry of any other vector have anything but a 0, right? It is just going to contain 0's everywhere except in this first vector. This is going to be incredibly important later when we talk about linear independence and other things of these vectors. Just bear that in mind we have not yet introduced the notion of linear independence. I am not going deep into that, but we will talk about it later. We will revisit this representation and what about the entries corresponding to x1, x5 and x7. So, how many vectors do you think are here? Exactly the as many as the number of free variables, right? So, do you now recall this picture that we had where I said that the total sum of the number of degrees of freedom and the number of constraints must be a fixed number and in fact that is equal to what? The total number of variables that is n, right? Is equal to the number of what? Leading once. That is nothing but the number of constraints, yeah? Because the number of leading ones is effectively the number of equations that you are looking to satisfy. The rest of them do not matter, is it not? Plus the number of free variables which is your degrees of freedom. In fact, at this juncture I might as well define the so called row rank of a matrix in the following manner. So, this is clear, right? I can erase this part. So, in fact we will say that the row rank of a matrix A can be defined as the number of numbers of non-zero rows or if you like you can call it the number of leading ones. It is all the same in R, R, E, F, A. Later we shall see that row rank and column rank does not matter is the same, not a very intuitive result. And therefore, in general we can get rid of the prefix row or column and we can just talk about rank of a matrix. Again, things for later, but now this is the way we can define the row rank. In that case, we can just say this N is equal to the rank plus the number of free variables, right? Getting close to the so called rank nullity theorem those of you who have encountered it earlier might immediately notice, right?