 Welcome. In this lecture, we will be discussing operational fundamentals of linear algebra. In particular, we will be talking about range and null space, their dimensions, rank and nullity respectively, basis for representing vector in a space, change of basis and an introduction to the technique of elementary transformations. To start with, consider a mapping A from R n to R m. The matrix A will be of m by n size, which will be something like this. It will have m rows and n columns and therefore, it will multiply a vector of size n and give a vector of size m. And therefore, we say that the mapping is from R n to R m, x is in R n and y is in R m, m dimensional vector. We observe the same two, we make the same observations as we made in the last lecture, that every x in R n when multiplied like this will certainly result into a y. That is every x in R n has an image y in R m, but on the other hand, every y in R m does not have to have a pre-image x to which multiplying with a we will get y, such is not necessary. Now, then we can talk of those vectors in R m for which there is some x and the set of all these vectors is called the range space or the range. So, we find that the range space is a subset also a subspace of the co-domain containing images of all x. That means R m could be a larger set and then we can single out all those members in R m, which come as a result of multiplying the matrix A to some vectors to or to all the vectors in the domain which is R m. And this subset of the co-domain is called the range, that is the range of the linear transformation. In the co-domain, there may be larger space available, but the entire space available in the co-domain is not used by the linear transformation A. It is only a subset of it, which is the range of A is being utilized for the purpose of mapping. The second observation that we made is that the mapping from x to y in order to be a mapping must be unique, that is for a given x we have a unique y. That is obvious because in this matrix multiplication the result is always unique, but the reverse does not have to be true. That is for a given y it is not necessary that the x should be unique and no other x should map to y that is not necessary. So, the second observation says that image of x in R m to R m is unique, but the pre image of y does not have to be. That means for the same y in the co-domain or in the range for that matter there could be several x which map to the same y, why several there could be infinite vector in R m which map to the same vector in R m. Now, if such is the scenario with the matrix A that it maps infinite vectors from the domain to the same vector in the range, then that will in particular be true for the 0 vector in R m as well. Then we find another definition from this observation and that is the definition of null space. Those vectors in the domain in R n which map to the 0 vector in the co-domain or in the range they form a subset of the domain and that subset is called the null space. So, we find that the null space is a subset or subspace of the domain which contain in contains all pre images of only 0 pictorially or schematically we can see it like that is this entire space is the domain R n, this entire space is the co-domain R m. Now, the entire R n domain must get mapped and but then it may get mapped to a subspace of the co-domain that subspace is the range space. On the other side in the domain there is a subspace here has differently which all gets mapped to the 0 vector in the co-domain. This subspace here is called the null space that is it does not have any information content in it all that goes to 0. Now, we introduce two more terms two more definitions that is rank and nullity before doing that we need to ask what is the dimension of a vector space. Note that here we are talking about four vector spaces the domain co-domain the subspace of the domain which is a null space which is also a vector space and the subspace of the co-domain that is the range space which is also a vector space. So, we are talking about all these four vector spaces. So, we ask what is the dimension of co-domain and that is n what is the dimension of the co-domain that is m. Then to introduce rank and nullity we ask another question which we should have asked by now and that is what is the dimension of a vector space what is the dimension of a vector space. So, for that we need to talk about linear independence of vectors linear dependence and linear independence. In a vector space suppose we take several vectors x 1, x 2, x 3 up to x r r vectors we have taken. Now when do we call these vectors as linearly dependent or linearly independent. So, the simple straightforward idea of linear dependence and independence is that if x a vector x can be expressed as a linear combination of several vectors like this then in such a situation we say x is linearly dependent on x 1 and x 2 which means that x can be formed by a linear combination of x 1 and x 2. In other words we can say x 1, x 2 and x are linearly dependent together. So, this is these three then we will form a linear dependent linearly dependent set. So, there is a linear dependence among these vectors. When we talk about the linear dependence of several vectors together then typically in the most general sense we say it in this manner that is if k 1 x 1 plus k 2 x 2 up to k r x r if the whole sum being 0 necessarily implies that all these contributions are independently 0 that will mean that these vectors are linearly independent. In the sense that from one place on earth if you need to go to another place on the earth on the surface then you think of two directions how much east west I have to go and how much north south I have to go. Now this east west and north south they are two independent directions then you say that wherever I need to go I can go by moving a little east west or a little north south. Then basically you will be talking about a little east west plus a little north south and this together gives you a movement on earth and that movement is then linearly dependent on these two individual movements east west movement north south movement. Now these three directions that you have got the east west direction north south direction and the direction in which you actually travel these three are linearly dependent. Now this statement that these vectors are linearly dependent if the whole sum equal to 0 necessarily implies this entire set of k 1 k 2 k 3 being totally 0 that means that a movement on ground is 0 it is no movement if its east west component is 0 as well as it is north south component is 0 that shows that these are linearly dependent vectors. So that means any change from 0 in any of these will not be compensated by any suitable change in any other that means the moment you change any of these coefficients this sum will turn out to be non-zero no other compensation from any other component will be possible. So in the true sense these vectors x 1 x 2 x 3 etcetera are independent of each other is any change in one cannot be compensated by any suitable change in others. Now we say that if in a vector space we are hunting for vectors which are linearly independent among themselves. So we find one vector the moment we find one vector in that vector space many other vectors in the vector space become linearly dependent on it we hunt for another which is linearly independent of it. So we get two such vectors like this as we go on looking for vectors which are linearly independent of the vectors which are already collected finally we might reach a situation where the process ends where no more vectors are found which are linearly independent of the vectors which are already collected. Now we see how many vectors we have already collected which are linearly independent among themselves that number is the dimension of this vector space the earth surface as a dimension 2 because after you exhaust east west direction and north south direction no more direction is necessary with these two directions moments in these two directions you can reach any place on the earth. In this three dimensional world in which we live why we say three dimensional world because here there are three linearly independent directions you go east west and you go north south and you go up down with these three independent moments you can reach any point in this physical space that is why physical space is three dimensional and this idea of the dimension of a vector space to be the number of linearly independent vectors contained in it we define two more terms as we have already discussed range of a is the set of all images of all x's in the domain null space of a is the set of all those x's in the domain which map to 0 in the range. Now the dimension of the range space is called the rank similarly dimension of the null space is called the nullity these definitions are extremely important and should be remembered in all the discussion that we conduct subsequently. Now to describe vectors in a vector space we talk of basis that is in terms of which vectors we can describe all the vectors in a vector space in this manner we want a handful of vector x 1 x 2 etcetera as linear combinations of which we would like to represent all vector in that vector space and this is the idea behind looking for a basis these members x 1 x 2 etcetera will be called the basis member to define a basis we ask this question given a vector v in the vector space question is can we describe it in this manner as a linear combination of r chosen vectors v 1 v 2 v 3 up to v r r chosen vectors can we do this as if we have chosen r vectors to form the basis to describe all other vectors in that vector space and then in that vector space a candidate vector v appears and we are trying to express this vector v as a linear combination of the chosen vectors is it possible or not where this this can be written in short hand in this manner because these vectors collected as column vectors and multiplied with these k's it actually gives us k 1 v 1 plus k 2 v 2 plus k 3 v 3 and so on up to k r v r the way it has been shown here and that is why this matrix this large matrix we can call as capital V and this vector as k now the question is that with v 1 v 2 v 3 v 4 up to v r known we essentially know this matrix and the vector v small v that has been given to us we know this and we are interested in finding out suitable k 1 k 2 up to k r such that this turns out to be right that is the new vector v gets represented as a linear combination of the vectors which are there in our collection. So, the question has been raised and the answer to this question is that it is not necessarily always possible now if it is not possible always then we say that then these vectors what these vectors are doing these vectors among themselves in this kind of a linear combination in this kind of a linear combination will generate a subspace in the vector space that subspace will have lesser dimension that the than the entire vector space and that subspace sometimes we denote as the span of these vectors. So, this is the span of this vector and we say that the subspace described or generated by these vectors is called the span of these vectors denoted by these angle brackets and in other words we say that these vectors span this particular subspace. So, span is used either as a verb or as a noun then when we look for the complete basis of a vector space we define it in this manner a basis of a vector space is composed of an ordered minimal set of vectors spanning the entire space that means that after we collect so many members v 1, v 2, v 3, v 4 etcetera such that all vectors in the vector space can be expressed in this manner when finally the answer to this question turns out to be yes after collecting enough then we say that can we do with less number of vectors in this collection if we can then whichever vectors are not required we throw them out that is the meaning of the word minimal here and then we need an ordering among them this is our first basis member this is our second basis member this is our third basis member and so on what remains gives us the basis of a vector space. For example, when we want to describe the points or vectors in this world in which we live we say we took we take one direction that is north many vectors many movements in this world can be represented by this all northward movement to be precise then we need one more direction then we say is note that west is same as east just the negative sign then we say that all places on this plane consisting of north and east directions are covered including here in the other quadrant right then we say that still we remain on the ground only what we need to do is to go up also then suppose we try this direction north east and it does not help it does not help us to go upward by going north east we do not go upward so then by going north east to which are the directions in which we can go we find that by going north east we go a little east and a little north that means this direction of north east is actually linearly dependent on east and north so it is not needed and we will look for a minimal set so this is not needed so the direction of north is not needed so which is the direction that is needed any direction which has an upward or downward component will help so we take a direction which is upward or downward it can be perpendicular to this plane but it need not be so let us call it up away from the blackboard as I draw it now we have got three directions east north and up and with these three directions we have formed a triad like this so with these three directions at our disposal we will be able to represent any other direction any other movement so there is no need of going for effort and we keep only three you notice that three were barely minimum which were needed and more than three we will not keep so why three were barely needed because the dimension of this space which we are trying to describe is three and therefore we also say that the basis for an n dimensional space will have exactly n members all linearly independent exactly n members to keep the set minimal and all linearly independent to be able to describe all vectors in the vector space finally we need to order them we have to say east is our first coordinate north is our second coordinate and up is our third coordinate as we do this our job for forming the basis is complete because after doing this now when we say that the coordinate of a particular point in the atmosphere happens to be alpha beta gamma then the description of this is completely known we have to go alpha distance in the east direction then beta distance in the north direction and gamma distance in the upward direction so only when the ordering is also prescribed then the statement of the basis gets completed sometimes we talk of a particular kind of basis which is called orthogonal basis in which these vectors are perpendicular to each other in this particular situation we have actually taken the example of a basis which happens to be orthogonal because in the directions between the directions east and north there is a right angle and between the direction of east and up also there is a right angle and so on. So mathematically we can say v j t v j transpose v k is equal to 0 when this holds for all j and k which are not equal that is v 1 transpose v 2 v 1 transpose v 3 v 2 transpose v 3 when all these three are 0 that means when this angle this angle and this angle or are all are right angles like this then we say that this basis is orthogonal right angle similarly we take another special case in which all three have unit vectors of the same size that is if we consider 1 kilometer in the east and 1 kilometer in the north and 1 kilometer in the upward direction as of the same size in the description then we will say that this is orthonormal basis orthonormal right sized not only right angle but also of right size in that case we say that this v j transpose v k is not just 0 for j and k different but it is 1 for j and k same you might note that members of an orthonormal basis they will have several unit vectors all perpendicular to one another. So these members these vectors together written like this will form a matrix which is orthogonal matrix and of course you already know that these are the properties of an orthogonal matrix it is inverse is the same as transpose or you can say v v transpose is identity this is the test for a matrix being orthogonal in particular it is determinant also is known it will be either plus 1 or minus 1 sometimes we describe vectors without mentioning the basis the way till now we have been doing by just writing x 1 x 2 x 3 x 4 in a column what we are doing what basis we are using then then we are actually without mentioning using a particular basis which is which comes naturally to us and that is called the natural basis natural basis members are just this 1 0 0 0 0 0 then 0 1 0 0 0 then 0 0 0 1 0 0 0 and so on. So these are the natural basis members and they are quite often represented with these symbols e 1 e 2 e 3 e 4 up to e n this n shows the dimension of that vector space now after describing the vectors of a vector space for quite some time in this basis the natural basis or say some other basis we want to change the basis that is we do not want suppose we do not want these directions and we do not want these lengths also we want to describe the vectors in this three dimensional world in terms of 5 kilometer distance in this direction which is a little north of e then 3 kilometer movement in this direction which is a little east of north and the third direction is not directly upward but a little upward inclined like this in particular we could do that and for example in this we will take a very small unit this is just 200 meters we can do it we can describe the three dimensional world in terms of three coordinates which are the locations in which are the distances along this direction along this direction and along this direction with the units being different that kind of a thing is possible and sometimes it makes sense to do that for example if you want to prescribe locations on a road how will you do it suppose this is the road and you want to describe locations points on this road then it will not help to describe the distances along north and east or for this road if you want to specify points on this road it will not be a great idea to describe in this manner that is this much is movement in north east and this much is the movement in north that will not be a good idea besides using the same unit on both sides also will not be a good idea because in that case the descriptions will be always quite lopsided the correct description would be movement along this direction which is in kilometers and movement along this direction which is in meters because otherwise if you choose the same unit say kilometers then for the direction along the road you will be talking about 10 kilometers 100 kilometers and so on and for this you will end up telling 0.3 kilometer 0.15 kilometer and so on which will not be very nice on the other hand if you choose meter then also you will have fine description for the breadth of the road but not for the length of the road so therefore it is quite often possible to require to need the change of basis let us take an example suppose you have a table which is not rectangular but which is of this shape and in this rectangle in this table it is parallelogram shape and on this table on the surface of the table there are mark like this and the sizes of these small parallelograms are 3 centimeter in this direction 2 centimeter in this direction so 3 2 3 2 like this so this is suppose the top of the table in which this kind of marks are made it is a design now suppose this particular table is kept and you want to know the distances along this and along this edge of the table which will take you here distances not in centimeters but in terms of the steps how many steps in this direction and how many steps in this direction will take me here. So, for that suppose our our original frame of reference in terms of north east or whatever in the rectangular Cartesian frame suppose that Cartesian frame happens to be let us call it x 1 and x 2 now when we say we want to know how many steps along this edge and how many steps parallel to this edge will take us here we are essentially asking for the coordinates of this point in the table system and table system has its axis like this. So, then first thing we need to know what is the description of the units of the table system in the original x 1 x 2 system. So, for that we take this unit and say how much it is from here to here so this point in the original system suppose it is described by a vector c 1 which is this much along x 1 this much along x 2 like this. Similarly, this unit which is 2 centimeters in this case we are want to find out its this vectors description in the original x 1 x 2 system and call it c 2. Now, we have this movement as a vector c 1 this movement as another vector c 2 and now we want to find out how many c 2 c 1 movements and how many c 2 movements will take us here. So, then we say how many c 1 movement plus how many c 2 movement will take us here x 1 x 2. Now, this left side we can write like this these are column vectors this vector c 1 and this vector c 2 those basis members for the table system those basis members for the table system are sitting here into c 1 c 2 as a column vector note that this matrix vector multiplication will give us c 1 c 1 plus c 2 c 2 what is here. So, or we can concisely write it as the square matrix c into this vector t is equal to the actual location of this in the x 1 x 2 frame that is x. This gives us an equation relating the number of steps along the edges of the table t 1 t 2 to the location of this point with we know in the old x 1 x 2 system and if we want to find out t then we need to invert the matrix c and get it like this. So, that will tell us how many steps along this we go here and here to find this point. So, this tells us the conversion from the old coordinates to the new coordinates and this tells us the conversion from the new coordinates to the old coordinates. As we formalize this methodology we say that suppose x represents a vector or point in R n in some basis. Now, if we change to a new basis consisting of these as the basis members then how does the representation of the vector change. We say that if with x 1 bar x 2 bar x 3 bar moments along c 1 c 2 c 3 basis members we can achieve the complete vector x then we will be representing x as a linear combination of these individual components along c 1 c 2 c 3 etcetera. This entire sum can be shown in this manner. Note that matrix vector multiplication will give us x 1 bar c 1 plus x 2 bar c 2 and so on. Now, these column vectors sitting here will have n column vectors each of size n and that is a square matrix. We can write it in this manner with c c 1 c 2 c 3 etcetera to c n this manner then we will have x is equal to c into x bar vector and that gives us the conversion system from the new coordinates to the old coordinates. By inverting this matrix c we get the transformation from the old coordinates to the new coordinates and make note that this inversion will always be possible that is this matrix is invertible because we already know that to form a basis c 1 c 2 c 3 c 4 up to c n and that gives us a methodology for conversion from the new system to the old system. And when we want the reverse that is from the old to the new when we want the conversion we have to invert this matrix c and we get the formula as x bar is equal to c inverted as we found in this case. Now, in a special case we have c matrix c as orthogonal that means in that case the basis members will be all unit vectors and mutually orthogonal and that means it will be an orthonormal basis and in that case this type of a basis change we call as orthogonal coordinate transformation. Now, we have seen that how the change of basis affects the representation of vectors the vector x in the old system gets represented as vector x bar and the relationship between the old description and the new description is given by this matrix consisting of the basis members as its columns. Now, we say that how does such a change of basis will affect the description of a linear transformation how does this basis change affect the representation of a linear transformation. What is this mapping a from R n to R m in this manner a multiplying with x produces y so that gives the description of a linear transformation. Now, we change the basis of the domain through a matrix p that means p will be an n by n matrix with its columns as the new basis members for the domain. Similarly, q will be another square matrix of m by m size in which the columns will be the basis vectors for the co domain that is for y. In such a situation based on the discussion that we just now had the new vector x bar in the domain and the old vector x in the domain will be related like this through the matrix p. Similarly, the new and old vectors in the co domain corresponding vectors in the co domain will be y bar and y representing the same point geometrically and they will be related through this matrix q which contains the basis members for the co domain. Now, originally the transformation the mapping was represented with matrix a between x and y. Now, the same thing we write here a x equal to y and as we do that a x is equal to y. Now, for x we use this and say x is nothing but p inverse p x bar x is nothing but p x bar p x bar and what is y? y we get from here y is q y bar q y bar now remember what we wanted to find out? We wanted to find out the matrix representation of the same linear transformation in the new basis as a result of the basis change that is we wanted to find out the matrix representation of the linear transformation which will map x bar to y bar that means we wanted something such that something into x bar gives us y bar to get that we can pre multiply with q inverse and get that as q inverse a p x bar is equal to y bar this matrix will now map x bar to y bar that means it will describe the same linear transformation as the basis in the domain and in the co domain. Have been changed to p and q respectively and that formally we say that the new matrix representation of that same old linear transformation will be q inverse a p. In the special case where the mapping is from R n to R n itself we have m is equal to n and p is equal to q this gives us what is called a similarity transformation. This is an issue this is a topic to which we will have ample amount of time devoted later when we discuss the matrix Eigen value problem or the algebraic Eigen value problem before that in this lecture we consider one more important topic which will be very crucial in our discussion of systems of linear equations and that topic is elementary transformations. Now, suppose you have a number of linear equations among a number of unknowns. Now, we need to recognize that certain reorganization of the equations in the system have no effect in the solutions or further matter on the description of the system itself it does not have too much effect it has the basic difference of expressing the same relationship in a different manner. For example, these three we will find that if we interchange two equations the first equation we write at the third location and the third equation we write in the first location then the system does not change and in the way of writing the equation in this manner a x equal to b that will amount to the interchange of two rows of a and b. If we interchange the first equation with the third equation that will basically mean that in this matrix we have the effectory representation of the system of equations that is equivalent to changing of two rows. This is called one of the three transformations that are called elementary row transformations. So, this is one elementary row transformation. Next, if we multiply the seventh row with two or one third does it change the system no it does not. So, that is equivalent to changing the scaling of a particular equation. So, that is scaling of a row. So, if we multiply one of the equations in the system by a scalar all over on the left side as well as on the right side then that does not change the solutions of the system of equations. So, scaling of a row is another elementary row transformation. The third one is addition of a scalar multiple of one row to another that is suppose earlier we had one equation and a second equation. Now, how does it change the system if in place of the second equation we consider a sum of the two equations. It will not change the solutions of the system of equations. One individual equation will be basically replaced by another equation which is equivalent to the old one in relationship with the rest of the equation. So, this gives us the third elementary row transformation addition of a scalar multiple of a row to another corresponding to these three elementary row transformations. We have elementary column transformations also similar operations with columns equivalent to corresponding shuffling of not the equations but of the variables. In that case interchanging of the first and second column will mean the equivalent of writing x 1 and x 2 in the reverse manner. Now, two matrices are called equivalent if you can change from one of the matrices to the other through a series of elementary transformations. So, these matrices which can be shifted from one to other through elementary transformations are called equivalent matrices. They are equivalent matrices they are equivalent in the sense that they satisfy the requirements of equivalent relations. Now, through elementary transformations you can reduce any matrix to this normal form in which there will be a leading block leading square block of r by r size. So, this is the equivalent of identity matrix and then everything else is 0. Any matrix you can reduce up to this form through elementary row transformations and elementary column transformations. You can note that in this other than dimension and rank of the matrix nothing else survives the rank shows up here in r. So, this identity matrix is of size r by r. Now, up to this much reaction is possible but not of great significance it does not have enormous amount of applied use. Most of the time in applications we use only row transformations or only column transformations for desired end. This is an important issue that after we keep on reducing the matrix through equivalent row transformations elementary row transformations we cannot get rid of this r that shows that elementary transformations do not alter the rank of a matrix. There is one very interesting way of looking at an elementary transformation. Let us do that with a small example. What we do here is that we take this matrix 3 1 2 1 0 minus 1 and apply a linear transformation an elementary row transformation to it. And the same elementary row transformation we also apply to an identity matrix. Suppose elementary row transformation that we decide to apply is add 2 times the first row to the second row. If we do that then what we get here first row remains unchanged the second row gets twice the first row added to it to get this. When you apply the same transformation same elementary row transformation to the identity matrix we get twice the first row gets added to the second row we get this matrix. Now, if we have applied the elementary row transformation to the identity matrix of the appropriate size and got this then it would also be possible to get this same matrix not through a direct application of the elementary transformation on this matrix. But by pre multiplying this matrix here I am reproducing it here for convenience. Now, if we multiply this matrix with this then 3 plus 0 3 1 plus 0 1 2 plus 0 2 and the next row 2 into 3 plus 1 7 2 plus 0 2 then 4 minus 1 3. So, that shows that to get this matrix we actually had 2 ways available either apply the elementary row transformation on this matrix directly or pre multiply this matrix with this square matrix which was found through the application of the same elementary row transformation on an identity matrix and this matrix in that case is called an elementary matrix. So, we get this situation and elementary row transformation on a matrix is equivalent to a pre multiplication with an elementary matrix and that elementary matrix is the one which is obtained through the same row transformation on the identity matrix of appropriate size. Similarly, an elementary column transformation is equivalent to a corresponding post modification these issues will be found extremely important when we try to apply elementary transformations for the solution of systems of linear equations in the next lecture. For the time being we summarize the important points that must be noted from this lesson. The important issues are first the concepts of the range and null space of a linear transformation, second effects of change of basis on representation of vectors and representations of linear transformations from one vector space to another vector space. Third important issue from this lesson is the idea of elementary transformations as tools to modify essentially simplify systems of simultaneous linear equations. These concepts will be extremely useful for our subsequent lessons on the systems of linear equations. Thank you.