 Let us now discuss an important concept of matrices which are called the Eigen value and Eigen vectors of a matrix. So let us understand what is meant by an Eigen value and Eigen vectors associated with a matrix. Let us consider a matrix A. We will start with an example to find out given a matrix A how do you compute its Eigen value and Eigen vectors and look at their properties. Let a given matrix will take a very short matrix small size 1 of size 2 cross 2 and let this be the matrix for which we need to compute the Eigen value and Eigen vectors and let a vector x which has two components x1 and x2 the two components here because the matrix is of size 2 cross 2. So we are talking of a square matrix here in general however if the matrix A is of size n cross n the vector will be of size n. So it will have n components n will be the dimension of a unit vector. At this point we will not discuss Eigen values associated with a non square matrix. So let us look at how to compute this x. So let x be the first Eigen vector the question is how many of this x's will be there that depends on some properties of basically depends on the size of A. We will now take that the number of possible values of the Eigen vector a number of Eigen vectors x will be dependent on the size of A. So let x be the Eigen vector associated with an Eigen value for the time being we will use a notation lambda to denote an Eigen value. So if x is an Eigen vector and it is associated Eigen values lambda you can write an expression as x which we can write by substituting these two let us do that. This is a simple substitution of A here and then it is corresponding Eigen vector x1 x2 and this is equal to lambda times x since some sense we are writing A x equals lambda x this also can be rewritten in this particular form as A minus lambda i times x by this to the left hand side this is equal to 0. So this is actually called the characteristic equation for the matrix A and this will yield some solutions for different lambdas. How many solutions I just talked about depending upon the size of the matrix A. So if the matrix A is of size 2 cross 2 or the size 2 because we are talking about square matrices we can expect there are two values of lambda lambda 1 and lambda 2 and also there will be correspondingly two solutions of this vector x well I will change the notation here to be consistent let us say this is a vector x with these two components. So you will get in some books the notation small x written here with a vector sign which indicate it is a vector so you can do that as well. So it is alright if you had actually written like this then you had to put a small x with a vector sign but we will restrict ourselves the capital X which indicates a vector. So I am putting the vector here small x with subscript x 1 x 2 are these two components of the vector x. So the question now is given this characteristic equation A minus lambda x ix equal to 0 how many solutions are possible and let us try to write this in this particular form. So A minus I am writing this again it is called the characteristic equation or characteristic polynomial for the matrix A. The degree of the polynomial will be dictated by the size of the A and hence the number of solutions which are possible. Now depending upon the value of A the question is how many solutions are possible. Let us take a situation when A minus lambda i this part remember i is an entity matrix we introduced this earlier and lambda is just a scalar value. So if we take this matrix and look at its determinant value if you look at this matrix and look at its determinant value and assume if this is singular that means if this is equal to 0 that means this is singular what do you think will be the possible solutions for x what do you think will be the possible solutions of x. So we have an homogeneous system of equations here and we look at the case when the determinant of this matrix is singular and we are trying to find out that under this condition what are the different values of lambda which provide these singularity condition within this matrix which will actually then give this possible values of lambda and using those values of lambda after we substitute it here the values of the vector x which satisfy this particular equation will be called the Eigen vectors. So based on this fact that we have to find out solutions for this characteristic equation of the determinant of this term equal to 0 we will substitute this value of A here on to that expression and find out what are the values of lambda which satisfy this constraint and do that if you substitute this is what you will get is this correct if you substitute it here if you take the values of A the elements 2 cross 2 matrix substitute it here this is what you should get and this will in turn give you equation which is very simple to derive I leave this as a trivial exercise for you to check that you can write it in terms of these factors what I mean is this particular thing here is going to give this giving us two possible solutions for lambda which are very simply 2 or 3. So why there are two values because the psi 2 cross 2 if the size of A is n cross n you will have n different possible values Eigen values n different possible Eigen values and correspondingly n different Eigen vectors will use these two Eigen values to find out the Eigen vectors in this case but if the size of A is very large say 10 cross 10 matrix are even higher in certain cases which we require a lot in the field of pattern recognition for clustering as well as classification there are sophisticated algorithms to compute do what is called an as an Eigen value decomposition to find out what are the corresponding Eigen values and then the corresponding Eigen vectors we will talk of that later on as to what algorithms one can use to actually do an Eigen value decomposition to get the corresponding Eigen values and Eigen spectrum and I will just name that one such algorithm which is called the singular value decomposition which you can use for large matrices but this is typical the case you can actually solve it even the matrix size is about 3 cross 3 you will get a polynomial of order 3 and you can get 3 different Eigen values but let us finish this problem where you have 2 Eigen values and using this say you choose the first one which is lambda equal to 2 you do it here to get the corresponding equation so what you are looking at is either A minus lambda x equal to 0 or you can also write x equals lambda x is given here I repeat again x equals lambda x or you can write something like this and this will actually help you to form an equation like this A x minus is let me let me take this fine so if you substitute lambda here let me see what do I get lambda equals 2 will give me minus 1 1 minus 2 where is this value will be 2 multiplied by say one such value of x which is given by x 1 x 2 is equal to 0 so using the value of lambda equal to 2 as the Eigen value you can substitute in this characteristic equation and get this and you have to find out a set of values for x 1 x 2 which will satisfy this constant actually what you will get is the same same equation if you try to substitute and write to write this in the form of linear homogenous equations you will get 2 equations which are both identical and the value of x 1 comma x 2 which actually satisfies this can be given in this particular case as this so this is an Eigen vector in fact to be very precise this can be multiplied with an arbitrary value of k which is any scalar quantity so k multiplied by this that means you can replace 1 1 by k a k and k here and all such Eigen vectors with this quantity for all possible values of k will actually give you possible Eigen vectors which will satisfy this equation with this particular lambda now we will move to the other Eigen value lambda equal to 3 and write the characteristic equation there as well so if you do that the equation which you will get now is this is what you will get as a minus lambda i and multiplied by x is equal to 0 this is again giving you the same is very clear that you will get the same equation which will be satisfied by a value of x equals x 1 x 2 and actually what you are getting from this is a relationship of this form what you will get is minus 2 x 1 plus x 2 is equal to 0 which in turn specifies that x 1 is equals to x 2 by 2 any pair of values x 1 x 2 which satisfies this will be a possible solution for the Eigen vector here one such example could be well say one or two or you can also write this as half and one point five and one and multiplied by k because for any arbitrary value of k here they will all satisfy this particular equation so this is a very simple method where you solve for the Eigen values and look for the Eigen vectors which satisfy this next we will take an example now where the size of the matrix A is of size 3 3 cross 3 so now we will look at some examples of obtaining Eigen values and Eigen vectors from a matrix of size 3 cross 3 the dimension is 3 let us take an example 3 cross 3 matrix so there are 3 rows and 3 columns very simple earlier we have taken examples of 2 cross 2 so the matrix is minus 21 minus 9 12 0 6 0 minus 24 minus 8 plus 15 so we will look at the characteristic function a minus lambda i which gives that should be it so determinant of this matrix I leave it as an exercise for you that when you write it in this form you should be able to evaluate in fact there will be a dominant diagonal you should be able to write I am leaving this as an exercise for you to write it in terms of factors what is the third factor lambda plus 9 you should be able to write it like this this gives 3 corresponding Eigen values so the Eigen values are of course you can ask me a question whether you will get 3 such distinct factors or 3 distinct Eigen values we will see an example next where you may not get 3 distinct Eigen values the Eigen values are lambda equals 6 lambda 1 equals 6 sometimes it is called lambda 2 equals 3 and lambda 3 equals minus 9 these are the 3 corresponding Eigen values so based on these 3 Eigen values we will now try to obtain the corresponding Eigen vectors so for each of these corresponding Eigen values I need to find out the corresponding Eigen vectors so what you need to do you need to form equations where this is satisfied we know the values of lambda so you need to find out the corresponding v's which will be satisfied and they are called the Eigen vectors but for this solution of the equations they will be called the non-trivial solutions they are called the non-trivial solutions of this homogeneous system of equations so let us start with the first Eigen vector let us say we will choose minus 9 to start with so lambda 3 which is equal to minus 9 you can start with lambda 1 also does not matter so this will give a plus 9 i what will you get that means the a is given here plus 9 so you can see the diagonal elements will only change correct so that will give you what will be the first element minus 12 minus 21 plus 9 which is minus 12 the rest of the non-dagonal elements will remain the same is not it so this is 0 6 plus 9 15 0 minus 24 minus 8 then 15 plus 9 24 now you have to find out what value of v this is satisfied so the method which is even adopted there are lot of algorithms to find out the corresponding these but what is done is basically you try to reduce this form of a minus lambda i you got a plus because you have a minus here these two minus produces a plus 9 okay so you we call this is a row reduce that means you try to operate row wise on this matrix of course you can operate column wise also and you try to get a row echelon form if you operate column wise you will get a column echelon form that is also possible but here we are talking about row reduce to get to get up to obtain the row echelon form of this matrix I leave it as exercise for you to learn this if you do not know this what we will do is end of the course we will run one simple example to show given a matrix how to obtain a row echelon form the basic idea is that the leading entry of any particular row of non-zero rows should be 1 or it can be any other non-zero value all the 0s will be at the bottom and correspondingly as you go down the position of the starting element of each such non-zero rows shifts to the right okay there are certain properties of this row echelon form we will see one simple example of trying of the method to reduce a matrix to a row echelon form or the row reduce form this is the example that is all so I am I am skipping this derivation here and if you do this row echelon form matrix and somebody work out and tell me the values of this row echelon form I think you will have a one starting here okay then since this then this has to be 0 so the ones could start this all of the rest could have been 0s also so that is also a valid row echelon form okay and then you have so this equivalent to say that this is equivalent to this in some form this is something like a scalar multiple because creating the row reduction form is basically multiplying the matrix in which the equivalence is established the equivalent form exists and the corresponding eigenvectors have to be found out with correspondingly say that means what I will say some v1, v2, v3 if these are corresponding components of the eigenvector this should be equal to 0 okay so actually if you see here you can exploit the first two rows of this matrix to get one constraint in which you will get from here or maybe should I let write here then okay so from this I am proceeding here towards this let me write here so you can see v1 – v3 equals 0 so I will write that how do you get this first row multiplied by this similarly and row also you will get this v2 equal to 0 so the third row does not give us any particular constraint because it is all null so based on this you can set a constraint here that let us say v3 equals some constant c1 arbitrary real value 1 then what you will have is that this will actually also lead to v1 equals – c1 or plus because v1 equal to v3 okay so you will have something like this some books will try to write eigenvectors as row as in that case they will probably give a transpose okay so that will be this will be one solution this will be one solution corresponding to lambda 1 equals – 9 okay we started with this value so corresponding to this eigenvalue so you will have any of these for any arbitrary value of c1 is what you can put here multiplied with this vector will give you the corresponding set of eigenvectors and if you apply the same process for the other two eigenvalues 6 and 3 okay we will get two of the different solutions for eigenvectors so as I say here so this solution comes from here lambda 3 equals – 9 gives the corresponding solution for the first eigenvector okay so if you want I can write here as so this superscript 1 indicating that this is the first eigenvector or the third one corresponding to lambda equal to 3 okay now similarly you the other two eigenvalues are lambda 2 equals 6 and lambda 2 equals 3 and lambda 1 equals 6 will give me the solution so lambda 1 equals 6 will actually let me look at the solution is this correct so that is the final solution so these are the corresponding three eigenvalues for this matrix and the corresponding eigenvectors are 101 actually you can take c1 to be equal to 1 okay and the other two eigenvectors are given by this corresponding proof okay and these three eigenvectors actually form a basic dimension okay we will talk about this basis vector basis and span subspaces a little bit later on but for the time just remember that this corresponding three eigenvectors are orthogonal to each other the span yeah so let us take another example of a matrix of size 3 dimension 3 and look at the eigenvalues and eigenvectors whether we get first of all whether we get a unique set of eigenvalues that is more important so let us take this example let us say so straight away we will go to this which will give us okay 5 minus lambda 0 minus 1 0 h minus lambda 0 h minus 3 0 then 7 minus lambda correct so that if you take the corresponding matrix a determinant of this matrix and write write in a polynomial form and then factor it you will get this I will give this as an exercise you should be able to take it okay maybe you should get a somebody one of my students is pointing out right away that this is it should be negatives that is what you will get okay so this gives us a shot of three eigenvalues where lambda 1 equals 4 lambda 2 equals lambda 3 equals 8 this is one way of writing this set of eigenvalues this is one way of writing this set of eigenvalues three eigenvalues the other way of saying this is that lambda 1 equals 4 is of multiplicity 1 and lambda 2 is with equal to 8 is a multiplicity 2 that means it occurs twice okay that is two eigenvalues are same the question comes is how many eigenvectors well you can expect from this now that this will give eigenvector alright and this pair of equal eigenvalues will give a another pair of eigenvectors so you should get three eigenvectors okay that is what you should do so let us try the first one a minus 4 lambda a minus 4 i that means what I have done in this expression I have substituted lambda 1 which is equal to 4 okay and if you do so if you look at this expression here what will you get tell me the first row 0 minus 1 that is trivial then 0 8 minus 4 that is 4 0 minus 3 0 3 this if you low reduce which is left as an exercise and you solve it out and tell me I will wait so what is the solution you will get first row 1 0 minus 1 last row so what we are looking is that 1 0 minus 1 0 1 0 0 0 0 which is the same as the row reduce form here with correspondingly v 1 v 2 v 3 the three components of the first eigenvector corresponding to lambda 1 1 this should be equal to 0 and this will give a constraint which we had last time what did we have v 1 equals v 3 or basically we will get v 1 minus v 3 equals 0 and then v 2 equals 0 this is the solution we had in the previous example as well so I am repeating that so you can look back into your nodes that means correspondingly for this lambda 1 equals 4 you will get the first eigenvector remember what I am using is subscript indicates the dimension and component of an eigenvector where at the superscript indicates the corresponding eigenvector for this index is the same as this for the corresponding eigenvalue. So what did we have last time as a solution v 1 1 0 1 so you can write like this if it is a row vector then you can write this as a transpose so this is the first one which is over let us start with the lambda 2 equals lambda 3 equals 8 what is a minus lambda i what is this matrix check it out very easy this will be minus 3 0 minus 1 0 0 0 minus 3 0 minus 1 good. So I am writing in short R R indicates row deduce you will get R 0's I did tell you in the row echelon form that the leading entry will be a non-zero element in most cases this is considered as 1 but it can be more than 1 as well. So this with that corresponding if you write in this form will actually give you one condition that 3 v 1 plus v 3 will be equal to 0 with no constraint on v 2 or in other words you can say that 3 v 1 equals minus v 3 that means you have to now formulate a pair of eigenvectors from this constraint which will satisfy this particular condition one of such conditions I am going to give you is the following v which will be so this is one way of writing possible solutions for this eigenvectors under the constraint you look at the condition here 3 v 1 equals minus v 3 which is the first and third dimension that constraint is satisfied here and v 2 there is actually no constraint so you set it to 0 and 1. So this 2 plus c 1 c 2 you can choose arbitrary constraints and they will give you some combination of two eigenvectors. So what you basically have now is that the first eigenvalue lambda 1 equals 4 has given you the corresponding eigenvector here. So this is v 1 and the v 2 and the v 2 and the v 3 can be chosen from here by choosing arbitrary values of c 1 c 2. So choose arbitrary values of c 1 c 2 you can choose say 1 and 1 or choose 1 and 2. So two such combinations will give you two vectors v 2 and v 3 corrects combining with this v 1 will actually give you three such eigenvectors which again spans the three dimension space. So this is a simple example of a situation where you have three eigenvalues and the eigenvectors which you get are non-specific eigenvectors but you can actually obtain them from arbitrary constraints. Let us take is this always the case let us take another example of a 3 cross 3 matrix which will have multiplicity but a different result of the eigenvectors and that matrix an example which I am taking now reads as 0 1 3 0 6 0 minus 6 2 9. Why am I taking different examples of dimension 3? We have started with an example of dimension 2 and got two eigenvalues and two eigenvectors. I am taking different examples of dimension 3 because we are getting three eigenvalues some of them with more than one multiplicity that means they are duplicated and they are giving rise to three corresponding eigenvectors. These eigenvectors may or may not span the three dimension space. We will talk about vector space soon immediately after this in which your concept of vector space subspace and this span will be clarified. So let us proceed with this and the corresponding a minus lambda i matrix will be minus lambda 1 3 0 6 minus lambda 0 minus 6 2 then 9 minus lambda and we need to take the different determinant of this matrix and what you will get is a form I will leave this as an exercise for you you should probably get this just check are you getting this again you have a multiplicity of two in one particular case we will start with this. So this will give you lambda 1 equals 3 lambda 2 equals lambda 3 equals 6 is there a minus sign here? No does not matter actually get this there is no minus sign here fine. So let us start with the first eigen value which lambda is equal to 3. So this will give us a minus 3 lambda sorry sorry a minus 3 i what will it that is a. So you will have minus 3 1 3 then 0 3 0 and minus 6 2 6 correct where it is this tell me what you will get is basically a row wise operation then 0 and all 0s at the bottom. So the corresponding eigen vector should be 1 0 minus 1 0 1 0 0 0 0 0 0 0 then of course v 1 v 2 v 3 this is equal to 0 like the process we have been doing earlier find out the eigen value substitute into the expression a minus lambda i times e is equal to 0. So this will again give you constraint v 1 minus v 3 equals 0 v 2 equals 0 this we already had again it is coming back this constraint and you know the solution by heart now I think we have done that that this will give the solution first eigen vector tell me 1 0 1 we want to put a transpose if you want to represent it is a row vector otherwise you can leave it as a column vector. So this is 1 with the corresponding lambda 1 equals 3. So lambda 1 equals 3 give us this solution here let us take the other one what is the multiplicity here 6 lambda 2 equals lambda 3 equals 6 what is the matrix from a here this is minus 6 1 3 0 0 0 minus 6 2 is this correct row reduce this. So row reduce this you will get prompt me tell me yeah this is different from the previous cases correct we did not have this form earlier. So this gives 2 v 1 minus v 3 equals 0 and v 2 equals 0 because the first row will give this and the second row will give this. So we can form the corresponding solution straight away here this is unlike the previous case which will give us what 1 0 2 correct because v 3 equals 2 v 1 you can multiply this by c 1 if you like but it does not matter we do not have v 3 in fact v 3 is also the same as v 2 same eigen vector. So this is a case where you can see that this is one eigen vector this is the second eigen vector and what we have from this eigen vector set is now subspace in three dimension. These 2 v 1 and v 2 put together because v 2 will be actually same as the v 3 now because if you substitute lambda 3 equals 6 you are going to have to this is different from the previous case when we had multiplicity here as well lambda 3 equals 8 if I have kept it on the board I have not dropped it to show you that if this is equal to 8 then we substituted got the low reduced form and then we got this constraints we did not have a constraint for v 2 in the previous example now we have it here this constraint from the second row that was not the case here. So this free constraint from v 2 forced us to write the solution in this particular form for the eigen vectors which is dependent on two arbitrary constants c 1 and c 2 which satisfy the constraint as given here and that is gave us the corresponding to free v 2 v 3 as functions of c 1 c 2 and then we had three eigen vectors that is not the case here in both case 6 for this particular matrix example we got first the solution for 3 we got this eigen vector for the corresponding multiplicity 2 here for the eigen values this is the eigen vector we have and this spans so here v 1 and v 2 spans we will say a subspace two dimension we will talk about vector spaces very very soon.