 In the previous two lectures, we have considered two ways of finding two-tapel similarity transformation for solving the algebraic eigenvalue problem. In this lecture, we will briefly discuss the other two. The first that we discussed now is the method based on matrix decomposition or factorization. The most important matrix decomposition for eigenvalue problem is the QR decomposition. In these two chapters of the book, you will find that there are quite a few sections with asterisks. These star marked sections are a little advanced. So, we will not cover these topics in depth and rather we will try to concentrate on methods so far as its use is concerned. First, the idea of QR decomposition. The decomposition itself is quite straightforward. The QR decomposition is the decomposition of a matrix into two factors Q and R in which the first one the Q factor is orthogonal and the second one the factor R is upper triangular. The good point about QR decomposition is that it always exists. Whatever matrix you give, one can decompose it into Q and R factors. So, as long as you give a square matrix, the Q and R factors will be square. So, for every square matrix, you have a QR decomposition. So, it will always exist. Next, performing this decomposition is pretty straightforward. In fact, if you have come to this point through the exercises in the previous chapters of the textbook, then you would have already decomposed one matrix into Q and R factors. In chapter 4, one very simple QR decomposition problem was said for you as an exercise in which the steps for the decomposition were also suggested. And the third important issue in QR decomposition is that it has a number of properties which are extremely crucial for the solution of eigenvalue problem. First, we see how we conduct the QR decomposition of a given matrix. Say, this is the matrix A in which A 1, A 2 etcetera are columns. So, this is to be decomposed into Q and R factors. This matrix is orthogonal. That means, its columns Q 1, Q 2, Q 3 are all unit vectors which are mutually orthogonal. And this is an upper triangular matrix. Below the main diagonal, everything else is 0. Now, first we quickly review the same process of affecting this decomposition which was suggested in the exercise of chapter 4. This method is based on the procedure of Gram-Schmidt orthogonalization. And Gram-Schmidt orthogonalization process itself is actually another very simple algorithm which was introduced even earlier in the exercise of chapter 3. Now, here we again just like our normal decomposition processes, we try to determine these decomposed parts through term by term multiplication. So, first we can write A 1 as the first column of this matrix product. And that will be this matrix multiplied with the first column. Now, that will give us R 11 into Q 1 plus everything else is 0. So, that means we get R 11 Q 1 as the first column of the matrix A. Now, the matrix Q is supposed to be orthogonal. That means this vector column vector Q 1 should be a unit vector. And that will mean that the entire magnitude of this vector A 1 should be R 11. So, what we do? We take the norm on both sides. And we find that the norm of A 1 is R 11. Once we find R 11, then we can divide this vector A 1 by R 11 that gives us the unit vector Q 1. Precisely this step you will find when you try to conduct this whole thing for j equal to 1. This part is missing for j equal to 1 because there is only one term in the sum and there is no previous i. So, this term is this part is missing. You start from this part. Here also this sum is completely missing and you have this A j itself A 1 itself sitting in the place of A j prime. And then R 11 is its norm as I just now mentioned. And when you divide that column vector A 1 with R 11 you get Q 1. After that you equate the second column. Second column A 2 is R 12 into the first column here plus R 22 into the second column here. Other terms are 0. So, from here the actual work starts. So, R 12 into Q 1 plus R 22 into Q 2 is the second column. You do not know anything on this side except Q 1. On this side you know A 2. Now, if you know Q 1 and you also know that Q 2 is supposed to be orthogonal to Q 1. Then both sides if you take inner product with Q 1 that is you multiply this both sides of the equation with Q 1 transpose. Then here Q 1 transpose Q 2 will become 0. And here you will have Q 1 transpose Q 1 is 1. And then what will remain on this side is R 12 and that is Q 1 transpose A 2. That is what you find here. For j equal to 2 only i equal to 1 is possible and R 12 you get as Q 1 transpose A 2. After you have found that you can subtract that from A 2. And in this particular case this summation will include a single term this Q 1 transpose A 2. After you remove that is this R 12 into Q 1 this part R 12 Q 1. Now that you have got R 12 R 12 Q 1 you subtract from A 2. What you are left with is this that is demarcated here as A 2 prime. And if you are left with this thing only then its norm will give you R 22. So, its norm will give you R 22. Once you have found R 22 then you can divide the remaining thing A 2 minus that stuff with R 22 and you get the unit vector Q 2 that is here. This way you keep on proceeding. So, in the next step you will find R 13 R 23 and then the two terms subtracted from A 3 will give you R 33 into Q 3. Its mod will give you R 33 and when you divide with R 33 you will get Q 3. The trouble will arise if you find that R 33 or R k k at the k th point if R k k turns out to be 0 then what then you cannot divide this. Still R k k equal to 0 is a valid output which you will place in this location wherever in the diagonal entry it is supposed to come. That will signify that the given matrix is singular. Apart from that so far as the question of determining Q k is concerned it is not a problem at all. Because Q k in the sum for A k is actually not making a contribution because it is supposed to get multiplied with 0 to give the contribution. The trouble started because this entire vector A j prime turned out to be 0. At that stage you cannot divide it like this and therefore, you cannot get Q k by this formula. But that does not mean that the process stops there. This R j j or R k k being 0 only means that Q k is left unconstrained. Q k expression is not available from here and that means that as long as Q k satisfies that column satisfies the orthogonality requirement it is fine. So, what you do till now whatever columns of matrix Q you have found Q 1 Q 2 Q 3 up to Q k minus 1. Any new unit vector which is orthogonal to all these k minus 1 old column vectors is acceptable as Q k. So, that means that if R j j is non-zero then this is a constraint based on which you determine Q j. If R j j is 0 then that constraint is removed and in that situation any vector satisfying this orthogonality requirement is acceptable for Q j. With this process till the end you can go and decompose the matrix into Q and R factors. Now, this is a very simple straight forward method by which you can effect the Q R decomposition. However, I should mention for record that the sophisticated or practical methods which are utilized in most of the computational algorithms is actually not this. Though this is a valid method, but actually the Q R decomposition is affected in most professional programs is through householder transformations. The same householder transformation which we studied in the previous lecture, but with a little difference. Here there are two small differences compared to the way we applied householder transformation in the previous lecture. In the previous lecture we were using householder transformation matrix on both sides left as well as right because we were concerned with similarity transformation. Right now we are concerned with actually factoring a given matrix into two factor Q and R. We are not yet so much concerned with a similarity transformation. We will be talking about similarity transformation based on this a little later, but right now our focus is on affecting a Q R decomposition. Therefore, there is no compulsion for us to multiply on both sides to maintain the similarity. Our main focus is to effect a Q R decomposition is one point. Second point we arrive at when we consider this situation that we apply a householder transformation on a given matrix in which we do not consider this much as u, but we consider this whole thing as u. If we consider this whole thing as u and work out the corresponding v as we did in the previous lecture, then what will happen is that v will have its first entry same as the norm of this entire vector from top to bottom and all other entries below that will be 0. That will mean that contrary to the case of the previous lecture where this much of the first column was converted to 0 was annihilated. Now this much will be annihilated in the first step. That is why here I am saying not u 1, but u 0. So, u 0 is the entire first column. v 0 is a matrix of the same dimension a vector of the same dimension with only the first entry being non-zero and having a magnitude same as the norm of u 0. And based on that we work out w 0 and then h n and therefore, p 0 in this manner and when we apply that only on the left side not on the right side, then from here to here all of these become 0. And next we will apply p 1 based on h n minus 1 to be operating on this much to get everything below a to 2 to be 0 and so on. Note that the same thing we could not do in the previous case when we were trying to affect similarly transformation because the way the left side multiplication with the householder transformation matrix affect these entries and these entries over the full thing. So, the similar transformation in the previous case would require us to multiply the householder transformation matrix on this side also. That would spoil the 0 set in the left side multiplication and that is why in the case when we were using householder transformation for tridiagonalization, then we did not want the already established 0 to be spoiled by post multiplication. That is why there we left one extra term free and applied householder transformation with one dimension less. But here we are not going to multiply the same matrix from this side we are applying the householder transformation only one sided. So, we can take the full vector and therefore, under the diagonal all terms we can make 0. So, with one step we with the u 0 and therefore, with p 0 we get the sub diagonal entries in the first column 0 and then we get this. The entire magnitude of a 1 the first column sits here below that everything else is 0 and then p 1 is applied which will make a 3 2 a 4 2 a 5 2 etcetera up to an a n 2 all 0 and so on. That will look like this here there will be some non 0 items and so on. So, then we apply p 2 to get everything under a 3 3 r 3 3 as 0 and so on. So, like that by the time we come up to p n minus 2 then we have got a completely upper triangular matrix. Now, note that all these transformations from p 0 up to p n minus 2 are orthogonal matrices and therefore, this entire product p n minus 2 to p 0 is orthogonal product of orthogonal matrices is also an orthogonal matrix. And now if we call q as the transpose of this whole thing which is this then whatever we have p multiplied can be called q transpose. So, what we have got? We have got q transpose a as r upper triangular matrix which means a is equal to q r and q we have got stored by cumulative multiplication of the intermediate householder transformation matrices. So, this is one practical method of affecting a q r decomposition. There is one more based on the given rotations through one sided plane rotations. Now, before going into the application of q r decomposition in Eigen value problems let us quickly have a look quick look at a side issue. If we already have the q and r factors of a matrix then even in the solution of a linear system of equations a is equal to b it can be utilized because in place of a if we write q r with q and r factors already available in our hand then first step is free multiplication with q transpose that will mean r x equal to q transpose b. And transposing a matrix and multiplying that way vector is a task of very little computational effort after that what remains is r x equal to a known vector r being an upper triangular matrix that means only a sequence of back substitutions will give us the solution. The solution is actually extremely cheap if we already have q and r factors in hand, but otherwise in order to solve this problem a is equal to b typically we would not going to the process of q and r q r decomposition method because other methods that we have studied earlier in the course are comparatively better. The actual use of q r decomposition algorithm is in solution of Eigen value problem let us study the decomposition in that life. First point is that if we multiply the q and r factors in reverse this is a very weird proposition to begin with. We have decomposed a matrix into q r factors and the q and r appear q first r next and now we are suggesting that let us see what we get if we multiply them in reverse r q it is a weird proposition to begin with, but if we do that then we notice something interesting. Just now we have seen that r is q transpose a because a is q r so r is q transpose a. So if in place of r we write q transpose a then see what we have got q transpose a q wow what we have got is a similar transformation that means that if we factor a given matrix a into q and r factors and then multiply them in reverse then what would happen is that we would have we would have a matrix new matrix a prime which is the result of a similarity transformation over a itself the similarity transformation matrix being the same q. So this is a similarity transformation not only that it is an orthogonal similarity transformation because q is orthogonal with a few interesting properties. First is that if a is symmetric then a prime is also symmetric that we can see because we have multiplied from both sides q and q transpose so that preserves the symmetry. Second is if a is in upper Heisenberg form then a prime is also in upper Heisenberg form that you can see very easily because in the factorization of a into q and r factors q 1 the first column of q q 1 must be in the direction of a 1. Now if the original matrix a is in upper Heisenberg form which is this that means if the first column of the matrix a has only the top two terms then first column of q also will have only top two terms below that everything else is 0. Next the second column of a has three terms but then that is the linear combination that is the linear combination of the first two columns of q. Now first two columns of q through a linear combination is giving us the second column of a and the first column is only having two top terms so the second column can have only three top terms which are non zero and so on. So that way considering that way we find that q also will have the same shape as a if a is Heisenberg. So if a is Heisenberg then q is also Heisenberg. Now in the reverse multiplication what is happening is that we are multiplying r q r q it is this. So that means this matrix the first column of this matrix having only top two entries will mean that the first column of the product will have only top two entries because the this matrix multiplied with the first column gives us the first column of the product. So this number into first column plus this number into second column and first column has only one entry and second column has only two entries at the top which are non zero. So the composition will have only two top two top entries as non zero. Next the second column of the product will be these three things into these three columns and these three columns have up to only third entry. So the second column will have only top three entries which are non zero and so on. So that way you will find that if the original matrix is Heisenberg then so is q and so is r q. So that means if a is in upper Heisenberg form then so is a prime and now symmetry preservation and upper Heisenberg form preservation together will mean that if the original matrix is symmetric tridiagonal then the new matrix is also symmetric tridiagonal. So till now we have found through these properties that through this transformation that is factoring q r and then pre multiplying in the reverse order r q nothing is going wrong that is if we have given a matrix to begin with we have got a similarity transformant form of it. If we have given a symmetric matrix then we get back another symmetric matrix. If we have given an upper Heisenberg matrix we get back an upper Heisenberg matrix. If we have given a symmetric tridiagonal matrix then what we get back is also a symmetric tridiagonal matrix. That means in the form and feature of the matrix nothing is spoiled nothing is lost but the question is what is gained and in that we find that there is a deep result which shows which establishes that if we follow this algorithm in one iteration not much will take place in one step like this not much will take place but the advantage accrues when we follow this repeatedly. What is the algorithm? Algorithm is very simple set a 1 is equal to a and then for k equal to 1 2 3 4 5 6 go on doing these two steps decompose a k into q k r k factors and then multiply in reverse call it a k plus 1 r k q k. So algorithm is just two step algorithm factorize multiply in reverse. What you get factorize that multiply that in reverse and so on. So from a 1 through these two steps you get a 2 again from that through these two steps you get a 3 and so on. You go on doing this and the deep result assures you that as k tends to infinity that is as you go on doing this step again and again and again and by you reach a convergence situation where the matrix a k approaches a quasi upper triangular form not really upper triangular but quasi upper triangular form and this happens for any kind of matrix. Now what is this quasi upper triangular form? It is this form quasi means almost. So in this quasi upper triangular form it is almost upper triangular except for some diagonal block that might remain which is spoiling the diagonal feature diagonal form a little because some of its elements may be below the diagonal and sometimes this kind of 2 by 2 diagonal blocks may come. And in this quasi upper triangular form there is another interesting feature that the largest magnitude Eigen value gets organized here next here next here and so on. So the magnitudes of the Eigen values that you get here along the diagonal are in decreasing order. So whenever there is a convergence to upper triangular form you know that Eigen values will appear in the diagonal entries but Eigen values get sorted also. Now such diagonal blocks actually correspond to Eigen spaces of Eigen values which have almost equal or very close magnitudes. The decoupling the dissociation of subspaces that is actually behind this arrangement does not work so efficiently in the case of Eigen values which are which have extremely close magnitude. That is why sometimes such diagonal blocks may appear 2 by 2 diagonal blocks can appear when there are complex Eigen values complex pair of Eigen values and that will happen only for non symmetric matrices obviously. So for such cases this kind of blocks may remain which do not break further because for breaking further complex arithmetic needs to come into play to get complex values here. As long as we operate with real arithmetic this kind of 2 by 2 blocks will remain for non symmetric matrices if there are complex pair of Eigen values. In particular for symmetric matrices a quasi upper triangular form actually will mean a quasi diagonal form because for a symmetric matrix we have seen that symmetry is preserved. So all these zeros will also mean that here also we have got corresponding zeros. So we will get actually quasi diagonal form for a symmetric matrix and such things are any way not possible for symmetric matrices. So this is the form to which the q r iterations will make the process converge. And finally after we have got this kind of a form Eigen values are mostly available except for this kind of situations where we may need to conduct a little further post processing. Now I have told you what happens but how this happens this I have not told you. In my teaching typically in mathematics teaching particularly I try to establish most of the results. But this particular result I will not try to establish in the class rather I will tell you what actually happens behind because this is very time consuming. And it is important that when you get ready for appreciating the actual process that is happening behind you should actually conduct the analysis on your own possibly guided with the book. So if you are ready for going deep into it then I will suggest that you go through the section in the book and try to work out the whole thing on your own then you will get better confidence. In a nutshell what actually happens in the background in q r method in q r decomposition method is that it operates on the basis of relative magnitude of different Eigen values. Almost the way the power method used to work but here the proved operation of power method is actually conducted in a much more sophisticated manner and at a global level. At every level the same refinement of subspaces based on the magnitudes of their Eigen values and the segregation of Eigen spaces of small Eigen values from the large ones takes place at all levels. So as iterations proceed this separation takes place over the entire spectrum of Eigen values and that is how the Eigen values get sorted Eigen vectors get arranged in suitable subspaces. The way power method works on a single vector here the q r decomposition method at the same time operates over a multitude of Eigen spaces and inside each Eigen space it works for finer adjustments, finer sorting and so on all these processes take place together. And this is the reason why the diagonal blocks it cannot break down completely if quite a few Eigen values are of almost equal magnitude that is the only limitation. So if you follow through this conceptual basis then you will find that towards the end you get subspaces which are sort of separated from each other based on their Eigen value magnitude. There is another important thing in relation to it is that the rate of convergence depends rate of segregation of different Eigen spaces is actually directly related to the ratio of Eigen values lambda i by lambda j. And this observation you can make if you go through one actual process of q r decomposition for that what you can do is that you can take a square matrix and write a small procedure to effect the q r decomposition perhaps based on the method that we discussed earlier. And then actually apply this algorithm and at the end of every iteration you see the result intermediate result then you will see how the subspaces are getting separated. Based on this observation of convergence rate there are also some sophisticated algorithms which try to shift the matrix shifting all its Eigen values leftward and rightward in order to artificially decrease this lambda i minus lambda j which will mean that if this is very small then over successive iterations the errors will drop at very fast rate. So such an algorithm is called q r algorithm with shift it is a little complicated and that is why we will not be discussing it here. But that is what is professionally implemented in most of the professional software library routines. But we will not go into this what we will do rather is that just like this brief introduction of the q r decomposition method for general matrices in general and for symmetric tridiagonal matrices in general particularly we will consider the last topic also the fourth method of effecting suitable similar transformations. Now one point that needs a special mention is that q r decomposition algorithm will operate on any matrix and try to give this kind of a quasi-oper triangular form. It operates on all matrices it is effective but is it efficient when we ask that question we need to notice that implemented properly the q r decomposition algorithm the iterative algorithm is a linear time algorithm for symmetric tridiagonal matrix a quadratic time algorithm for an upper Hessian work matrix and cubic algorithm for the general matrix. This is why we say that if you have got a symmetric matrix then it is very important that you first reduce it to tridiagonal form and then apply q r decomposition if you want to apply even though on the full matrix also q r decomposition method would work. Similarly in the case of a non symmetric matrix it is better to reduce it first to upper Hessian work form and then apply q r decomposition. This is the issue which will be considering next. The eigenvalue problem of general matrices that is non symmetric matrices is relatively much more complicated and therefore we find that the theory is much more involved and computational algorithms are a limited and b relatively unstable compared to the orthogonal transformation based method for symmetric matrices. Therefore in this course we will not go deep into those theories rather on these star marked sections will very briefly gloss over and try to discuss the basic points here and the important post processing issues here. First point regarding non symmetric matrices is that it may not be diagonalizable effect that we have discussed earlier a symmetric matrix is always diagonalizable but not so for a non symmetric matrix. So therefore rather than trying to diagonalize the given matrix if it is non symmetric what we try here is first to triangularize it and get an upper triangular form. Next point is that with real arithmetic we cannot avoid 2 by 2 diagonal blocks that means even if the matrix is actually triangularizable and that also with unitary transformations if we stick to real arithmetic and do not go to complex arithmetic then even triangularization process will not be really complete because 2 by 2 diagonal block will remain which will signify complex pair of eigenvalues. So whenever complex pair of eigenvalues are there such blocks are inevitable. Next is that the computational complexity of the algorithm is higher convergence is low and sometimes the numerical process may turn out to be unstable. These are the typical difficulties with non symmetric matrices and what we say is a non symmetric matrix is usually unbalanced. You see if the norm of kth row and kth column of a matrix is same for every k then you call that matrix as balanced. A symmetric matrix to begin with is balanced and through orthogonal transformations it remains balanced. A non symmetric matrix is usually unbalanced and therefore it is prone to numerical round off errors more than the symmetric matrix algorithm. Therefore in the case of non symmetric matrix is as a preprocessing step we sometimes use balancing we try to balance the given matrix a little and one way to balance it is that if a row is of much higher norm than the corresponding column then we can try to adjust that by multiplication of the row with some number and division of the corresponding column with the same number so that we can balance their norms little bit. Now, only multiplication to a row is not correct is not the right step to do because that will be a transformation which will not be a similar transformation. If we multiply a particular row with half and the corresponding column with two then that will be a similar transformation because the multiplication of the kth row with half is equivalent to a pre multiplication with an elementary matrix the corresponding inverse as post multiplication matrix involves the multiplication of the corresponding column with two and so on. So, balancing is a preprocessing step for non symmetric matrices. Now, through the similarly transformations which are orthogonal a balanced matrix does not go unbalanced if the similarly transformations that we apply if they are not orthogonal then even in originally balanced matrix may go unbalanced again. Now, what we do with the matrix in order to triangularize it. So, there are three possible alternatives for the first step for a non symmetric matrix. The first step for a non symmetric matrix is to reduce it to upper Hessenberg form which you can do with a fixed number of arithmetic operations without any iterative process and then further we will conduct iterations on that upper Hessenberg matrix. So, the reduction to Hessenberg form can be done in several different ways one is a full sweep of given rotation as we have discussed earlier another is a sequence of n minus two steps of householder transformation which also we have discussed in the previous lecture. Now, these two things one of the two each of them conduct converts a symmetric matrix into symmetric tridiagonal form. The same operations in the case of a non symmetric matrix will convert it to the upper Hessenberg form the zeros under the sub diagonal will be arrived at, but that will not mean that over the super diagonal also you will get zero that you will not get in general. Apart from these two for a non symmetric matrix you can also apply a third method which is a cycle of coordinated Gaussian elimination. To make the record state this kind of a thing is possible for a symmetric matrix also, but in the case of a symmetric matrix this we will typically not do because in the symmetric matrix case the orthogonal transformation based algorithms are superior. Here there is no symmetry to preserve and therefore, it really does not harm to take the third option. In the third option what we do we apply Gaussian elimination in a well coordinated orchestrated manner. Note that the straight forward Gaussian elimination which we were applying earlier for solving a linear system of equations that was only based on rows only row operations we were doing that same thing we cannot do as it is here because every step every transformation of the matrix has to be made through a similarity transformation that means whatever matrix we pre multiply with we need to post multiply with the corresponding inverse and therefore, the Gaussian elimination here must be coordinated. So, this Gaussian elimination or elementary transformations which are applied in a coordinated manner are actually applied from two sides one through a pre multiplication and the other through a post multiplication. That means the pre multiplying matrix corresponding to the elementary row transformation and the post multiplying matrix corresponding to the matching column transformation must be inverses of each other that is how the similarity will be maintained. So, there are two kinds of steps in these elementary transformation as always one is the pivoting and the other is the elimination. So, in the pivoting step typically you try to interchange rows and accordingly interchange columns also and in the elimination step typical pre multiplying factor will look like this which will mean subtraction of some multiple of the pivotal row from the lower rows that is why minus k here is a vector and the corresponding column transformation here which will be equivalent to appropriate addition of all the later rows all the right side rows to that pivot right side columns columns into the pivotal column. This will actually compose the elimination step through such pivotal through such pivoting and elimination steps we convert it into the Hessenberg form first and then we apply the q r algorithm on the Hessenberg matrix and in that process as we keep on applying the q r iterations as soon as a sub diagonal zero appears we split the matrix into two parts and continue with the smaller part and so on. And if towards the end a smaller part turns out to be a one by one part then that itself is an eigen value if the smaller part turns out to be a two by two sub matrix then from that through a quadratic equation solving we determine two eigen values and so on. So, like that we keep on decomposing into smaller diagonal blocks and solve the eigen value problem in terms of only eigen values to begin with. After that is over then we go into a method which is called inverse iteration. This is a very valuable method because at the end of the reduction up to upper triangular form all that you have is only eigen values and not eigen vectors and quite often it may also happen that the eigen values are also only crude estimates. In such a situation when you have got an estimate of the eigen values not their exact values then to refine the values of the eigen values and the corresponding eigen vectors to get the values of the eigen vectors inverse iteration comes as a very good method. For that the assumption is that matrix A has a complete set of eigen vectors and lambda i 0 is a good estimate of the ith eigen value of a lambda i. So, lambda i 0 is an estimate of lambda i. In that case what you do is that in order to find the eigen value lambda i precisely and to determine the corresponding eigen vector you first make a guess of any random vector y 0 with magnitude 1 and solve this system. As you solve this system you find that the solution y is a very good estimate of the eigen vector v i. This is very interesting the reason is that if we are constructing this matrix A minus lambda i 0 i that will mean that all eigen values of this matrix get shifted by lambda i 0. If lambda i 0 were an exact estimate of lambda i then this would be a singular matrix and in that case as it is you would not be able to solve it. But if it is a good estimate but estimate nevertheless not exact value and that will mean that through this shifting the ith lambda, ith eigen value becomes of very small magnitude and that means in the solution which is basically y that is this matrix inverse into y 0 you get that particular eigen vector over emphasize because the eigen value which is extremely small in the inverse the corresponding eigen value will be extremely large. So, that eigen vector gets over emphasize and then therefore in y you will find that the components of the other eigen vectors become very less in comparison to the eigen vector corresponding to which we have made an estimate here. And y is a good estimate of the corresponding eigen vector and through a little further processing you can see that the difference of the correct value and estimate will be given by this. So, therefore the expression here gives you an improvement in the estimate of the eigen value and then that estimate you can utilize here and continue this inverse iteration. Now, this is a very peculiar algorithm in the sense that here you cannot complain of ill conditioning in the solution process of this linear system because it is that ill conditioning itself that is actually driving this algorithm that is because of that ill conditioning your intended eigen vector is actually be getting expressed is getting emphasize in the solution process and you are able to identify it. Not only that in the later iterations towards the convergence this matrix will become even more and more ill conditioned because lambda i estimates will become more and more resize. So, this algorithm is tricky and we have to handle it carefully the algorithm is start with an estimated eigen value and guess y 0 which is normalized with magnitude 1 and then every iteration you solve the system find and normalize the next y y 0 in place of y 0 then it will be y 1 y 2 y 3 and so on next vectors to be used here in this system. So, normalize this and make the improvement over the eigen value and then if the previous right side and the new right side are extremely close to each other then terminate that is the convergence. So, this is the algorithm which gives you refinement over the eigen values and the values of the eigen corresponding eigen vectors. So, this can be done as a post processing step after you have triangularized the matrix this can work for symmetric as well as non symmetric matrices. Now, the important issues here is that every time that you update the eigen value the coefficient matrix actually changes that may mean the solution of the system may turn out to be more and more costly. So, one way to handle this is to update the eigen values only once in a while not at every iteration. Next a small artificial number may need to be used as an artificial pivot while solving this linear system because as we were discussing earlier this matrix is known to be conditioned. Therefore, when you try to pivot in the linear system solution you will have difficulty in pivoting because all the entries all the candidates for pivoting will turn out to be extremely small. And therefore, a tiny number of your choice you can insert there as an artificial pivot rather than the 0 that might appear otherwise. The point is that in the beginning we said that this works under the assumption that the matrix has a full set of an eigen vector. If the matrix is defective or if it has complex eigen values in that case you know it may not converge as easily as you would expect because the underlying theory expects the expression of the chosen vector in the form of a sum a linear combination of eigen vectors. And that may not happen that may not be possible in case of a defective matrix and then in case of complex eigen values the pair of complex eigen values are both of the same magnitude. And therefore, for that eigen value the situation may be tricky because the magnitude only one magnitude does not get expressed equally high I mean unequally high compared to others because two eigen values are of the same magnitude at least. Even repeated eigen values or extremely close magnitude eigen values will similarly inhibit the process. Now, with this much we complete our discussion on the eigen value method and at the end we need to have a quick look at the entire spectrum of methods that we have studied till now in the last five vectors. And make a sort of choice on the kind of methods that we typically utilize while solving eigen value problems of different kinds of matrices kinds and sizes. There are three parts three columns here one is reduction kind of preprocessing then the actual main algorithm and then the post processing. For general matrices if the size is extremely small up to four then you do not need any sophisticated method you can reduce it based on definition itself get the characteristic polynomial. And the algorithm is actually simply polynomial root finding and after that you solve the linear systems to get the eigen vectors. For intermediate size matrices say four to twelve you can either use Jacobi sweeps to reduce the matrix to an extent and then rather than using further sweeps you can use selective Jacobi rotations to diagonalize it completely no post processing required. Alternatively for this range of size of the matrix symmetric you can first try diagonalize by given rotation or householder method and then you can use stream sequence property and bracket and bisection use bracketing and bisection to find the rough eigen values. Once the rough eigen values are determined then you can use inverse iteration to refine the eigen values and determine the eigen vectors. If you have symmetric matrices which are large say larger than twelve by twelve size then reduction must be carried on for the sake of efficiency up to try diagonalization here usually you will use householder method because that is much more efficient compared to given rotation for larger matrices. And then the main algorithm will be typically your decomposition iterations for symmetric try diagonal matrices. If you are working with non symmetric matrix with intermediate or large size then first you will reduce it through balancing and then to the Heisenberg form either these methods or through Gaussian elimination based methods and then you can use your decomposition iterations to triangularize the matrix and get the eigen values and finally you can use inverse iteration to refine the eigen values and determine the eigen vectors. For general matrices symmetric as well as non symmetric if the matrix is very very large but the requirement is only selective that is if you want to determine only the top and bottom eigen values or you want to determine top few eigen values in such situations you can use power method shift and deflation to pick up only those pieces of information which you need. So, with this we complete this small some module module of the algebraic eigen value problem earlier we studied the first some module of linear algebra that was systems of linear equations. Now in the next lecture we will combine these two and see the connections between these two problems into the one of the most important lessons of this course which is singular value decomposition that we will discuss in the next lecture. Thank you.