 Hi, in this lecture, let us learn an interesting and important theorem called Gashgorian theorem. Gashgorian theorem can be used to localize Eigen values of a given matrix. You may be asking why are we doing this theorem in between our discussion on power method? Well, Gashgorian theorem gives us how the Eigen values are located in the complex plane that can be used to verify some of the hypothesis of power method. Well, if you recall power method can be used to compute the dominant Eigen value and a corresponding Eigen vector of a given matrix. Well, this may be viewed as a disadvantage of power method because it cannot compute other Eigen values of a matrix when applied once. It only can capture dominant Eigen values. In fact, in the next lecture, we will see how to modify the power method to get other Eigen values also. But the fact is that when applied power method once, we get only one Eigen value and a corresponding Eigen vector. If we are directly applying the power method, then we will get the dominant Eigen value and a corresponding Eigen vector. However, it looks like a disadvantage. In many applications, we want only the dominant Eigen value of a given matrix. One good example is the well known Google search engine. Google search engine uses page rank algorithm to find the relevance of a site with the keyword given for searching. What Google search engine does is that for each site, it assigns a number called page rank score. Then it arranges all these numbers in the form of a matrix called Google matrix and Google will then compute a vector called page rank vector. The page rank vector is the Eigen vector with certain properties of the corresponding dominant Eigen value of the Google matrix. Therefore, Google does not need all the Eigen values of the Google matrix. It only wants the dominant Eigen value of the Google matrix. In that way, power method is highly preferred because the Google matrix in general can be with few billion dimensions. Therefore, if you go for computing all the Eigen values of the matrix, say for instance, if Google uses QR method which can give all the Eigen values of the matrix, then it will be very costly computationally. Whereas, power method will specifically capture only the dominant Eigen value. Therefore, power method is computationally very efficient in this case. Therefore, in fact, Google search engine uses power method. Now, we understood the importance of power method. The only disadvantage of the power method is when we want to understand the convergence of the power method, then we have to check certain hypothesis which are not practically possible to check. However, note that these hypothesis are not required if you want to just implement power method because to implement power method, we only need to give an initial guess vector which can be chosen arbitrarily. It is only that when we want to understand the convergence of the power method to the dominant Eigen value, then we need certain hypothesis. For instance, we need to know that the given matrix has a unique dominant Eigen value. And also, we need to know that the matrix has a complete set of Eigen vectors. That is, a set of Eigen vectors should form a basis for R n when the matrix is an n cross n matrix. Now, the question is can we somehow sense these hypothesis without explicitly knowing the Eigen values and the Eigen vectors? Well, the answer is may be possible. In certain cases, we can use the Gasgorian disc to conclude this. Now, I will introduce you to Gasgorian theorem through a couple of examples and then I will state the theorem. Let us first consider this 3 cross 3 matrix. First, we have to construct Gasgorian discs. Each Gasgorian disc is associated with a row of the matrix. Therefore, if we are working with a n cross n matrix, then we can construct n Gasgorian discs. How to construct that? Let us take the first row of this matrix. You have to take the diagonal element as the center of the disc and then take the modulus of all the non-diagonal elements of that row and then sum them up. That will be the radius which we will call as row 1 and the center is the diagonal element. In that way, the disc 1 which is coming from the first row is given by the set of all z in the complex plane such that mod z plus c is less than or equal to 1. Remember, these discs are constructed in the complex plane because we know that a matrix can have complex eigenvalues. In fact, Gasgorian disc theorem can localize the complex eigenvalues also. That is why we construct this disc in the complex plane. For the second disc, we have to take the second row of the matrix. Again take the diagonal element as the center of the disc and take the modulus of the non-diagonal elements of that row and sum them up. In this case, it sums to 0.35. So, you have to take these two numbers, take their modulus and then sum them. That gives you the radius as 0.35 and the center is 2. Similarly, the disc 3 is given by the set of all z in c such that mod z minus 5, 5 is the center is less than or equal to mod 0.2 plus mod 0.5. That sums to 0.7, which is the radius of the disc 3. So, this is the set of Gasgorian disc for the given matrix. Let us visualize this. The disc D1 is shown in the yellow color. The disc D2 is shown in blue color and the disc 3 is shown in green color. Now what the Gasgorian disc theorem says that all the eigenvalues of the matrix A will lie in this disc. In this case, in fact the Gasgorian disc theorem says that you will have exactly one eigenvalue. Let us call it as mu 1 that belongs to this set and another eigenvalue say lambda 2 lies in the disc D2 and the third eigenvalue say lambda 3 lies in the disc D3. They can be anywhere in the disc. Suppose if it is here, it means it is a complex eigenvalue. If it lies on the real line, it means it is a real eigenvalue. But what the Gasgorian disc theorem surely says that you cannot find a eigenvalue which is outside the disc. That is not possible. That is what the Gasgorian disc theorem says. This is what we mean by saying that the Gasgorian theorem localizes the eigenvalues. It means it just gives you a particular set in which all the eigenvalues lie. In this case, it is the union of this disc. In fact, let us see what are all the eigenvalues of the matrix A that we have taken. You can see that one eigenvalue is given by minus 2.9591 and so on. You can clearly see that it lies in this disc and the second eigenvalue that we have computed is 4.95 and so on and that you can clearly see that it lies in this disc. Whereas the third eigenvalue that we have computed is 2.0085 and so on and that lies in this disc. So, this particular matrix respects the Gasgorian disc theorem very well. Let us take another example. Here we have a 5 cross 5 matrix. Again you have to construct the Gasgorian disc. Since there are 5 rows, we will have 5 Gasgorian disc. This D1 is given like this where the center is minus 3 and the radius is obtained by summing the absolute values of these elements. Similarly, the second disc D2 has center 2 and the radius is obtained by summing 0.25 plus 0.1. The third disc is again the center is at 5 and the radius is 0.7 because you are summing the absolute values of these elements. The disc 4 has center at 3.5 and the radius as the sum of the modulus of these elements that adds to 1.35. Finally, the disc D5 has center at minus 2 and its radius is 0.5. Again let us visualize this disc. D1 is given like this, D2 is this and D3 is this disc, D4 is this and D5 is this disc. Now, Gasgorian disc theorem also says that you have precisely 2 Eigen values in this region. Why 2? Because this is the union of 2 discs. Therefore, 2 Eigen values lie in this set. Similarly, 3 Eigen values will lie in this set. That is what the Gasgorian disc theorem says. That is you can find 3 Eigen values lying in this set. Let us now precisely state the Gasgorian disc theorem. You are given a matrix A which is a n cross n matrix. First you have to find the Gasgorian disc. Each corresponding to a row of the matrix. Therefore, if it is a n cross n matrix, you have n Gasgorian disc. The radius is given like this and the discs are given like this. Now, the conclusion of the theorem is that each Eigen value of the matrix A lies in one of the discs Dk. It means what? You cannot find an Eigen value lying outside the union of this disc. That is what it means. The theorem also concludes that if you have some m number of discs whose union is disjoint from some n minus m number of discs, then precisely you have m Eigen values in the first set and the n minus m Eigen values in the second set. That is precisely what we have visualized here. You have m number of discs whose union is disjoint from n minus m number of discs. Here the m is equal to 2. So, you have 2 discs which are intersecting each other and their union is disjoint from another set of discs which consists of 3 discs here in this particular example. Therefore, you can precisely have m number of Eigen values in this set. Let us call this as R 1. So, R 1 will have 2 Eigen values in this example and this set can have 3 Eigen values. Let us call this as R 2. That is what the theorem says. Suppose among the discs, there is a collection of m discs whose union is disjoint from the union of the rest of all n minus m disc. We are calling the first set as R 1 and the second as R 2. Then the theorem says that you have exactly m Eigen values lying in R 1 and n minus m Eigen values lying in R 2. Let us prove only the first part of the theorem. Let lambda be an Eigen value of the matrix A and the and let V be the corresponding Eigen vector that we have chosen. Then you can write A V equal to lambda V. That is the definition of the Eigen vector. Also note that the Eigen vector should be non-zero. Now, let us take the maximum norm of the vector V and that is given by maximum of modulus of V 1, V 2 and so on up to V n. Let this maximum norm be achieved at the coordinate say R, where R is an integer between 1 to n. Then the Rth equation can be written like this. Why? Because I am just taking the Rth coordinate of this vector to the left hand side and writing this equation. Now, what we will do is we will take this term alone on the right hand side and keep all the other terms on the left hand side and then divide both sides by V R. That is we are taking this to other side that gives us A R R minus lambda into V R and then dividing both sides by V R. And that gives us lambda minus A R R is equal to V 1 by V R A R 1 up to V n by V R A R n. Now, take the modulus on both sides and use the triangle inequality. We get modulus of lambda minus A R is less than or equal to this expression. Now, you see V R is the greater than or equal to all the other components V i i is equal to 1 to n. That implies mod V i divided by V R is less than or equal to 1. Therefore, each of this quantity is less than or equal to 1. So, you use this property and write this right hand side expression like this. You can see that mod lambda minus A R R is less than or equal to this, where each of this term is dominated by 1 here. Now, if you recall the disc D R is nothing but the set of all z in C such that mod z minus A R R is less than or equal to the radius of the earth disc is precisely given by this expression. So, that is what we want to prove for the first conclusion of the theorem. The theorem says that each eigen value of A lies in one of the discs. How we picked up that disc is by first picking up the maximum coordinate of the vector V and then we are trying to pack lambda into that disc D R, where the maximum norm is achieved. So, this completes the first part of the proof of Gashgorian theorem. We will not prove the second part. Now, let us see how to use Gashgorian theorem to check the hypothesis of power method. Let us again take our first example, where we have taken a 3 cross 3 matrix right. If you recall we have also constructed the Gashgorian discs for this matrix. Now, let us see whether we can get some idea about the hypothesis of the power method. Recall that the second hypothesis of the power method demands that A should have a complete set of eigenvectors. That is all the eigenvectors should be linearly independent right, but by seeing the Gashgorian disc you can see that one eigenvalue is sitting here, one eigenvalue is sitting here and one eigenvalue is sitting here. Therefore, this matrix has distinct eigenvalues. Therefore, all the eigenvectors are going to be linearly independent right. So, Gashgorian theorem gave us the information that all the eigenvalues of the matrix A are in fact, distinct. Now, let us check the first hypothesis, where we need to see whether A has a unique dominant eigenvalue. First observe that the dominant eigenvalue is coming from this disc right, because this point is 4.3 right and this is 5.7 right, the center is 5. Therefore, assuming that A has real eigenvalues, because we always work with real eigenvalues. In that way, the eigenvalue of A which is coming from this disc is going to be something between 4.3 to 5.7 and that is surely going to be the dominant eigenvalue. Why? Because we have to check this condition, lambda 1 should be strictly greater than lambda i for i equal to 1, 2 and so on. Here it is only 3 right. So, to check that what you have to do is, you see whether the disc which is lying on the negative side of the plane, when it shifted to the positive side, it should not intersect the circle which has the dominant eigenvalue. That is the idea. Let us shift this disc to the positive side and see, well the disc when you shift to the positive side, it is not intersecting the disc d 3 right. That shows that whatever may be the value that is coming from this part of the interval is surely going to be different from this part as well as this part. Therefore, from here you can in fact conclude that A has a unique dominant eigenvalue right. Let us take the next example where we had a 5 cross 5 matrix and here we have seen that the Gashgorian discs are given like this. You can clearly see that we cannot use Gashgorian disc theorem to check the hypothesis of the power method. Why? Because some of the discs are intersecting. For instance, the dominant eigenvalue is coming from this disc and it is intersecting this disc also. Therefore, it may happen that there is a eigenvalue with algebraic multiplicity as 2 or even 3 and also when you shift this disc to the other side that is when you take modulus of the eigenvalue which is sitting here, it may also coincide with the dominant eigenvalue right. Therefore, Gashgorian disc theorem cannot be used to verify the hypothesis of the power method. It does not mean that A is going to violate the hypothesis of the power method. It is only that the Gashgorian theorem cannot be used. Now, what is the idea should we give up or is there any other way to check the hypothesis of power method through Gashgorian theorem? Well, there is a nice idea that why not we apply the Gashgorian theorem to A transpose. Why? Because the eigenvalues of A and the eigenvalues of A transpose are one and the same. Therefore, we can also apply the Gashgorian theorem to A transpose and see if that gives us a good information. If so, then that can be borrowed to conclude how the eigenvalues are going to be located for the matrix A itself right. So, in this case what we have to do instead of taking the radius along the rows, you have to now take the radius along the columns that is take the absolute values of the non diagonal elements of the columns. Let us see in that way how the Gashgorian disc for A transpose are looking like. You can see that the disc D 1 is given like this. Again you can observe that the centers of these discs are not going to change, only the radii will change. For the first disc now you have to sum along the columns right. In the previous case we took the sum along the radius row and therefore, we had the sum as 1 for the radius of the first disc. Now the first disc will have radius 0.551 right. So, that is the advantage in this particular example it may be that A transpose may give a bad information then A sometime, but in this case A transpose is seems to be giving a better information than A. Similarly, for D 2 again you have to take the diagonal element 2 as the center of D 2 and then now the radius is computed along the column elements. In that way we got 1.3 for the disc 2 and similarly disc 3 has radius 0.55 again and the center is at 5 and similarly the other 2 discs. Let us see how they look like in the complex plane. You can see now that all the discs for A transpose are disjoint. Therefore, you can have the hypothesis of power method verified from the Gashgorian theorem applied to A transpose. You can in fact shift these 2 discs to the positive side and see whether they are intersecting the discs from where you get the dominant Eigen value. Note that the dominant Eigen value is coming from this disc right. Let us see how they look like when I shifted these 2 discs from the negative side to positive side. Why we are doing? Because the dominant Eigen value condition is checked with modulus of the Eigen values right. It goes like this. So, therefore, we have to check whether the absolute value of the Eigen values are going to coincide with the dominant Eigen value. That is why we are shifting this this to the positive side and now you can see that even when you shift them to positive side they do not intersect the discs from where you get the dominant Eigen value. Therefore, the Gashgorian theorem applied to A transpose gives us a good information and it tells us that the power method is going to converge. If at all you choose your x naught correctly again that is not a problem because what you need is that you have to choose your x naught such that c 1 should not be equal to 0. If you recall c 1 should not be equal to 0 what is c 1? c 1 is nothing but the coefficient of the first term in the representation c 1 v 1 plus up to c n v n right. This is not something very serious because we may choose c 1 v 1 plus 2 or 3 vectors and run the power method. If all these are going to converge to the same Eigen value then it is very likely that that is going to be dominant Eigen value because it is very unlikely that you will choose x naught 3 times for which all the 3 vectors will have 0 c 1 right. So, that may be very unlikely. In that way this is practically not very serious hypothesis whereas, the other two are really serious concern to check and Gashgorian theorem gives us a possibility that we may verify those hypothesis without having explicitly the idea about how the Eigen values are. Let us also state the Gashgorian theorem on A transpose. So, there is nothing new here you just have to apply the radius column wise now that is the only difference in this theorem when compared to the theorem that we have stated previously otherwise the conclusion remains the same. Well, before ending this lecture let us have another interesting application of the Gashgorian this theorem. If you recall in the last class we have studied the convergence theorem for iterative methods in particular for Jacobi and Gauss-Seidel method. We had given a necessary and sufficient condition for the convergence. What is that? That is using the spectral radius of the iterative matrix B that is the spectral radius of the iterative matrix should be less than 1. This is the necessary and the sufficient condition for the convergence of an iterative method right. What is spectral radius? Well if you recall the definition of spectral radius is nothing but the maximum over all the Eigen values lambda i right. In a way you can see that the spectral radius is nothing but the dominant Eigen value of the matrix B. Therefore, you can in fact compute the spectral radius using power method. If your purpose is to only see whether the Jacobian method or the Gauss-Seidel method is going to converge or any iterative method is going to converge or not then even you can go for checking the Gashgorian theorem. How you will do? For instance let us take the Jacobi method. The iterative matrix for the Jacobi method is given like this. You can see that all the Gashgorian disks of this matrix are having their centers as 0. In fact, Jacobi method for any system will have the diagonal elements as 0. You can verify that in this case what you have to do is you either go row wise and get the disks are also you go by column wise and see the disk whether all the disks are having their radii as something less than 1. If that is something less than 1 it means what all the Eigen values are going to lie in this disk right and if you know that the largest disk has its radii something less than 1 it means the spectral radius which is the dominant Eigen value will in the worst case be sitting somewhere here and that will be strictly less than 1 right. So that is the idea. Let us see how the Gashgorian disk is placed for the matrix BJ. You can see that the lost disk has radius more than 1. Therefore you cannot conclude the convergence of Jacobi method using the Gashgorian disk applied on BJ. Let us see whether we can conclude the convergence by applying Gashgorian disk on BJ transpose. Well you can see that when you take the disks with BJ transpose the radius of the first disk is the sum of these two numbers which amounts to be around 0.757 and the second disk as the radius something very near to 0.9 but still it is less than 1 and the third disk is pretty small. Its radius is only around 0.4. Therefore the Gashgorian theorem when applied to BJ transpose tells us that the Jacobi method is surely going to converge. In fact you can use the power method to compute the spectral radius also. In this case the spectral radius of BJ is 0.63825 and so on. Well the convergence may be little slow but still the Jacobi method converges. Without going to the explicit calculation of the Jacobi iteration we can understand either by using power method or more easily using the Gashgorian disk theorem we can conclude this. Let us go to see whether the same exercise can be done for Gauss-Seidel method. Well in the Gauss-Seidel method the Gashgorian disks for the iteration matrix BG is given like this and you can see whether you take row wise and apply the Gashgorian disk theorem or you take the disk column wise you can see that all the disks have radius something very less than 1. In fact you can see that the spectral radius of the iterative matrix of the Gauss-Seidel method is very very small that in fact shows that the Gauss-Seidel method in fact is going to converge very fast to the solution of this system. With this note let us end this lecture. Thank you for your attention.