 So the last time we looked at some interlacing theorems. So that concludes this chapter four of the Hohn and Johnson textbook. Today we will start with chapter six, which has to do with location and perturbation of eigenvalues. So if you recall, we looked at, we've already studied a little bit about the condition number and its relationship with the sensitivity of solutions to linear systems and the sensitivity of inverses of matrices to perturbations in these matrices. So, and vectors B, that is the right hand side of a linear system of equations. So also of interest is the question of how sensitive are the eigenvalues and eigenvectors of a matrix to the perturbation of its values? So to take a simple, so consider a matrix A, zero, zero, zero, zero. And say this is 10 power four, zero, 10 power four, zero, zero, zero, zero, 10 power four, zero. Okay, so what are the eigenvalues of this matrix? Yes, so basically P A of lambda equals lambda power four, all the diagonal entries are zeros and this is upper triangular matrix. So lambda equal to zero is the only distinct eigenvalue of A. Now consider a matrix which is a slightly perturbed version of this matrix. So I'll call it D. This is zero, zero, zero and say 10 to the minus four. And the other entries are the same. So this is 10 power four, zero, zero, zero, zero, 10 power four, zero, zero, zero, zero, 10 power four and zero. So now what are the eigenvalues of B? So this you can't say directly by looking at the matrix because it's not upper triangular but it's not difficult to work out P B of lambda. So if you do determinant of lambda I minus B and say expand along the first column, you will get minus lambda times minus lambda power three minus 10 to the minus four times 10 power 12. The determinant of this matrix which is equal to lambda power four minus 10 power eight. So that means lambda, if you solve this equal to zero, you will get lambda equal to plus or minus hundred and plus or minus hundred I, okay? So we see that a small perturbation by adding a 10 to the minus four term to the bottom left here has changed the eigenvalues from lambda equal to zero to lambda equal to plus or minus hundred plus or minus hundred I. So it caused a large deviation in the eigenvalues. And so it would be, what we would like to do is to understand when a matrix would exhibit such a property and when it's matrix, when a matrix is such that a small perturbation in the matrix will lead to a small perturbation in the eigenvalues. In fact, the latter point is more important because then we can say things about stability of systems, linear systems or otherwise, so that we can be assured that even if we have the matrix slightly wrong and the true matrix is a slightly perturbed version, the eigen space of that matrix is not severely perturbed by the perturbation, okay? And what are such matrices which would be less sensitive to perturbations? So for example, if you take diagonal matrices or upper triangular matrices like the one we've considered here, the eigenvalues have the diagonal entries, okay? And eigenvalues are in fact, continuous functions of these diagonal entries. And so small changes in the diagonal entries will result in small changes in the eigenvalues. So the natural question you can ask is, so yeah, so just to illustrate this from this matrix's point of view, if I had perturbed any of these entries slightly by 10 to the minus four, then the corresponding eigenvalue would have perturbed by 10 to the minus four, that's it. It wouldn't have led to such a large perturbation in the eigenvalues. So a natural question then is, what if a matrix is nearly diagonal or nearly upper triangular? Then that means that the off diagonal entries are small, which is not the case here, the off diagonal entries are quite big. But if the off diagonal entries were small compared to the diagonal entries, then can we say something about perturbations in the matrix and how that affects the perturbations in the eigenvalues? Such matrices arise, for example, in when you're computing the covariance of a random process that is nearly white. So the diagonal entries will be dominant and the other entries will be smaller. A related question to this is, can we approximately figure out where the eigenvalues are? So this is useful, for example, in linear system theory, where there is a notion of stability, that is a stability of a system of differential equations, for example. And you have a matrix that describes the state of the system and how this state evolves and you look at the eigenvalues of this matrix. And if the real part of those eigenvalues are all negative, then we say that the system is stable. So in these kind of scenarios where we are interested in understanding the stability of linear systems, we are interested in knowing whether the real part of the eigenvalues are negative. We don't really need to know exactly what the eigenvalues are. And similarly, if you want to show that a matrix is positive definite, we need to show that the, say it's a matrix which is Hermitian symmetric and you want to see whether it's positive definite or not, we just need to show that the eigenvalues are strictly positive. We don't necessarily need to show that the eigenvalues are, I mean, we need not, we may not want to know exactly what the eigenvalues are, but just show that the eigenvalues are positive. So how do we approximately locate eigenvalues? One way to do this is to find a bounded set which is guaranteed to contain these eigenvalues. And if this bounded set is in the strictly positive real line, then we know that the eigenvalues are all positive. So one trivial bounded set. So let me say this here, bounded sets that contain. So for example, one trivial bounded set is that we know that all the eigenvalues are less than or equal to the induced two norm. Or in fact, for this purpose, I can even just use the Frobenius norm, okay? So the way to look at this is that if I take the complex plane and if I draw a circle whose radius is equal to the L2 norm of A, then all the eigenvalues will be contained inside the circle. And so this is one way to bound eigenvalues, but this is not the kind of bounds that we are looking for. We are looking for something a little more precise than this. The key point is that eigenvalues are ultimately continuous functions of the entries of a matrix. And this is something that we'll take on faith and we won't prove that here. It requires a different kind of mathematics that we haven't really encountered or we haven't used very much in this course. But eigenvalues are actually continuous functions of the entries of a matrix. And so if we perturb a matrix by a small enough quantity, then the eigenvalues will not change too dramatically. That is the essence of what we are going to discuss. Okay, now in order to state or prove a core theorem on the location, approximate location of eigenvalues, we need one interesting theorem, which is called the diagonal dominant theorem. So what this theorem says is that, so let A in C to the n cross n be a matrix. So it's not, so we've moved away from this thing of Hermitian symmetric matrices, okay? This matrix need not be Hermitian symmetric. It's just square. If mod of aii, the magnitude of the ith diagonal entry of a is strictly bigger than sigma j equal to one to n, j not equal to i, mod of aij. So I'm adding up all the entries in the same row, but in all other columns of the matrix, except the diagonal entry. And the sum of all the entries in the same row is strictly smaller than the magnitude of the ith diagonal entry. And this is true for i equal to one to up to n. Then A is invertible, okay? So the condition is basically telling us that the diagonal entry of each row is dominant, over all the other entries, okay? So let's just quickly see how, why this is true. The proof is by contradiction. In other words, we'll show that not this implies not this. So if A is not invertible, then there must be some i for which this condition is getting violated. Okay, so suppose A is not invertible. That means A is a singular matrix. So in other words, its rank is less than n, so there exists an x not equal to zero, non-zero x, such that A x equals zero. So I'll just write that out in full form. So what that is saying is the first row here is saying A11 x1 plus A12 x2 plus et cetera, plus A1 n xn equals zero. The second row is saying A21 x1 plus A22 x2 plus A2 n xn equals zero. And we proceed like this. And the last equation will read A n1 x1 plus A n2 x2 plus A nn xn equals zero. Now, what we'll do is this x is some vector which is in the null space of A. One of its entries must be bigger than or equal to all the other entries in magnitude. So we'll just choose, suppose that is the ith entry. So choose i such that mod of xi is greater than or equal to mod of xj, j equal to one, two, three, up to n. So i is the largest magnitude entry in x. And note that this mod of xi must be strictly greater than zero because x is a non-zero vector. So some entry will have magnitude which is not equal to zero. So now we'll consider the ith equation. What is it saying? It's A i1 x1 plus et cetera plus A i n xn equals zero. A i n xn equals zero. Now, I'll just take the ith term to the other side. So minus A ii xi equals sigma over j equal to one to n, j not equal to i, A ij xj. Now we'll take the magnitude and, so if you take the magnitude on both sides, mod A ii times mod xi. And if I take the magnitude inside the summation, I'll get the less than or equal to inequality. Sigma over j not equal to i, mod A ij times mod xj. But mod xj is less than or equal to mod xi. So if I replace mod xj with mod xi for all, if I replace mod xj with mod xi for all j, then I'm only increasing the value. So this is in turn less than or equal to mod xi times sigma j not equal to i times mod A ij of mod A ij. And as I already remarked, mod xi is strictly positive. And so this means that mod A ii is less than or equal to sigma over j not equal to i mod A ij, which contradicts what we said in the beginning that mod A ii must be greater than the summation over j not equal to i of A ij for all i. But for this side, we see that the inequality goes the other way. And so this is a contradiction. Okay, so for example, if I take the matrix N 11, one N 11, one N 11, one N 11, one N 11, and this is an N cross N matrix, then the sum of all these guys is N minus one. And that is strictly less than N. The sum of all the other terms here is N minus one, strictly less than N and so on. So this matrix is invertible by the diagonal dominance theorem, okay. Now, obviously the converse of the diagonal dominant theorem is not true, meaning that a matrix may be invertible without being diagonally dominant. So a trivial example is 0, 1, 1, 0. This matrix is not diagonally dominant, but it is invertible. Okay, now let's proceed. So the thing is that any matrix A in C to the N cross N can be written as A equal to D plus B, where D is a matrix containing the diagonal entries A and this has all other entries. So pictorially I can write this as if A is a matrix which has some lower diagonal part, a diagonal part and an upper diagonal part. And I can write this as this diagonal part, which I'll call D plus everything else, which is this and I call this B. Now, let's consider this matrix A of epsilon, which I'll define to be D plus epsilon times B, okay. Now, A of 0 is equal to this diagonal matrix D and A of 1 is the matrix A, okay, it's just D plus B. And so as epsilon goes from 0 to 1, the matrix A of epsilon transitions from D to A. Now, if I take A of 0, it's a diagonal matrix and its eigenvalues are easy to find, they're just equal to the diagonal entries of A. So eigenvalues of A of 0 are lambda I equals AII, I equal to 1 to N, okay. And for small epsilon, the eigenvalues of A of epsilon, okay, they will be maybe, I'll say maybe, but in fact, it is true, close to AII, okay, so they're diagonal entries of A. And as epsilon increases, they're going further away. So the theorem that I'm going to state now is going to make this notion much more precise. This is one of the very central theorems of related to approximate location of eigenvalues.