 So, just to recap where we were the last time, we discussed, so just let me, maybe, you know, we discussed the spectral norm which we defined to be maximum square root of lambda, where lambda is an eigenvalue of A Hermitian A. And the spectral radius which we defined to be rho of A is the maximum magnitude eigenvalue where lambda is an eigenvalue of A. We also saw that rho of A is less than or equal to the norm of A for any matrix norm. And if there is a norm such that norm of A is less than 1, then limit k tending to infinity A power k is equal to 0 and such matrices were called convergent. And we saw that A is convergent if and only if rho of A is less than 1. And further this rho of A, it is a lower bound on any norm of A and it is also the greatest lower bound in the sense that given A in c to the n cross n and epsilon greater than 0, there exists a norm such that rho of A or the norm of A is between rho of A and rho of A plus epsilon. So, in other words, I can find a norm such that the norm of A will be as close to rho of A as I wish. And further the spectral radius can be used to bound the entries of A power k. So, if A is in c to the n cross n and epsilon is greater than 0, there exists a constant c such that A power k i j is less than or equal to c times rho of A plus epsilon power k. And this is true for every k i and j. We also saw that A is invertible if there is or there exists a matrix norm such that norm of i minus A is less than 1. And if so, A inverse is equal to sigma k equal to 0 to infinity i minus k power k. So, about this I made a note the last class that A may be invertible if say rho of i minus A is greater than 1 i.e. norm i minus A is greater than 1 for all matrix norms. So, I just want to maybe make this point a little more clear because I think the last time it was not entirely clear the way I said it. So, basic logic says that if you have a statement A and if we say A implies B, then it means, so if A implies B then not B which I will write as B complement implies A complement, but B does not imply A. This is the basic rules of logic. Now, in this case what is A? A is the statement that there is a norm such that norm of i minus A is less than 1. The statement B is that A is invertible. And so, now let us look at these statements and see what they mean. So, what the result says is that A is invertible if there is a matrix norm such that norm of i minus A is less than 1. So, that means if it is true that there is a matrix norm such that i minus A is less than 1, then A is invertible. That implies that A will be invertible. So, this A norm of i minus A less than 1, this A implies B. So, this is true for these two statements as I have written here. Now, what is B complement here? B complement is that A is singular. So, B complement, so if A is singular, it means the complement of A. What is A complement? A is the statement that there is a norm such that norm of i minus A is less than 1. The opposite of this statement is that norm of i minus A, there is no norm such that i minus A norm is less than 1, which means i minus A is greater than or equal to 1 for all matrix norms or for any, let us say. So, it means, so B complement implies A complement is the statement that if A is a singular matrix, then if you compute norm of i minus A, you will always get a number that is greater than 1, no matter which norm you pick. And what is B does not imply A here? B is the statement that A is invertible. It does not imply that there must be a norm such that norm of i minus A is less than 1. So, B does not imply A. In other words, it is possible that there is a norm such that i minus A is greater than 1, but A is invertible. So, the invertibility of A does not imply the existence of a norm such that i minus A norm is less than 1, but if this condition is satisfied, there is a norm such that i minus A is less than 1, then A will be invertible. I hope that is a little more clear than how we discussed it the last time. And at the end of the previous class, we saw the Banach Lemma, which said that if B is an n cross n matrix and this is an operator norm, then if norm of B is less than 1, then i plus B is invertible 1 plus norm of B inverse is less than or equal to norm of i plus B inverse is less than or equal to 1 minus norm of B inverse. So, one thing I will say is that a form like i plus B arises very frequently in many problems. And I will show you an example today of where you will end up with a form that looks like i plus B. And then what happens is you know something about this matrix B. In particular, if you know some norm of that matrix, this allows you to bound the norm of i plus B inverse in terms of how big the norm of B is. So, that is why these kind of results are useful. So, this is basically about the recap of what we did the last time. So, yeah. Sir, could you explain the difference between any and every? In this context? Yes, yes. There is no difference. Okay. So, the thing is that now, yeah. So, in this kind of context, there is no difference between any and every. Now, where it will make a difference is for example, if A were a random matrix, okay. Then you would be associating a probability with which this event will happen. We will say i minus A is greater than or equal to 1 with probability greater than 1 minus epsilon or something like that. In those kind of contexts, there is a difference between saying any and every. What you mean by any is that you will fix a norm first. And then you will take different instantiations of this matrix A. And you will look, you will try to see what is the probability that or what is the percentage number of cases where i minus A norm is greater than or equal to 1. And that is the probability that you are bounding when you say for any matrix norm. But when you say for every matrix norm, what you will do is you will pick an instantiation of A and you will ask, what is the probability that all matrix norms of i minus A are greater than 1, greater than or equal to 1. And a matrix fails this if there is a norm for which norm of i minus A is less than 1. And then you ask, if I take different different instantiations of this matrix A, what is the probability of success of this event? This is well beyond what we want to cover in this course. But I mentioned this only to tell you that in this context, any and every are actually this one and the same because we have fixed the matrix A. We are only looking at whether this norm is greater than or equal to 1 for this given single matrix A. But when we start talking about random matrices, there is a difference between any and every. Okay, so there's just one or two more results I want to mention. So one use of this result which says that if there is a norm such that the norm of i minus A is less than 1, then the matrix A is invertible, is in showing the following corollary, which is called the Levy-Deplank theorem. Okay, so which says A n cross n and suppose mod A i i is greater than sigma j equal to 1 to n, j not equal to i mod A ij for i equal to 1 to up to n. What does this mean? It means that if I take the diagonal entry and I compare that with the sum of the magnitudes of all other entries in the same row but all across all other columns, then the diagonal entries magnitude is strictly bigger than the sum of all other entries in magnitude in the same row and such matrices are called diagonally dominant matrices and they are very important from a matrix theory point of view. We are going to see a lot lot more about such matrices because in many signal processing manipulations, you end up with matrices which are diagonally dominant. So for example, if you think about computing the covariance matrix of a random vector, the diagonal elements of the covariance matrix represent the individual variances of these random variables while the off diagonal entries represent the cross covariances between these random variables and typically the cross covariances are small compared to the variances of the random variable. And so under some assumptions, of course, about the underlying set of random variables that you are looking at, these cross covariances could be small enough that the matrix satisfies a property like this. And so then everything we say about diagonally dominant matrices apply to covariance matrices so kind of a form of a dominant matrix then A is invertible. So this is interesting because Because what it says is that, so if I take a 2 cross 2 example, let's say, and let's say just for fun I write 2 and then 7 here. Now I'm allowed to write, for variety I'll say minus 7. I'm allowed to write any number here and any number here with the only requirement that the magnitude of this number, so I call it X, the magnitude of X must be smaller than 2 and I can, I'll call this Y, then magnitude of Y must be smaller than 7. You can, you can try your luck with putting down any number here, even be a complex number. Yeah, this will always be invertible. Okay, that's just to give you an idea of what this theorem is saying. So let's show this. So AII is strictly bigger than the sum of all the off diagonal terms in the same row and these are all non-negative because it's the magnitude of some number. So what this means is that this, this itself is greater than or equal to 0, which means AII is strictly bigger than 0 by hypothesis. That means that all the diagonal entries of this matrix are strictly positive. So if I take the matrix D defined to be diagonal of A11 through Ann, this will be invertible. It's a diagonal matrix with strictly positive entries, strictly positive magnitude entries. In fact, the inverse of this is just A11 inverse, A22 inverse up to Ann inverse. So then, if I consider the matrix B equal to I minus D inverse A, what this will do is this is a diagonal matrix which is operating on A. So what happens to the diagonal entries? The diagonal entries get scaled, the first diagonal entry will get scaled by A11 inverse. The second diagonal entry of this matrix will get scaled by A22 inverse and so on. And so when I consider D inverse A, all its diagonal entries will be equal to 1. And so this matrix B will have, this identity matrix of course has 1's on the diagonal. So when I subtract these out, the 1 and 1 will cancel and this matrix will have, has 0's on the diagonal. For the off diagonal entries, it will have minus aij over aii as ij the entry, i not equal to j. Okay, this is just the fact that when I am pre-multiply by a diagonal matrix, every row of A gets scaled by the corresponding entry of D inverted. Now, consider the, of course, the off diagonal terms in the identity is 0. So it becomes 0 minus aij divided by aii which is minus aij over aii. Okay, now let's consider specifically the max row sum norm. So you can already probably start seeing how this proof works out. I am going to add up the entries across each row in magnitude and then I will take the maximum value. Now, since sigma j equal to 1 to n, j not equal to i, mod aij over mod aii is less than 1 for every i. Okay, now I am doing j not equal to i but the corresponding i comma i the entry of B is just 0. So when I compute the row sum norm, each of them will be some number like this and it's less than 1. And why is this less than 1? It's because of this condition here. I am just taking mod aii to the other side. So this summation, the summation of j equal to 1 to n aij over aii for is going to be less than 1. And this is true for every i which implies that the norm of B infinity which is just the max of all these guys is also going to be less than 1. So B infinity is less than 1 that is this L infinity norm of this is less than 1. So there is a norm such that the norm of i minus D inverse a is less than 1 which then means that i minus B which is equal to D inverse a is invertible. But D is a non-singular matrix so which implies that a is invertible. So basically all diagonal dominant matrices are invertible. Then you put the point y a is invertible. See this is a product of two matrices. A product of two matrices will be invertible only if both the matrices are invertible. You've seen that already in previous classes also you've done homework problems on it that if you write a matrix as a product of two matrices and if that matrix is invertible then each of the constituent matrices must be invertible. Okay. So now there are just a couple of more points before I close out this particular line of discussion which has to do with remember we talked about equivalence of vector norms. There is a similar thing we can say about equivalence of matrix norms. Sir, so the converse is valid or not in Bob theorem? Which theorem? Do all invertible matrices need to be diagonally dominant? What do you think? Is this matrix invertible? Of course. It is determinant is minus 1. Yes sir. In fact, it is inverse is itself. You multiply this matrix by itself you will get the identity matrix. Yes sir, I got it. It's not diagonally dominant. So again, it's a good point because all these theorems are usually very carefully stated. If there's any error in the if there's any anything weird or inconsistent in the statement that's purely my mistake in writing the theorem down. The textbook actually is very, very good. The theorems are all extremely carefully stated. And so when it says if so what it's saying is if a matrix is diagonally dominant, it will be invertible. It does not mean that every invertible matrix will be diagonally dominant. The converse is not true. Yes, please. That matrix which I represent above that is 0 1 1 0. So is there any name for it? Anti diagonal? No, sorry. It's a permutation matrix. Okay, okay. See if I multiply x y. Okay. What I'll get is y x. Okay. It permeates the entries of the vector. It's a permutation matrix and permutation matrix have lots of very nice properties. Okay. Okay. And this is an example of a two cross two permutation matrix. Of course, the only other permutation matrix in two cross two is the identity matrix, which doesn't do any permutation actually. Okay. Is the identity permutation because you have only two entries. You can either keep them where they are or you can exchange them. Nothing else you can do.