 Now, I want to discuss a new object, which is called, yeah, go ahead please. Sir, the triangle inequality for matrix norm, how does it hold? It was not mentioned in the properties of matrix norm satisfy triangle inequality, that is the definition of a matrix norm. A matrix norm must satisfy non negativity, positivity, homogeneity, triangle inequality and submultiplicativity. So, basically matrix norm satisfies all the properties of the vector norm. In addition, it needs to satisfy one extra property, which is that of submultiplicativity. So, the spectral radius rho of A of a matrix is rho of A is the maximum mod lambda such that lambda is an eigenvalue of A. So, this is the definition. So, given a matrix A, you find all the eigenvalues of the matrix and then you look, you ask which eigenvalue has the largest magnitude and that value is called the spectral radius of a matrix A. But you know, this spectral radius is a fascinating object. It has lots of very, very interesting properties. I will tell you one to begin with. And so, it has the following universality property. So, suppose lambda is an eigenvalue, the matrix A. So, what that means is that mod of lambda is going to be less than or equal to rho of A because rho of A is the largest magnitude eigenvalue and there is at least one eigenvector corresponding to lambda. So, suppose that, so there is and there exists at least one eigenvector corresponding to an eigenvalue with magnitude rho of A. So, now of course, there is also an eigenvector corresponding to lambda. But I am interested in the eigenvector corresponding to the eigenvalue whose magnitude is rho of A. Yeah, is there a question? Yes, sir. Sir, what if lambda is imaginary? Lambda can be complex value, no problem. Still there will be eigenvector. Yes, of course. Okay. Okay. So, the point is that given any n cross n matrix, it will always have n eigenvalues. Some of these may be real valued, some of them may be complex valued. So, since complex numbers are a generalization of real numbers, we can say that any n cross n matrix will have n complex valued eigenvalues. Some of these eigenvalues may be repeated. Now corresponding to, we will see this later, but corresponding to every distinct eigenvalue, there will always be at least one eigenvector. When eigenvalues are repeated, it is possible that you cannot find. But when I say distinct, I mean a linearly independent eigenvector. So, in other words, an n cross n matrix will always have n complex eigenvalues, but it may not have n linearly independent eigenvectors. But corresponding to every eigenvalue, every distinct eigenvalue, there is always at least one eigenvector. Because by definition, an eigenvalue eigenvector pair satisfies the equation Ax equals lambda x. And so, if you cannot find an x such that Ax equals lambda x, then that cannot be an eigenvalue of the matrix. So, basically, since, since rho of A is the maximum magnitude eigenvalue, if I consider that particular eigenvalue that gave me rho of A, there is a vector x such that. So, if, so if I, if I consider, so let us say consider. Hello, sir. Yeah. Sir, based upon the definition you gave, will it be the interpretation like this that the spectral radius defines the capability of matrix A to the maximum amplification it can give to a vector x? That is correct. Okay, sir. But it is something that you have to prove. So, and when you say amplification, you have to also keep in mind, when you say amplification in what sense? Okay, what, what property of the vector x are you thinking of getting amplified? Okay, so, for example, direction of x will not be changed in the same direction amplification should be considered. That is true. That is true for any eigenvector, any eigenvector of A. Okay, that is true for any eigenvector of A, not necessarily, not only for the eigenvector corresponding to the eigenvalue whose magnitude is rho of A. Yeah. Okay, so when you are thinking about, so you made a statement, you said that the rho of A is the maximum amplification that a vector can experience when it is multiplied by A. So, in other words, in your mind, what you are thinking of is that the vector x has a certain size or length. Okay, you measure it using some, typically you want to measure it with the notion of a norm. Okay, so for some particular norm, the vector x has some value. And then you look at the norm of A x and you see how much bigger the norm of A x is compared to the norm of x. In other words, you are looking at the ratio, norm of A x divided by norm of x. And you are saying that the maximum value this can take, norm of A x divided by norm of x, the maximum value it can take is rho of A. So, that is correct. But it's important that to keep in mind that this is measured in terms of the Euclidean norm. It's only when you measure it in the Euclidean norm that the largest possible value of norm of A x divided by norm of x is going to be rho of A. Okay, so now let me just continue with this argument here. So, suppose I consider this, the particular eigenvector corresponding to rho of A. Okay, so there is some lambda corresponding to rho of A and corresponding to that eigenvalue, there is an eigenvector x and that satisfies A x equals lambda x. Now, I will consider a matrix x, which is the size n cross n and whose columns are all equal to x. So, I just repeat this. Then what I will do is and note that A x is equal to the same scalar lambda times the matrix x. That's because each of these columns are multiplying A and then when you multiply this column with A, it will become equal to lambda times that column and then every column gets multiplied by lambda. You can pull that right out of the matrix and you will have lambda times x. Now, so if you are given any matrix norm, is a matrix norm, then if I consider mod lambda times matrix norm of this x, this is equal to the matrix norm of lambda x. This is just by homogeneity, but lambda x is equal to A x. And then by submultiplicativity, this is less than or equal to the norm of A times the norm of x. Now, x is nonzero and therefore, the matrix norm of this matrix capital X is always going to be nonzero. So, if I compare this last step with this first step, I can simply cancel out the matrix norm of x and I have the conclusion that norm of A is greater than or equal to mod of lambda which is rho of A. So, this is a fascinating property. So, we defined rho of A to be the largest magnitude eigenvector of eigenvalue of the matrix A. And it turns out that rho of A is a lower bound on any matrix norm you can define on A. So, however you define the matrix norm, it can never give you a value which is less than this spectral radius of A. So, I will write that out because it is a very interesting and fascinating result. If A is any matrix norm, if A is in c to the n cross n, then rho of A is less than or equal to the norm of A. So, a natural question you can ask is, okay, rho of A is the maximum magnitude of all the eigenvalues. So, basically it maps an n cross n matrix to a nonnegative number. And so, is rho of A a matrix norm? Excuse me, sir. I have doubts. Yes. Sir, in the above expression you have written mod of lambda is less than or equal to mod of A into like matrix norm of A into matrix norm of x, right? Then how did you conclude from here to next? Is it for all lambda above the expression? No, no. This is for, so I wrote that here. Mod lambda is rho of A. Okay. Only for that particular lambda. Okay, thanks. Yeah. So, I am considering the eigenvector corresponding to the eigenvalue whose magnitude equals rho of A. Okay. Thank you. Of course, the other eigenvalues in the matrix A have a magnitude less than or equal to rho of A. So, the norm of A is actually going to be greater than or equal to the magnitude of any eigenvalue of the matrix A. And in particular, it will be greater than or equal, I mean, norm of A is greater than or equal to the maximum magnitude eigenvalue of A, which is rho of A. It is like tighter bound, right? This is tighter bound than tighter lower bound. Yeah, it is a lower bound on, so rho of A is basically a lower bound on any matrix norm. Yes, yes. Okay, so rho of A is not a matrix norm. Okay. In fact, it is not even a vector norm on C to the n square. Okay, you can show this. It is not difficult. But it is a lower bound on any, for any norm of A. Okay. Of course, you know, I can always say, you know, if I want a lower bound on norm, I can always say a lower bound on the norm of A is norm of A greater than or equal to 0. It is a non-negativity property of the matrix norm. But that is a trivial lower bound. It is not very interesting. But norm of A greater than or equal to rho of A is a very non-trivial lower bound. All right. And so that is the property of this rho of A. Now, it turns out that although rho of A cannot be a matrix norm, it is actually the, it is the biggest lower bound you can find. But rho of A is the greatest lower bound, the values of all matrix norms. Okay. What do I mean by this? So, we just discussed that norm of A is greater than or equal to rho of A. So, pick any norm, the value of that norm, that norm of A is going to be at least equal to rho of A. I found one lower bound on norm of A. You can ask, can I come up with a different, a bigger lower bound on norm of A? That is, instead of rho of A, think of maybe you can find some other function and write an inequality that this is greater than or equal to some other function zeta of A. Okay. Then what is this zeta? And this zeta of A is greater than or equal to rho of A. Can you find such a function as something that is an even better lower bound on any matrix norm you can define on A? The key thing is it has to hold for every possible matrix norm you may choose to define on the matrix A. Okay. Even once that we have not yet discussed and we haven't know that maybe people haven't yet discovered, can you find a bigger lower bound? The answer turns out to be no. So, rho of A is in fact the greatest lower bound you can find that will hold for all matrix norms of A. Okay. That is what we will show next. Sir, sir, I didn't get the concept of how it's amplification that a matrix can give. So, that's a side point. We will come back to that later. Essentially, we were discussing about if you look at norm of A x and then you divide that by norm of x. Okay. This is in one way you can define this to be what is the amplification that x is experiencing when it is multiplied by A. Okay. And the spectral radius is the largest such amplification you can get. Sir, is this same spectral norm only? Spectral radius and spectral norm are two completely different things. So, if I go up here. Sir, actually my apprehension was that it requires it to be in same direction. What if there is another vector that is rotated but has still greater amplification by A? So, I think you need to spend a little time thinking about it, whether this is possible or not. So, first of all, let me maybe make one small point that the, I mean, there is many more things we have to discuss before I can properly answer this question. And if I start discussing those, we will just completely leave the point here. So, but the only point I want to make right now in terms of clarifying this is that the spectral norm is defined like this. It's the maximum root lambda where lambda is an eigenvalue of A Hermitian A. Whereas the spectral radius is the maximum mod lambda where lambda is an eigenvalue of A itself. Okay. It turns out, I mean, it's very, it's a good point you make because it's, what is interesting here is that these two are very different objects. The spectral norm is a matrix norm. Okay. Whereas the spectral radius is not a matrix norm. Okay. So, now just let's just compare these two. So, keep this in mind. Norm A2 is the max root lambda where lambda is an eigenvalue of A Hermitian A. And in fact, we can show something like this. The square of this is equal to the, I have to look at the largest value of Ax square subject to x2 equal to 1. Whereas the spectral radius is rho of A is the maximum mod lambda where lambda is an eigenvalue of A. Okay. So, these are two different objects. Sir, isn't that definition the induced induced by L2 norm that we have for spectral norm? Isn't that saying that this is the maximum amplification that we can get in terms of Euclidean norm? Yes. So, they're both related. And if you're looking at a quantity like, right, what you can show is that this quantity is actually equal to x transpose A transpose Ax. And this you can show is less than or equal to this norm of not four bars. Okay. So, yeah, I think your point is basically that when you're talking about amplification to this norm of x, you should really be comparing this. You should be looking at when you're looking at Ax L2, this is actually equal to square root of Ax L2 squared, which is equal to square root of x transpose A transpose Ax. And so, it is actually more related to the spectral norm than to the spectral radius. However, we will see the connection between these two more closely in a few classes. Okay. Let me get rid of this. I don't know if that answers your question. Yes, sir. I think it's more related to spectral norm, but in general, that is true for spectral radius with respect to any norm. Correct. So, later we'll actually explicitly discuss these kind of amplification concepts when we discuss this thing known as the Rayleigh quotient. And in fact, that can be used to derive an algorithm to find the eigenvalues of a matrix, in which is a different algorithm compared to finding the characteristic polynomial and trying to solve for the zeros of the characteristic polynomial. So, we'll discuss that more later. But right now, I want to say that the row of A is a lower bound on the, on any norm on the matrix, you can, you may choose to define on the matrix A. And in fact, you cannot find a better lower bound. This is the greatest lower bound. And that is because of the following lemma. Let A be a matrix of size N cross N and epsilon be some number, small number, which is strictly greater than zero. Then there is a matrix norm such that row of A is less than or equal to this matrix norm of A. So, this matrix norm I'm going to denote by these three bars is less than or equal to row of A plus epsilon. Okay, so what is this thing? It's saying that no matter how small and epsilon you choose, I will be able to find a matrix norm such that the matrix norm of this particular matrix A is between row of A and row of A plus epsilon. In other words, you cannot find a different lower bound which will work for all A's and all norms. Okay, so maybe I'll write that here. So, it works for all A and for all norms. Okay, so you won't be able to find another norm, another lower bound which is actually bigger than this row of A. I can get, I can define a norm such that the norm of A is as close to this row of A as I wish. That is what this lemma is saying. So, how do we show this? The proof actually uses one result that we're going to again show later, which I'll state here, but you have to take it on faith for now, but we'll prove it later on. It uses a result called the Schur Triangularization Theorem. What this says is that given A of size n cross n with eigenvalues lambda 1 through lambda n, then there exists a unitary matrix U which is of size n cross n such that A is equal to U Hermitian delta U where delta is an upper triangular matrix, diagonal entries delta ii equal to lambda i. Okay, so this is the result that we will use. Okay, any matrix, any n cross n matrix can be decomposed as U Hermitian delta U where U is a unitary matrix and delta is upper triangular with the eigenvalues along its diagonal. Okay, so now let's set dt to be the matrix, a diagonal matrix with t, t squared up to t to the n along the diagonal and zeros everywhere else. Okay, and then if you compute dt delta dt inverse, you can show, you can try a simple 2 cross 2 example to convince yourself that this is true, but you can show that this is equal to lambda 1, then t inverse delta 1 2, t to the minus 2 delta 1 3 up to t to the minus of n minus 1 delta 1 n, 0 lambda 2, t inverse delta 2 3, etc. And so all the diagonal entries will remain the same, I will get lambda n down here and I will have t inverse delta n minus 1 comma n down here. Okay, and all these are zeros and sub-diagonal, you have zeros everywhere. Okay, this is what happens if you take dt delta dt inverse, where dt is this diagonal matrix and delta is an upper triangular matrix. So basically if I choose t to be a very large number, I can make all the off diagonal entries as small as I wish. So for large enough t, we can ensure that the sum of the absolute values of all off diagonal terms is less than or equal to epsilon. So that means that this, if I look at dt delta dt inverse, this the one norm is going to be less than or equal to rho of A plus epsilon. I think I am out of time, there is actually a couple of more steps and I don't want to rush this last part of the proof. And so it will take me maybe three or four minutes and I don't want you guys to feel like I rushed through this proof. So I just say to be continued. So a fun exercise for you to, for you could be to see if you can actually show that norm of A is less than or rather. So ultimately what we want to show is that rho of A is less than or equal to, for this particular, so okay, let me do this. I'll just say one thing here. We'll define the norm. So we want to define a norm such that the norm of A is between rho of A and rho of A plus epsilon. And we're going to define it to be the L1 norm of dt u, b, u-humation dt inverse. So this is using that S inverse AS form. So if I define it like this, it turns out that for this particular norm, rho of A is less than or equal to A is less than or equal to rho of A plus epsilon. So this is what we want to show, but we will continue this and show it a little more slowly in the next class.