 The last time we looked at spectral norm and some of its spectral radius, and some D I U S and some of their properties. And we started discussing about the error in inverses. So today we will continue the discussion about the error in computing matrix inversions. And we'll also talk about the errors in solving systems of linear equations. So just to recap, so we were looking at in a matrix A in C to the N cross N, which is non-singular, and we want to compute A inverse. So A inverse exists, but because of various reasons we don't end up computing A inverse. Instead of A inverse, we compute A plus E inverse. So yeah, I understand if my voice is breaking, but I think I'm connected to the IACW LAN. So this is the best connection I can get. No, sir, your voice is okay actually. Maybe you might be- Sir, voice is okay. Thanks. Yeah, this is the best network I can get. So I can't really do much more than this. So yeah, so we were looking at the error, which is equal to A inverse minus A plus E inverse. And we wanted to sort of get a feel for how this error, how big this error can be. And we did some algebraic manipulations, which I won't repeat again today. You can go back and look at the notes from the previous class, but what we showed is that if the norm of, so let me put it this way. If I look at the spectral radius, row A inverse E, this is going to be less than or equal to the norm of, since some norm, it doesn't matter which one. So some matrix norm A inverse E is less than one. Okay, then these infinite summations can be evaluated in closed form. And so we showed that the norm of A inverse minus A plus E inverse can be written as the norm of the summation K equal to one to infinity because the first term drops off because it cancels with this A inverse term, minus one power K plus one times A inverse E power K times A inverse. And this by using the submultiplicated property, we split it as the norm of this thing power K times the norm of A inverse. And so we can show that this is equal to, actually less than or equal to norm of A inverse E divided by one minus norm of A inverse E times the norm of A inverse. So we said that the relative error, which we defined to be norm of A inverse minus A plus E inverse divided by norm of A inverse is less than or equal to the norm of A inverse E divided by one minus norm of A inverse E if norm of A inverse E is less than one. Okay, so this gives us a way to bound the relative error in computing the inverse of a matrix in terms of the norm matrix A inverse times E. Sir? Yes. Sir, in order to calculate the norm of A inverse E, wouldn't we need to know A inverse? You would need to know A inverse and E. So as written, it's not very useful. So, but, so it will be useful if there is some way you can in some independent manner obtain a bound on the norm of A inverse E. So for example, we have a row of A inverse E is less than or equal to the norm of A inverse E which can be further bounded as the norm of A inverse times the norm of E. And suppose this was also norm of A inverse times norm of E was also less than one. Now, in general, you actually don't know A inverse and you don't know E. And so you wouldn't, so, but then if you have some other way of obtaining a bound on the norm of A inverse and the norm of E, then you can find that product and utilize something like this. Yeah, but this is the nature of the results in a lot of the error analysis where it gives you a way to bound the error at least when you know what is the error matrix. So for example, if you are doing quantization of the entries of A, then you do know what the quantization error matrix is. And so if you knew the correct A matrix and then you know the quantized A matrix, you know the quantized error matrix. The other way these things can be useful is if you know that the error matrix comes from a certain distribution, then you can look at what values this right hand side can take as you take different value, different E matrices from that distribution and use that to obtain some kind of bounds on this. So you'll have to bring in probability on top of this and say what is this bound? Can this bound bound be shown to be less than something with high probability over all possible E matrices? So you'll have to use those kind of techniques to further make these useful. But if this is also less than one, then we can simplify this further. See, this is the norm of A inverse times norm E is something that's bigger than this. So if I substitute norm of A inverse times norm E over here, it only makes the numerator, it only makes the denominator smaller. So it makes this overall ratio actually bigger. And so we have that norm of A inverse minus A plus E inverse divided by norm of A inverse is less than or equal to norm of A inverse times norm of E divided by one minus norm of A inverse times norm of E. And I can multiply and divide by norm of A to write this as norm of A inverse times norm of A times norm of E divided by norm of A divided by one minus norm of A inverse norm of A times the same thing, norm of E divided by norm of A. So we'll define this quantity here to be kappa of A norm of A inverse times norm of A if A is non singular and infinity if A is singular. So this thing is called the condition number of A. So if we define it like this, so how many of you have heard of this, the concept of conditional number? Have any of you heard of it before? So it was discussed in the last tutorial. Ah, very good. Okay, but other than that, in your undergraduate program, has there any of you heard of the condition number of a matrix? Something like the ratio of maximum minimum eigenvalues? Correct, correct. If you take the spectral norm of the matrix, it's the largest magnitude eigenvalue of the matrix. And therefore one other side result is that if you know the eigenvalues of a matrix, the eigenvalues of the inverse of the matrix are the inverses of the eigenvalues of the original matrix. And so this condition number reduces to the ratio of the maximum to the minimum magnitude eigenvalues of that matrix. So yes, so yeah, so this is called the condition number. And note that if we consider this condition number K of A, this is for an invertible matrix, it's the norm of A inverse. So but the more general definition is valid for any norm, it's specific to the norm. So depending on which norm you choose here, you will get different values for this condition number. And this norm that you're choosing here is the norm under which relative error over here. So this is norm of A times the norm of A inverse, which is greater than or equal to the norm of A inverse A. This is just sub multiplicativity, which is equal to the norm of the identity matrix. And we know that for any matrix norm, this is greater than or equal to one. So the condition number for any matrix A is going to be a number, which is greater than or equal to one for any matrix norm. And further because it's lower bounded by one, K of A, even though it's mapping a matrix to the real line, can never be a matrix norm because K of A equal to zero will never happen. And so it will never satisfy this positivity constraint. So what is the, so for, in particular, if I take the all zero matrix, what is its condition number? Infinite. Infinity, correct. So, so thus we have this norm of A inverse minus A plus E inverse over norm of A inverse is less than or equal to K of A times, I'll write it like this, one-mindness K of A times norm E over norm A times norm E over norm A. So from this, we can see a lot of interesting properties. So for example, if this number is small, then we can neglect this term here. It may not be a lower bound anymore because then you are making this thing smaller, but assuming that this is small enough that neglecting it does not break this upper bound, you have kappa of A times norm E over norm A, which means that this K of A or the condition number of A, it represents the, it allows you to bound the relative error in computing the inverse in terms of the relative error in the matrix A itself. And if K of A is close to one, that is, it is small, then we say that the matrix A is a well-conditioned matrix because it only amplifies the norm E over norm A by a small amount when you compute the inverse. Whereas if K of A is a large number, the matrix A is ill-conditioned or poorly-conditioned and it can lead to potentially a large increase in the relative error or a large value of the relative error in computing the inverse, even for small perturbation E. So I'll just mention these three words that I dropped. Small K of A is called a well-conditioned matrix. Large K of A, say that it's an ill-conditioned matrix. And for completeness, if K of A equals one, we say that it is a perfectly-conditioned matrix. So what is an example of a perfectly-conditioned matrix? I, I did it. Yes. How about unitary matrices? Are they perfectly-conditioned? Yes, sir. It depends on which one you're using. It depends on which norm you choose to use. For unitary matrices, U transpose or U Hermitian is the same as U inverse. And so it becomes, K of A becomes norm of U Hermitian times norm of U. And so depending on which, which norm you're using, they could be perfectly-conditioned. So in particular, if you're using the spectral norm, then they will be perfectly-conditioned. Okay. So the punchline is that for well-conditioned matrices, the relative error in the inverse is the same as the relative error in the data. Now, one other property of the condition number is that if you take the product of two matrices, this condition number is less than or equal to the product of the two condition numbers. That's simply because the left-hand side is norm of AB times the norm of AB inverse, which is less than or equal to, now I just used some multiplicativity, norm of A times norm of B times norm of A inverse times norm of B inverse. Okay. Yeah, so actually there's a lot more one can say about this condition number. I will leave some of them as homework exercises and I'll come back to this later in the course if there is enough time. But right now I don't want to distract ourselves from discussing about computational errors. And so let's now discuss about bounding the accuracy. Sir. Yeah. Sir, in the condition number, when will the equality exist in the above inequality? In this one, you mean? Sir, in the KAB is less than K into KB. Yeah, that's hard to say when it will hold with equality. Yeah. So obviously if A or B is the identity matrix, it will hold with equality, but for, I mean, it's not easy to come up with very general conditions in the which equality will hold. It depends on the norm also. So not easy. I mean, there's no straightforward answer to that question. Sir, even if A and B are I, the equality will hold only when the norm considered is the greater norm, right? Correct. Okay. Thanks.