 So, if we are saying that A is diagonalizable, it means that it admits the basis right, so the first thing. The way we will go about this proof is not by showing this, this, then this, this and then this, this. We are going to go about it in a cyclic fashion. We will show 1 implies 2, 2 implies 3 and finally 3 implies 1 and we will be done with it. That is our line of reasoning. That is actually the minimal effort that you can put in right, because you have 3 conditions okay. So what is this? A is diagonalizable, A is diagonalizable. This implies there exists B, a basis for V such that, such that what? Okay, now I am going to use the hybrid notation okay. You might think A was a matrix, what is this nonsense about putting in a box bracket, but let us just say A is the operator. As I said I will not distinguish between it. So then A represented in terms of the basis, which then becomes a matrix. We do not confuse this okay. I know it is a bit confusing, but I just want to avoid writing phi every time. Ideally we have used phi for the operator and A for the matrix, but let us just say for now, I mean I am tired of those Greek letters, so let us get back to English okay. So this is what it is. And what is this? This is basically A V1's representation in terms of the basis, likewise till A Vn's representation in terms of the basis where the B is V1, V2 until Vn, is it not? Yeah. This looks like the following matrix, some lambda 1, lambda 1, I do not know some size of lambda 1's. Let us say this is d1 cross d1. From here on I will just write lambda 2 I d2 until lambda k I dk, where I is the identity of size dk cross dk here. So this representation turns out to be something like this. So this is a given. If A is diagonalizable, then of course there exists some such basis subject to which if I represent it, it is going to look something like this. I can order the basis suitably so that all the lambda 1 related terms suffers, then the lambda 2 related terms and so on. It is just an ordering. Because I have an existence of the basis, I can figure out which are the ones that are coming from lambda 1, which are the ones from lambda 2 and so on. If this is given, I have to show that this is true. Very straightforward verification. What is the characteristic polynomial of this now? It is nothing but determinant what? I am going to just ask for a very preliminary knowledge of determinant here. What is this? X i minus this matrix, is it not? So this is X minus lambda 1 to the power or rather X minus lambda 1 I d1, X minus lambda 2 I d2 until X minus lambda k I dk, agreed? Now what is the determinant of this? This first block, X minus lambda 1 to the power dk. This one is X minus lambda 2 to the power, sorry d1, this one is X minus lambda 2 to the power d2, so on. This is just product X minus lambda I to the power di i going from 1 through k, which means that the algebraic multiplicity of lambda i is di. What I have to essentially now show is that the geometric multiplicity must also be di and will be done until this point there is nothing non-trivial going on, I mean as in non-trivial calculation, right? This is straightforward. I am just asking you to get the determinant of a diagonal matrix, okay? So maybe I will just keep, I want to keep that statement here. So I will probably erase this first part here. Now think about wi. What is this wi? It is the kernel of lambda ii minus a. Again of course you might just say this is, this object here, okay, subject to this basis. I am not making any distinction between the operator and the matrix. I hope that is well understood by now, all right? Sometimes I might omit that box, you might understand from context it is a matrix. What is this going to be looking like? This object inside, this is going to be the kernel of, look at the matrix. This is just lambda 1 minus, sorry, lambda i minus lambda 1 id1 until the ith block turns out to be 0 times idi, right? And every other block is lambda i minus lambda k. Of course, needless to say that these lambda k's are distinct. Why would I otherwise write them as up to k? I would have written them as n, right? So these are distinct, all right? So let's just put it there, lambda i's are distinct. It's only there because they are distinct that I am able to call this as the algebraic multiplicity. What do you think is the dimension of the kernel of this? What is the rank of this matrix? It is n cross n. How many linearly independent rows or columns are there? It's a diagonal matrix after all. Apart from these di rows and columns, everything else is full rank. It has a non-zero at a very unique position. Every one of those rows and columns have non-zeros at unique positions. So definitely then the dimension of wi is equal to di is equal to geometric multiplicity of lambda i. Earlier we saw that this was the algebraic multiplicity. Now we see that this is the geometric multiplicity. So now we have to show that 2 implies 3, right? So we are starting with the condition that the algebraic multiplicity and the geometric multiplicities are going to be equal. So now suppose algebraic multiplicity of lambda i is equal to geometric multiplicity of lambda i for all i belonging to the set 1, 2 until k where there are k distinct eigenvalues, right? What does this immediately bring to our mind? From this summation algebraic multiplicity of lambda i is equal to summation di. What is this going to be equal to? N of course because it is an nth degree monic polynomial, right? It is an nth degree monic polynomial. That is what the characteristic polynomial is because this is nothing but the degree of the characteristic polynomial, right? It must be the degree of the chi. This must be equal to n. So summation di is equal to n which means what? This also means from this equality that the summation gm of lambda i is also equal to n which means that summation dimension wi is also equal to n. Now the claim is that these wi is all finite dimensional vector spaces of dimension di. So they will have a basis each of size di of cardinality di. So now we are going to use that earlier result that we have proved in the previous module. Consider bi as a basis for wi then by our previous result which I am not going to write again. Union bi is a basis for summation in fact a direct sum of wi that is what we have just proved the precise the result just before this that is exactly what we have painstakingly proved somewhat, right? So that is true. What is this after all? This is a subspace residing inside v of dimension n, v has a dimension n. So this must be equal to v, yeah? Does anybody have any doubts about this? Again please feel free to ask. What we are saying is this is given to us by our definition the algebraic multiplicities must sum up to n the degree of the characteristic polynomial and since the geometric and the algebraic multiplicity of each eigenvalue is the same therefore the geometric multiplicities of all those eigenvalues i is equal to 1 through k must also sum to n but that just means what that the dimensions of the individual w i's must also sum through to n. So therefore I am saying cook up basis for w i's individual w i's and you can always do that because these are all finite dimensional with dimension d i and take the union and just a while back we have shown that if you cook up individual basis for w i's and take their union you get a basis for the sum of those subspaces. In this case the sum of those subspaces by the dimension matching here turns out to be nothing but v because v is n dimensional any n dimensional subspace sitting inside another n dimensional subspace must be equal to the originals vector space itself. So what are each element what can you say about each element inside this set it is an eigenvector is it not I do not care for which eigenvalue but each object inside. So since each element in union b i is an eigenvector yeah and also we know that this is a linearly independent set by our previous proof so I am not going to use it I am not going to prove it again we already taken the trouble of proving this and union b i is linearly independent what do we have 2 implies right summation d i oh sorry yeah that should be k thank you yeah please ask if there is any doubt about this proof the fact that so we have shown that 1 implies 2 we have now shown that 2 implies 3 what is left to be shown is in fact 2 implies 3 is the trickiest part I would say not tricky but because we have done the groundwork it seems very brief if I had not done the groundwork that entire proof of that previous module would have come in as part of this proof you understand right so that is why I built it up I just sort of split it up into small little proofs so that in this case the proof does not grow very big alright now we have to show that 3 results in 1 but I would say that is very obvious is it not if you indeed have n linearly independent eigenvectors I do not care for which eigenvalue and whatever ordering and whatever you choose right so maybe I should erase this part yeah that is the oldest it seems alright so what we have is that a has linearly independent eigenvectors a has n linearly independent eigenvectors say v 1 v 2 through till v n since these are all eigenvectors therefore there exist lambda I not all distinct necessarily okay such that a vi is equal to lambda I vi now I am going to just stack it up together as we have done so often v 1 v 2 until v n so I am not now treating this like a matrix if you want to treat this like an operator again you have to take the individual representations and see the action of a on v 1 so if you call it and so maybe I will do that right after I am done writing in terms of matrices that is nothing but v 1 until v n times lambda 1 lambda 2 lambda n which essentially means that if I call this matrix v then it is v inverse a v is equal to lambda 1 lambda 2 lambda n okay if I want to be a little more sophisticated with the operator theoretic language I basically say that a v 1 it is operate its representation in terms of the basis then a v 2 its representation in terms of the basis until a v n its representation in terms of this basis what does it look like well this is now just giving it a coordinate representation so it is an ordered basis now right so a v 1 is what lambda times v 1 but lambda times v 1 represented in terms of an ordered basis where the first basis in that ordered basis is v 1 so it is just 1 0 0 0 0 so lambda 1 times 1 0 0 0 0 until n is it not in other words this is equal to lambda times just v 1 b then lambda 2 times v 2 b until lambda n times v n b what is v 1 in terms of the ordered basis where v 1 is the first element the matrix representation it is the it is just the principle no principle like you know basis vector 1 0 0 0 0 so on so this is nothing but okay let me draw a line here this is nothing but lambda 1 times 1 0 0 0 0 then this is lambda 2 times 0 1 0 like this until you have lambda n times 0 0 times 1 which is nothing but just this so whether you want to look at it as an operator or a matrix it matters not this clear right so we have shown now that 1 is equivalent to 2 2 is equivalent to 3 and 3 is equivalent to 1 and at this point we might have kind of closed the story on eigenvalues and eigenvectors and say that yeah I mean most cases you will encounter operators or matrices where even if the eigenvalues are not equal I mean even if the eigenvalues turn out to be equal maybe due to some perturbation or some numerical error here and there it is not a big deal in most cases you are likely to encounter matrices or operators where eigenvalues are only distinct and you will be right in inferring such I mean if you think of it like a probability problem where there is n squared numbers you have to fill out in an array you have to actually be very careful in choosing your numbers so that the eigenvalues actually repeat what are the odds that if you pick out a polynomial at random with some random coefficients that you will actually encounter a polynomial with repeated roots it is very unlikely right so you might say in most cases in my life I will probably never encounter this well that is not quite true firstly as I said in some physical systems modeling yeah it is not some numerical calculation that we have in mind always some physical system which has repeated modes whether you like it or not by the very design see for example that bridge which collapsed at its resonant frequency right it is a physical phenomena after all that number is just incidental but it has a resonant frequency it was badly damped so when soldiers were marching on it the bridge just collapsed because it maxed with the natural frequency so there are things there are physical phenomena which we have no control over so by the very nature of some circuit or some mechanical system it may so happen that the system is bound to have repeated eigenvalues and in such cases you cannot just say oh I will just tweak this number a little by some epsilon and make the eigenvalues distinct and then diagonalize it and say all that is the no as it turns out that such small changes in that matrix yeah and that x dot is equal to a is the dynamical system we had that small change might actually change the nature of the solution quite drastically right for those of you who have actually solved differential equations and have encountered this you would appreciate this that if you have repeated eigenvalues right and quite often you will see that the solution is not necessarily of the form e to the lambda 2 lambda t alone but rather some t times e to the lambda t right so just that eigenvalues being coincident do change the nature of the solution right so that is the reason why we will invest ourselves even though you will see that for much of the next few lectures we will just be trying to get our way around this Jordan canonical form just to motivate why we are putting in that effort yeah that is the motivation so what if these conditions are not met and you do end up with a system which cannot be diagonalized what is the best possible thing we can do we cannot just throw up our hands and say no no deal with it it is a 10 cross 10 system or a million cross million system so what we will then try to do is get it down to its bare minimum essentials maybe I cannot diagonalize what is the best thing about diagonalizations completely decoupled again in terms of differential equations you had nth order differential equation you reduced it to n first order differential equations maybe not first order but at least we know our way around second order differential equations so suppose we cannot diagonalize something but let's say there is some sort of a special basis which allows us nonetheless to block diagonalize a matrix so that the smallest block is still one but the largest block is maybe no greater than two that's also a deal right that's also winner that the block diagonals so let's say it's a 10 cross 10 matrix and maybe some of those blocks are not diagonals I mean some of those entries are not just diagonals but at least the biggest block that you have the block diagonal that you have is of size 2 cross 2 it means that the highest order of the differential equation you need to solve is just second order you can still decouple the system in some sense right if you are not getting the picture let me just draw it here so suppose you have a matrix that looks something like this all right everything else is 0 so normally this is what one 6 cross 6 so it's a sixth order differential equation probably being represented by this a matrix right if you have x dot is equal to this times x but on the other hand if you have brought it down to this form it turns out you can treat this part and you can partition this x into just 2 cross 1 here 1 1 1 and another 2 cross 1 here so you can treat this like 3 first order differential equations and 2 second order differential equations yeah okay yeah 2 sorry so here it is yeah okay getting my counting wrong now right so but that's still a winner right because instead of having to solve a sixth order differential equation you're going to solve at most the largest differential equation is the most complicated one perpetually is a second order differential equation and then this is completely decoupled from the rest so is this so if you can get it to this form that's also a nice thing right so in our bid towards a form that resembles this we will now study something which initially will look like completely disconnected something completely disconnected from what we have been doing now all right but we will see the connection very shortly so I'll just name that object in the previous sorry yeah no no if the lambdas are distinct all if the lambdas are all distinct that is there are n distinct eigenvectors then of course it's diagonalizable but in the previous proof we have said it's k k is less than or equal to n so you don't need a distinct eigenvalues that's what we approved as long as those kernels have sufficient size so if your eigenvalues are repeated a certain number of times but the corresponding kernels also have that dimension to match the number of times it is repeated in the characteristic polynomial then we are safe even if the eigenvalues are repeated we don't care about it yeah so we are now going to focus on something called an a invariant subspace so simply stated it is this suppose you have v sitting inside which is a subspace w of course a is finite dimensional vector space in general we'll take it to be n the size of the dimension of v all right for a which is an operator from v to itself suppose a v or maybe we call it w a w belongs to the subspace w whenever w belongs to the subspace w all right then w is said to be an a invariant subspace of v all right what it means is the action of a on w does not take you anywhere outside the subspace w so the action of sometimes in for notational purposes people say that a big w is contained inside w which means that you take fellows inside the subspace w let a act on them now look a is an operator from v to itself so there was a chance that if you take an arbitrary random vector inside w and a random operator an arbitrary operator then there is no reason to expect that after the operator's action has been performed it will still live inside w it can be anywhere in v which is not necessarily part of w but this is a very special kind of an a invariant subspace subject to a given operator yeah if you can find a subspace such that the action of a on any vector inside the subspace still contain still lives inside this same subspace okay now it's all very well and good to say that oh this seems like very difficult object to wrap our heads around do these objects even exist yeah so I will give you the simplest possible example of such an subspace and don't laugh at this but okay so this is a invariant is it not so at least it exists always another well let me cheat in the sense be very lazy of course v is its own subspace by definition it is a invariant okay so that doesn't make you happy I am sure you're asking for more better examples I at least guaranteed to you that it's not a meaningless pursuit but you might still feel it's meaningless because these are of course I mean obviously they're going to be a invariant yeah exactly every sub every vector space is its own subspace so the whole vector space is of course a invariant because the operator is defined on the vector space itself that's kind of obvious so these are very lazy examples so when we come back in the next module I'll give you some more interesting examples which should hopefully get you more excited about the these kind of subspaces a invariant subspace sorry projection map yeah it's image it's invariant to its own image yeah it's I it's I'd important that's what yeah yeah that's a non-trivial that's one non-trivial example yeah but we will we are not particularly now focusing on the projection map that was kind of done in duster we'll see more important objects because remember our study of a invariant subspace is now with a particular goal in mind the goal is if you cannot diagonalize it get it down to its best possible block diagonalization in the smallest possible blocks we are not happy with just a big block because that still entails the solution of a higher order differential equation right so you want to get it down to as small as possible and these are the objects that we expect to help us right so when we come back in the next module I will talk about a few more examples of this and that should get us started on this