 So, the last time we looked at unitary equivalents, so A and B are unitary equivalent if B is equal to U Hermitian Au, where U is a unitary matrix. And we also looked at the notion of Euclidean isometry, we finally showed one small result which said that A and B, if A and B are unitary equivalent, then the Frobenius norm square of the two matrices which is the sum of the squares of all the elements of the matrix will be equal. Okay, so today we will discuss some concluding observations about unitary equivalents and then we will cover this Schur's unitary triangularization theorem. Okay, so couple of remarks, one is that unitary equivalents, okay, two matrices are unitary equivalent, it means that they are similar because for a unitary matrix, U Hermitian equals U inverse. So, they satisfy the definition of similarity, but the converse is not necessarily true. So, two similar matrices need not be unitary equivalent, okay, that also means that unitary equivalents partitions the complex n cross n matrix space into a finer equivalence class compared to similarity, similarity based equivalence. So, within each similarity based equivalence, there could be many matrices that are unitary equivalent to each other, but many subclasses which are not unitary equivalent to each other. But any pair of matrices that are unitary equivalent are also similar, so they belong to the same similarity class. Okay, and we observed that the similarity transform is essentially, it corresponds to a change of basis, that is if you have a linear transform and you change the basis, then you ask what is the linear transform according to the new basis, that is given by the similarity transform. However, the unitary equivalence is also a change of basis, but a special one. It's a change of basis from one orthonormal basis to another. Okay, so to continue, oftentimes we like unitary equivalents because they are simpler to compute. So, I mean simpler to compute than similarity. So, for example, there is no matrix inversion involved, it's U Hermitian Au. So, and it's also numerically more stable because these unitary matrices are well conditioned. And so numerically computing a similarity transform is more stable than computing a similarity transform. Okay, now I will give you two examples of important unitary matrices that show up in many applications. So, the first is called plane rotations. So, we write U of theta ij is this matrix which has ones on the diagonal and somewhere in between it has a cos theta and then some more ones and then another cos theta and then maybe some more ones. And here it has sin theta or maybe minus sin theta and sin theta here and then zeros everywhere else. So, basically this is the ith row and this is the jth row and this is the ith column and this is the jth column. Okay, so the ith row and so it's basically the identity matrix except in the i and jth column where the column i column j sub matrix forms a 2 cross 2 matrix with cos theta minus sin theta and cos theta as its four elements. So, this is called a plane rotation. The second one is called the householder transform. It's named after the mathematician who came up with it not because it has anything to do with householders. So, let W be a vector in c to the n which is non-zero then Q W is defined as i minus 2 W W Hermitian divided by W Hermitian W. Okay, so an exercise for you is to verify that these two are indeed unitary matrices. So, this is what I wanted to say about unitary matrices and unitary equivalence but we are going to use that immediately in this next result which we are going to discuss which is Schur's unitary triangularization theorem. Yes? Sir, we understand the use of plane rotation matrix but what is the use of this householder transform? I mean can it carry some application? So, it's useful for example in computing the QR decomposition of a matrix. It's also useful in so one way to decompose a matrix is to write the matrix as the product of a series of householder transforms and a diagonal or an upper triangular matrix followed by another series of householder transforms. So, it's mainly used in at least one of the uses is in matrix factorizations into simpler forms. Okay, so thank you sir. So, this Schur's unitary triangularization theorem is a very important result in linear algebra. In fact, what Hohn and Johnson says about this theorem is that it is perhaps the most fundamentally useful fact of elementary linear algebra. So, if there is one theorem that you want to take away from this course, this could be one of them. So, essentially the theorem says that any complex n cross n matrix is unitary equivalent to an upper triangular matrix. And so, while this kind of decomposition of A and showing it to be unitary equivalent to an upper triangular matrix is far from unique, it still represents the simplest form that one can achieve using unitary equivalents. And of course, because the unitary equivalents preserves all the eigenvalues, obviously if you can find the unitary equivalent upper triangular matrix, then the diagonal entries of that upper triangular matrix are the eigenvalues of this matrix. So, the decomposition also reveals the eigenvalues of the matrix. So, let us formally write down what the theorem is. So, given c to the n cross n with eigenvalues lambda 1 to lambda n, there is a unitary matrix U such that U Hermitian Au is equal to T, which we will write as its elements as Tij. This matrix T, this U Hermitian Au equal to T is upper triangular with diagonal elements lambda 1 through lambda n. Further, A and all its eigenvalues real may be chosen to be real and orthogonal. So, this is the theorem. I will just sort of wait for a few seconds for you to read it again, because it is such an important theorem. So, you can look at it for a few seconds. So, quick question, if a matrix is, all the entries of a matrix are real valued, isn't it necessary that all its eigenvalues are real? Yeah, what is an example of a very simple, the simplest 2 cross 2 matrix you can think of, which has real valued entries, but complex valued eigenvalues? Sir, I think cos theta sin theta minus sin theta cos theta. So, all the entries are real value and what are its eigenvalues? We have to actually find it. So, let us do it just for the fun of it. So, if I do determinant of lambda i minus this matrix equals 0, I will get lambda minus cos theta the whole square minus and then it becomes plus sin square theta equals 0. Check me, check and let me know if I make a mistake. So, if I simplify this, this gives me lambda square minus 2 lambda cos theta plus 1 equals 0 and its roots are lambda equals 2 cos theta plus or minus square root of 4 cos square theta minus 4 divided by 2. Is that correct? So, let me know if I make a mistake. So, that is lambda equals cos theta plus or minus i times square root of 1 minus cos square theta is sin square theta. So, correct? So, basically its eigenvalues are complex even though the matrix is real valued. So, you cannot run away from complex numbers if you want to study matrices and its eigenvalue properties. So, this is an aside. So, now let us prove this theorem. So, that is why we need to say if A and all its eigenvalues are real, then it is true that you can choose U to be a real orthogonal matrix. So, let X1 be a unit norm eigenvector of A associated with lambda 1. So, this is a unit norm vector. What we will do is we will extend X1 to a basis. That is you find other vectors that are linearly independent of X1 such that X1, Y2 etc up to Yn form a basis of c to the n. Then we will apply Gram-Schmidt. This will give me a set of vectors which are all unit norm which I am going to call X1 say of course the first vector is already unit norm. So, when you apply Gram-Schmidt that does not change the first vector but it will change the second third all that. So, we will call it Z2 up to Zn which are orthonormal vectors forming a basis of c to the n. So, this is an orthonormal basis. Then what we will do is we will let U1 be the matrix X1, Z2, Zn. So, we will start with this matrix. Now, if I consider what happens to the first column of AU1 that is the same as X1 times A. The first column of U1 is X1. So, the first column of this product AU1 will be X1 times A which is equal to lambda 1 times X1. And also if I did U1 Hermitian times X1 what I will get is U1 is this matrix it is an orthonormal matrix. So, if I do U1 Hermitian X1 this is going to be a vector whose first element will be X1 Hermitian X1 which is equal to 1. The second element will be Z2 Hermitian X1 which is 0. The last element will be Zn Hermitian X1 which is also 0. So, this is actually just the vector 1 0 0. So, this means that if I do U1 Hermitian AX1 AU1 then I get the first column of this product is actually this lambda 1 times this column here. So, that will just give me lambda 1 0 0 and over here in the first row I will get some entries I actually do not care about them and I will call whatever I get down here as the matrix A1. Now, lambda 1 is here. So, what can I say about the eigenvalues of A1? The eigenvalues of A are lambda 1 to lambda n and I found a unitary matrix U and computed U1 Hermitian AU1 and this matrix has lambda 1 followed by zeros here. So, it is a block upper triangular matrix with lambda 1 up here and an A1 matrix down here. And so, what will be the eigenvalues of A1? Lambda 2 to lambda n. Exactly. The other eigenvalues will be the eigenvalues of A1. Now, what we will do is the same idea exactly but repeated with respect to A1. So, let X2 and now X2 will be in C to the n minus 1. This A1 is of size n minus 1 cross n minus 1. So, let this be let X2 n minus 1 be unit norm eigenvector of A1 associated with lambda 2. Then, again as before we extend X2 to form a basis of C to the n minus 1. So, the proof is constructive. So, we show that you can construct a matrix satisfying U Hermitian AU is equal to t where t is an upper triangular matrix with diagonal entries lambda 1 to lambda n. We are just constructing this matrix. So, this is X2 and say Y2 up to Y n minus 1. So, I need to find another n minus 2 vectors. There is one vector already. So, these n minus 2 vectors together form a basis for C to the n minus 1. And then we will apply Gram-Schmidt and we will get an orthonormal matrix which we will call U2 which is equal to X2 Z2 Z n minus 1. And this U2 is such that U2 Hermitian A1 U2 will be equal to what? It will be exactly in this form here. So, it will have lambda 2 here and zeros everywhere else in the first column. And a matrix which we will call A2 down here and here it has something which we do not care about. And again this matrix will have eigenvalues lambda 3 to lambda 2 lambda n and its of size n minus 2 cross n minus 2. But ultimately we want to multiply with the matrix A. So, it is not enough if we can find a U2 such that this happens. We need to find a matrix that I can multiply with A. So, let V2 be the matrix 1 0 0 and then zeros in the first row and U2 down here. Then V2 is unitary. You can verify this by direct multiplication V2 Hermitian V2 will give you the identity matrix. So, if V2 is unitary clearly or rather remember maybe that U1 V2 the product of unitary matrices is also unitary and U1 V2 Hermitian A U1 V2 will give us the matrix lambda 1 zeros and lambda 2 in the second row zeros and this can be something. And here also you may have something out here but you will have the matrix A2 here. So, now we lambda 1 lambda 2 on the diagram. Yes. Sir, could you explain V2 once more please? V2 is just a matrix where I have padded 1 0 0 0 on the left and then this the all 0 vector on this part on the right on the top. So, it is now this matrix is now of size n cross n. Okay. So, we have just extended U2 to n cross n. Yes. So, basically on the side on the left and top you insert zeros but in the top left you insert a 1. Okay. So, basically now we see the pattern. So, we continue this process to get this matrix U which is equal to U1 V2 up to Vn minus 1 which is unitary and U Hermitian A V A AU is equal to T which is in the desired form. Basically upper triangular with all the eigenvalues going on the diagram. Now for the last part of the theorem if A and its eigenvalues are real then the eigenvectors can be chosen to be real and all the above arguments can be applied with real arithmetic so that that that verifies the last assertion. So, if A and its eigenvalues are real all the above. Okay. Let me put it this way. Okay. The eigenvector of A can be chosen to be real and all the above steps done with real arithmetic. So, that is the proof. There is another version of this theorem for considering strictly real matrices and basically as we have seen strictly real matrices don't necessarily have real eigenvalues but it turns out that if the, so here is the statement if a matrix is real valued then its eigenvalues always occur in complex conjugate pairs. Why is that true? So, can I please repeat? If a matrix is real valued then the eigenvalues even if they are complex valued always occur in complex conjugate pairs. Sir trace is real and the trace is sum of all eigenvalues. Okay. So, if the sum of all eigenvalues is real does that mean that the eigenvalues must occur as complex conjugate pairs? Yes sir. Then the sum of the conjugate pairs becomes real if any one of them. That goes the other way right. If I take complex conjugate numbers and add them up I will get a real number. Yes sir. It doesn't mean that the only way to get real numbers is by adding complex conjugate pairs. Sir, can we say that the root of the characteristic polynomial's eigenvalue and the roots always occur in complex conjugate pairs? That is correct but why is that true? Sir, that I don't know. That's because the characteristic polynomial's coefficients okay are some strange combinations of the entries of the matrix A right and if A is real valued the coefficients of the characteristic polynomial are all real valued okay. So, if I have an equation like so again this is an aside. If I have an equation like say An lambda to the n plus An minus 1 lambda to the n minus 1 plus dot dot dot plus A naught equals 0. This is the characteristic equation. All these coefficients come from the matrix A. Okay they are just some strange combinations of the entries. It's a multi-multinomial combination of the entries of the matrix A and so all these are real valued okay. So, if there is a lambda naught for which this is true okay. If I just take the complex conjugate of this equation then 0 the complex conjugate of 0 is 0. So, I have that An lambda naught star to the n plus An minus 1 lambda naught star to the n minus 1 plus etc plus A naught is also equal to 0. So, basically if lambda naught is a 0 of the characteristic polynomial then lambda naught star is also a 0 of the characteristic polynomial. So, they always occur in complex conjugate pairs. Of course, if lambda naught is real valued lambda naught star is a root of the characteristic polynomial but it doesn't mean that it needs to be a it has to be a repeated root. It could be a solitary root okay. But in that case saying that if lambda naught is a root lambda naught star is also a root is not saying anything else anything new because lambda naught and lambda naught star are actually the same number okay. So, let's now discuss the next theorem. If A then there is a real orthogonal matrix q in r to the n cross n also real valued matrix such that q transpose A u A q is equal to a big matrix containing A 1, A 2 down to some A k along the diagonal zeros here and arbitrary things above the diagonal which is also real valued. Of course here all these are real valued. So, when I take their product it cannot suddenly become complex valued where each A i is real 1 cross 1 matrix or real 2 cross 2 matrix with a non real pair of eigenvalues okay. So, the only difference that it makes is that you won't necessarily get an upper triangular form you will get a block upper triangular form where these blocks are either 1 cross 1 blocks or 2 cross 2 blocks. If they are 1 cross 1 blocks they correspond to real valued eigenvalues of the matrix A and if they are 2 cross 2 blocks they correspond to non real eigenvalues of A which occur as complex conjugate pairs okay. I won't prove this theorem the proof is actually similar to the previous theorem and you can see the text but we will discuss some consequences of the short triangularization theorem because we've said it's a very useful result. So, let's discuss some outcomes or some interesting things you can show by using this theorem. Okay first before that yes go ahead please. Sir in the theorem we have told that if A have A and eigenvalues are real then U is real and orthogonal sir orthonormal matrix sir that you so for example A is orthonormal only. Yeah so my notation is that an orthon so when I say orthonormal matrix what I mean is I mean a matrix satisfying U Hermitian U is the identity matrix okay. When I say real orthogonal matrix I mean a matrix Q such that Q transpose Q is the identity matrix and the entries of Q are real okay. So there is a slight abuse of nomenclature here but when I say real and orthogonal it actually means real and orthonormal but I'm using orthonormal to generally represent complex valued matrices so instead of saying complex orthonormal and real orthonormal I'm saying orthonormal which is for the complex case and real orthogonal for the real case but really the columns of this real and orthogonal matrix are all unit norm and the columns are all orthogonal to each other. Yes sir thank you sir. So there is one interesting property I'll first discuss and then we'll show one cool result that you can show or establish by using the using short triangularization theorem. By the way have any of you seen this so suppose A and B are n cross n upper triangular matrices okay such that they have some special structure both are upper triangular but A is of the form zero and then of course below this will be zero and then it has an upper triangular form here and this can be arbitrary where this is a k cross k block of zeros and B is of the form this k cross k block can be upper triangular and nonzero and of course below this is zero and here below this you will have a zero and of course below this since it's upper triangular it has to always be zero and here it can be arbitrary but this part is again upper triangular and this is arbitrary again so basically this here is the k plus one comma k plus one element so this k plus one k plus one element is zero here there is a k cross k block of zeros then if I consider the product AB anybody wants to guess what will be the structure in this matrix when you multiply these together basically when you multiply this with this you'll get zero and here you're multiplying a column which has zeros down here with with this matrix and so it'll lead to a matrix which has a zero which is of size k plus one cross k plus one and of course below that it's all zeros and here it's arbitrary and here it is again upper triangular so products of upper triangular matrices is upper triangular but this is a special extra structure that I'm imposing as a consequence of which this matrix has a k plus one cross k plus one block of zeros so you can verify this by direct multiplication by considering entries with you know BIJ as the entries here and AIG of the entries here and see what happens when you take them take the product but this is true this property holds true okay so now we'll use this property in the result