 In our last lecture, we have considered Eigen values of some special matrices, like if matrix is self adjoint, then we saw that Eigen values are real. If a is q self adjoint, that means the conjugate transpose is equal to minus of the matrix a, then the Eigen values are purely imaginary or they are 0. Then for normal matrix, that means if a star a is equal to a a star, we saw that if lambda is an Eigen value of a, then lambda bar the complex conjugate is Eigen value of a star, whereas Eigen vector it remains the same. Now, using this result, we are going to show that for a normal matrix, Eigen vectors corresponding to distinct Eigen values, they are going to be perpendicular to each other. If we are looking at a general matrix, then Eigen values corresponding to distinct Eigen values, they are linearly independent. For normal matrices, we have got something more. So, we are going to look at this and then we will consider Eigen values of unitary matrices. So, our notation is a is n by n, either real or complex matrix, a u is equal to lambda u, where lambda is a complex number and u is a non-zero vector in C n. So, this is a star, a star is equal to a bar transpose, that is the conjugate transpose. Self adjoint matrix, that means a star is equal to a, they have real Eigen values, skew self adjoint a star is equal to minus a, then the Eigen values are either purely imaginary or 0. For a normal matrix, a star a is equal to a star and using this, one shows that norm of a x to norm, Euclidean norm is going to be equal to norm of a star x, it is to norm. And as a consequence of this result, a u is equal to lambda u, if and only if a star u is equal to lambda bar u. Now, let us look at a unitary matrix, that means a star a is equal to a star is equal to identity. So, we have in particular unitary matrix is a normal matrix. So, a u is equal to lambda u, apply a star. So, we will have a star a u is equal to lambda times a star u, but a star u will be lambda bar u. So, it is going to be equal to lambda, lambda bar u and thus we get u is equal to modulus of lambda square u. Since u is a non zero vector, it implies that mod lambda is equal to 1. So, thus for a unitary matrix, all the eigenvalues they are going to lie on unit circle. So, now, we look at the case of distinct eigenvalues for normal matrices. So, a star a is equal to a star, let a u be equal to lambda u, a v is equal to mu times v. So, we look at two distinct eigenvalues lambda and mu and u and v are associated eigenvector. So, we want to show that inner product of u with v is going to be equal to 0. So, u is going to be perpendicular to v. So, we have a star a is equal to a a star, a u is equal to lambda u, a v is equal to mu times v, where lambda is not equal to mu, u being a eigenvector, it is not a zero vector, v also is a non zero vector. So, let us look at. So, consider lambda times inner product of u with v, this will be lambda u comma v using linearity of inner product in the first variable. So, this lambda goes inside as lambda u. Now, a u is equal to lambda u. So, this will be inner product of a u with v, we have seen that a will go to the second variable as a star. So, it will be u a star v. Now, since a v is equal to mu times v, a star v will be equal to mu bar v. So, this will be equal to u mu bar v. And now, the inner product is conjugate linear in the second variable. So, this mu bar will come out as mu. So, this will be mu times inner product of u with v. Since lambda is not equal to mu, we get inner product of u with v to be equal to 0. So, if u and v are eigenvectors corresponding to distinct eigenvalues, then we get them to be perpendicular. So, this is property of normal matrices. In general, it will not be true. Now, what we want to do is we want to consider similar matrices. We want to show that similar matrices, they have the same set of eigenvalues and algebraic multiplicity as well as geometric multiplicity that is going to be preserved. So, let us first define what is a similar matrix. So, we have a and b two matrices. They will be similar if there exists an invertible matrix P such that P inverse A P is equal to b. So, this is definition of similar matrices. We are going to show that they have they are going to have the same eigenvalues. So, when you want to find eigenvalues of A, if you want to simplify your matrix, then what is allowed is similarity transformation. Elementary row transformations which we used in Gauss elimination method, they will change the eigenvalues. So, they are not allowed, but similarity transformations, they will be they will preserve eigenvalues with algebraic multiplicity as well as geometric multiplicity. So, this is two matrices A and B of the same size are said to be similar if there exists an invertible matrix P such that b is equal to P inverse A P. Now, this is our claim that similar matrices, they have the same set of eigenvalues. So, what we are going to do is we are going to look at the characteristic polynomial. We will show that the characteristic polynomial of matrix B is same as the characteristic polynomial for matrix A. The eigenvalues are nothing but roots of the characteristic polynomial. So, if we show that they have the same characteristic polynomial, it will mean that they will have the same eigenvalues. Then the algebraic multiplicity is defined as you factorize. So, you have got characteristic polynomial in that suppose lambda 1 is one of the eigenvalue, then you look at lambda 1 minus lambda raise to m 1 you factorize. So, whatever is the power that is the algebraic multiplicity. So, once we show that matrix B and matrix A, they have the same characteristic polynomial, it will also follow that the algebraic multiplicities they are preserved and showing the characteristic polynomial they are the same is by using properties of determinant. So, we have to look at say we have got our B is equal to P inverse A P, then we look at determinant of B minus lambda i that is the characteristic polynomial. This is going to be equal to determinant of P inverse A P minus lambda i substituting for B. This will be determinant P inverse A P minus lambda times P inverse P. For the identity I write P inverse P, this is determinant P inverse A minus lambda i P. Now, using the property of determinant this is going to be equal to determinant P inverse determinant A minus lambda i and determinant P. Now, let me combine these two. So, that is going to be determinant P inverse P determinant A minus lambda i and determinant of identity matrix is 1. So, we get determinant of A minus lambda i. So, thus both B and A they have the same characteristic polynomial and hence the same set of eigenvalues with preservation of algebraic multiplicity. Now, let us look at geometric multiplicity. The definition of geometric multiplicity is number of linearly independent eigenvectors associated with the eigenvalue. So, now, we want to show that matrix B which is P inverse A P and matrix A they have eigenvalues with the same geometric multiplicities. B is equal to P inverse A P minus lambda i. So, B U will be equal to P inverse A P U. Suppose, B U is equal to lambda U, then what we get is. So, we have got P inverse A P U is equal to lambda U. So, that implies A of P U is equal to lambda times P U. I am pre-multiplying by P to get this. Now, U not equal to 0 vector, it is an eigenvector. P invertible it implies that P of U also not equal to 0 vector. Because, if it were equal to 0, you can multiply by P inverse and get U to be equal to 0. So, that means, U eigenvector of B implies P U eigenvector of A and converse also is true. So, this is going to be if and only if. So, this is going to be if and only if. So, let us look at the number of linearly independent eigenvectors associated with B and lambda. Suppose, you have got U 1, U 2 up to U k. These are linearly independent associated B and lambda. That means, B of U j is equal to lambda times U j, j is equal to 1 2 up to k. Now, just now we have seen that this implies that A P U j is equal to lambda times U j. Our B was P inverse A P. So, this will mean that P U 1, P U 2 up to P U k. These will be eigenvectors associated with A and lambda. So, I am assuming that lambda is eigenvalue of B with geometric multiplicity k. Then I got k eigenvectors associated with A and lambda. If I can show that these are linearly independent, then that will mean that geometric multiplicity of lambda as an eigenvalue of A also is going to be k. So, see there is a 1 to 1 correspondence between eigenvectors of B and eigenvectors of A. U is eigenvector of B. That will mean that P into U is eigenvector associated with A and conversely. So, now, we are assuming that lambda is eigenvalue of B with geometric multiplicity say k. Now, we are assuming that then you look at k linearly independent eigenvectors associated with B and lambda. Then P U 1, P U 2, P U k these will be eigenvectors of A. So, only thing which remains to show is that P U 1, P U 2, P U k they are linearly independent. Now, they will be linearly independent because P is invertible matrix. If P is not invertible, then such a result is not true. So, let me show quickly the linear independence. So, we have got our assumption is U 1, U 2, U k these are linearly independent. This is given. Then P U 1, P U 2, P U k these invertible claim P U 1, P U k are linearly independent. And for the proof of this claim we start with alpha 1 P U 1 plus alpha k P U k is equal to 0 vector. If this implies that alpha 1, alpha 2, alpha k they all have to be 0, then that will mean that these are linearly independent. Now, in order to show that I am going to make use of the fact that U 1, U 2, U k are linearly independent and P is invertible. So, multiply by P inverse. So, we will have alpha 1, P inverse. So, we will have alpha 1, P inverse, alpha 1, P inverse, P U 1 plus alpha k, P inverse, P U k is equal to 0 vector. You are multiplying by P inverse, P inverse into 0 vector is 0 vector. Now, this is nothing but alpha 1, U 1 plus alpha k, U k to be 0 vector. And by using the fact that this is linearly independent, it follows that alpha 1 is equal to alpha k is equal to 0. So, this proves our claim. Similar matrices, they have got the same Eigen values with preservation of algebraic multiplicity and geometric multiplicity. So, if we have got matrices which are upper triangular matrices or diagonal matrices, we can calculate their Eigen values. In these two cases, the Eigen values, they are nothing but the diagonal entries. So, it will be good if I can find a invertible matrix P such that P inverse A P is either a diagonal matrix or an upper triangular matrix. Now, P inverse A P is equal to D. Such matrices, they are known as diagonalizable matrices. Now, P inverse A P is equal to D. Such matrices, they are known as diagonalizable matrices. That given a matrix A, if you can find a invertible matrix P such that P inverse A P is equal to D, then the matrix is called diagonalizable. All matrices, they are not diagonalizable. So, what we are going to do is, we are going to prove a characterization for the diagonalizable matrices and then using that characterization, one can give a counter example to show that not all matrices, they are diagonalizable. So, a matrix A is said to be diagonalizable if there exists an invertible matrix P and a diagonal matrix D such that P inverse A P is equal to D. So, this is the definition. Now, what is P inverse A P is equal to D diagonal matrix D 1 1 D 2 2 D n n. So, this will mean that A into P is equal to P into D. Let me write columns of P as P 1 P 2 P n. So, we have got A times P 1 P 2 P n is equal to P times D. So, when you consider A and then P 1 P 2 up to P n is equal to P into D. Now, this is nothing but A P 1 A P 2 up to A P n that is the property of matrix multiplication and what will be P into D? It will be P 1 P 2 up to P n multiplied by diagonal matrix D 1 1 D 2 2 D n n. So, when you post multiply by a diagonal matrix the first column will be D 1 1 P 1. The first column will get multiplied by D 1 1, second one will be D 2 2 P 2 and D n n P n. So, this is first column, second column, nth column. Here, you have got first column, second column, nth column and hence what we have is A P j is equal to D j j P j. P is invertible matrix P j is its j th column because it is invertible P j will not be a 0 vector. If you have got a 1 column to be 0 then that matrix is not invertible. So, this will mean that D j j they are eigenvalues of A and P j these are eigenvectors. P j is j th column of P and again since P is invertible P 1 P 2 up to P n this set is linearly independent so thus our matrix A will be diagonalizable if and only if you have got n eigenvectors linearly independent associated with A. We have seen example of a 2 by 2 matrix for 2 by 2 matrix. For that matrix it was upper triangular matrix with all entries to be equal to 1. So, the only eigenvalue is 1 and there was only 1 eigenvector associated. So, this one eigenvalue 1 had algebraic multiplicity 2 geometric multiplicity 1. So, only 1 linearly independent eigenvector. So, it is a 2 by 2 matrix and you have got only 1 linearly independent eigenvector. So, now we have seen that a matrix is diagonalizable if and only if you are looking at a matrix of size n then it should have n linearly independent eigenvectors. So, thus not all matrices they are going to be diagonalizable, but we will see there is a big class of matrices which is going to be diagonalizable. So, we have A is diagonalizable if and only if there are n linearly independent eigenvectors of A and that will mean that eigenvectors of A they will form a basis of C n because the dimension of C n is n it is a finite dimensional space. So, if there are n independent eigenvectors they will also form a basis. So, here is that example A is equal to 1 1 0 1 1 is the only eigenvalue 1 0 or any non 0 multiple of it is going to be an eigenvector. So, such a matrix is not diagonalizable. So, now not all matrices are diagonalizable, but when we are looking we are interested in eigenvalues of matrix. So, even if I cannot reduce it to a diagonal form if by using elementary not a single elementary row transformations by using similarity transformations if I can reduce it to upper triangular form then that will suffice because then I have got p inverse A p is equal to upper triangular matrix. The eigenvalues of A they are same as eigenvalues of upper triangular matrix and eigenvalues of upper triangular matrix they are the diagonal entries. So, whether all matrices they can be equivalent similarly or whether they can be similar to a upper triangular matrix. So, the answer is yes and that is Schur's theorem. So, we have got suppose A is n by n real or complex matrix then there exists a unitary matrix U. So, not only invertible, but we have got something better a unitary matrix U and an upper triangular matrix T such that U star A U is equal to T. Now, look at the statement of Schur's theorem it is very important that it says then there exists. So, the statement which we are making is existential that we are saying that there exists some U which is unitary such that U star A U is equal to T A and T they are going to have the same eigenvalues. There cannot be a constructive proof for this theorem because if you have got constructive proof it will mean that you can find its eigenvalues given any matrix A you can find its eigenvalues. Eigenvalues they are related to finding the roots of the polynomial. Now, as soon as your polynomial is of degree bigger than or equal to 5 there cannot exist a formula like if you have got a quadratic polynomial and then we know how to write down its roots. So, such a thing is not possible, but still it is an important theorem and the proof is by induction, but I am going to skip the proof. What we are going to do is we are going to look at the special cases like now we know that for any matrix U you can write U star A U is equal to T. So, what happens if your matrix is self adjoint or if it is Q self adjoint. So, let us see what one can deduce for this special matrices. So, we have got by Schur's theorem U star A U is equal to T where T is upper triangular. So, what will be T star? T star will be U star A U its star. Now, this will be A B star is B star A star. So, it will be U star A star and then U star its star. So, it is going to be equal to U star A star U. So, thus we have got T is equal to U star A U then T star will be equal to U star A star U. So, if A star is equal to A this will imply that T star is equal to T T upper triangular then T star is going to be lower triangular. So, you have got a lower triangular matrix is equal to upper triangular matrix and that will imply that T star is equal to U star A star U. So, this will imply that T is going to be a diagonal matrix D because left hand side is lower triangular right hand side is upper triangular. So, it has to be diagonal and because you have got D star is equal to D you are taking here conjugate transpose. So, that will mean that D is going to be real diagonal. Now, it fits in our whatever we have been saying the Eigen values of A are same as the Eigen values of T. Now, we showed that T is going to be a diagonal matrix and the Eigen values of A are the diagonal entries. It is going to be a real diagonal matrix. So, for self-adjoint matrix the Eigen values which are going to be the diagonal entries they are going to be real. This part we have seen earlier that T is going to be a diagonal matrix and self-adjoint matrices they have got real Eigen values. Now, the same idea or the same proof it tells us that if A is skew self-adjoint then you are again T will be a diagonal matrix, but now the entries diagonal entries they will be either 0 or purely imaginary. So, we have got u star A u is equal to T u star A star u is equal to T star A star is equal to minus A implies T star is equal to minus T this is lower triangular this is upper triangular. So, it has to be equal to a diagonal matrix D and D star will be equal to minus D. So, diagonal entries will be 0 or purely imaginary. Now, let us look at normal matrix. So, for the normal matrix it is not immediate to see that in this case also T is in fact a diagonal matrix. So, that we are going to do as a tutorial problem. So, that means the self-adjoint matrices then skew self-adjoint matrices and more generally normal matrices they are going to be diagonalizable. For diagonalizability what we wanted was existence of an invertible matrix such that P inverse D is equal to D. Now, we have got something more we have got a unitary matrix where u star is equal to u inverse. So, we have got u star A u is equal to D for normal matrices and this is known as spectral theorem. So, we have for the normal matrix let me write down u star A u is D, u star A star u is equal to T star. So, consider T T star that is going to be equal to u star A u and u star A star u this is identity. So, this will be u star A A star u now normal matrix. So, this will be u star A star A u and now let me introduce u u star here which is identity A u and this is nothing but T T star. And this implies T to be a diagonal matrix that is going to be a tutorial problem. So, normal matrices they are going to be diagonalizable. So, not all matrices are diagonalizable, but at least we have got a big class of matrices which is the class of normal matrices they are going to be all diagonalizable. Now, there is another class of matrix that is if your matrix has n distinct eigenvalues in that case the corresponding eigenvectors they are going to be linearly independent. So, in C n we will have n linearly independent vectors they will form a basis and we saw that diagonalizability it means existence of a basis of eigenvectors. So, if you have got a matrix with distinct eigenvalues then you are going to have p inverse A p is equal to D. In this case we will have matrix p will be only invertible it need not be unitary. The unitary matrix that is for normal matrices. So, we have this is the spectral theorem if A is normal that means A star A is equal to A A star then there exists a unitary matrix u and a diagonal matrix D such that u star A u is equal to D. So, once again I want you to notice that there exists we are not giving a recipe if you we could have done that would have been ideal but that is just not possible. So, we are going to have u star A u is equal to D. Now, using spectral theorem I want to calculate or I want to get an expression for Euclidean norm of matrix A we had defined matrix norms induced matrix norm. So, we had norm A 1 norm A infinity and norm A 2 norm A 1 is nothing but column sum norm norm A infinity is the row sum norm. So, these two norms you can compute we have got a formula in terms of the elements of the matrix whereas, for norm A 2 we had only an upper bound upper bound is that Frobenius norm. We are going to show that if A is a normal matrix then norm A 2 will be modulus of the biggest eigen value. Now, before we do that I want Euclidean norm A 2 you to notice that u star u is equal to identity it means the columns of u they are orthonormal. So, we have got u is a matrix I denote its columns by u 1 u 2 u n u star will be u 1 star u 2 star and u n star. So, when I look at u star u this is going to be this multiplied by this. So, it is going to be u 1 star u 1 u 1 star u 2 u 1 star u 1 and u 1 star u n then u 2 star u 1 u 2 star u n and u n star u 1 u n star u n and this is equal to identity matrix it will mean that u 1 star u 1 will be 1. So, that is nothing but the Euclidean norm of u 1 u 1 star u 2 will be 0 which will mean that u 1 and u 2 are perpendicular similarly u 1 and u n will be perpendicular. So, this u 1 u 2 u n which are columns of u. So, we are going to have columns of u are orthonormal. So, if you have got a invertible matrix then its columns they are linearly independent. If you have got a unitary matrix that means the inverse of u is nothing but its conjugate transpose. Then the columns of u they are orthonormal that means any two distinct columns they will be mutually perpendicular and the Euclidean length of each column vector is going to be equal to 1. Similar result is true for row vectors. So, for the row vectors we have to use the fact that u u star is equal to identity. Now, we have got by spectral theorem for normal matrix u star a u 1 star u 1 star u u star u is equal to d. So, the entries on the diagonal of d those are our Eigen values and columns of u those are our Eigen vectors. So, that means for normal matrix we have got Eigen vectors to be orthonormal. We already saw this for distinct Eigen values. If you have got a to be a normal matrix lambda and mu to be distinct Eigen values corresponding Eigen vectors they are mutually perpendicular. What we are saying now is if a is a normal matrix there is a basis of there is an orthonormal basis which consists of Eigen vectors. So, we have u star u is equal to identity that means inner product of u j with u i is 1 if i is equal to j and 0 if i not equal to j. The columns that means the columns of u are orthonormal u star a u is equal to d that means a u is equal to u into d. So, the columns of u are nothing but Eigen vectors. So, we can state spectral theorem as if a is normal then a has n orthonormal Eigen vectors not just vectors, but Eigen vectors. So, you have got a u j is equal to lambda j u j j is equal to lambda j u j j is equal to lambda j is equal to 1 2 up to n. These lambda j's they need not be distinct those are the Eigen values which may be repeated. Now, consider any z belonging to c n a vector in c n u 1 u 2 u n is going to form a basis. So, I can express z as a linear combination of u 1 u 2 u n. So, z is summation j goes from 1 to n alpha j u j. If you take inner product of z with u k this is going to be equal to summation j goes from 1 to n alpha j u j u k. Using linearity of inner product in the first variable this will be summation j goes from 1 to n alpha j u j comma u k. This is going to be 1 only when j is equal to k. So, this is equal to alpha k. So, any vector z can be written as summation j goes from 1 to n z comma u j u j. So, we have for normal matrix a u j is equal to lambda j u j norm u j to be equal to 1. Let me arrange Eigen values lambda 1 in the descending order of modulus mod lambda 1 bigger than or equal to mod lambda 2 bigger than or equal to mod lambda n. Then just now we saw that z can be written like this combination. So, a z will be take a inside a u j is equal to lambda j u j. So, this is going to be for a z and now norm z square is going to be nothing but summation j goes from 1 to n modulus of z comma u j square. So, we are going to look at the what I am trying to show is Euclidean norm of a or norm a 2 is going to be equal to modulus of lambda 1. So, for that we make use of the fact that if a is normal there are n orthonormal Eigen vectors. Write any vector z as a linear combination consider a z because for norm a 2 we have to look at maximum of norm a z by norm z. So, this proof I will complete in the next lecture and then we are going to look at the some localization results for Eigen values and then a proxy method methods for calculating Eigen values. Thank you.