 we are considering q r decomposition of an invertible matrix. Today, we are going to show equivalence of q r decomposition with Gram-Schmidt orthonormalization process. Then by putting the condition that diagonal entries of r should be greater than 0, we will prove its uniqueness and then we are going to consider q r decomposition by using reflectors. So, that is going to be an efficient way of calculating a q r decomposition, because q r method for finding eigenvalues of a matrix A, the procedure will involve repeated q r decomposition of certain matrices. So, we want an efficient method for finding q r decomposition of a matrix, then I will describe what is a q r method and then we are going to consider some example. So, our matrix A is invertible, we want to write it as q into r, where q is orthogonal matrix that means q transpose q is identity, r is a protrangular matrix, c 1 c 2 up to c n, these are columns of our matrix A. q 1, q 2, q n, these are columns of matrix q. Yesterday, we saw that q orthogonal it means the columns of q, they are going to form an orthonormal set that means each column vector is going to have Euclidean norm to be equal to 1 and if you consider inner product, standard inner product on R n, so q i comma q j, the inner product of q i with q j will be 0 if i not equal to j. For simplicity, we are assuming A to be a real matrix, a real invertible matrix, r upper triangular it means it is going to be of this form, below the diagonal all the entries they are going to be equal to 0. Now, what we are going to do is multiply and equate the columns, so we are going to have c 1 to be equal to r 1 1 multiplied by q 1. When you post multiply by such a matrix, the first column q 1 will get multiplied by r 1 1, then c 2 the second column will be r 1 2 times first column q 1 plus r 2 2 times second column q 2 and so on. Now, what is given to us is matrix A that means c 1, c 2, c n these are known. What we need to find is columns of q are going to be equal to 1 and entries of this upper triangular matrix are. So, if I look at the first one c 1 is equal to r 1 1 q 1, I know that Euclidean norm of q 1 to norm, norm q 2, norm q 1 to norm is going to be equal to 1. So, take norm of both the sides, so you are going to have norm of c 1 to norm is equal to modulus of c 1 into norm q 1 that is going to be equal to 1. So, mod r 1 1 is equal to norm of c 1, thus for r 1 we have choice, you can either choose it to be plus norm of c 1 or minus norm of c 1. So, thus we have determined r 1 1, once you determine r 1 1 q 1 is going to be c 1 divided by r 1 1. So, you have determined q 1, you have determined r 1 1. Now, look at the second column, so we need to determine r 1 2, r 2 2 and q 2, q 1 is already determined. So, what we will do is we will use the fact that q 1 and q 2 they are equal to 1 perpendicular. So, if I take inner product of c 2 with q 1, then the contribution from this term is going to be 0. So, we have c 1 is equal to r 1 1 q 1, c 2 is equal to r 1 2 q 1 plus r 2 2 q 2 and in general the j th column c j will be equal to r 1 j q 1. So, this is first entry in the j th column of r, then there will be r 2 j q 2 and then r j j q j. So, from here we see that span of c 1 c 2 c j is going to be equal to span of q 1 q 2 q j, j going from 1 to up to n. So, c 1 and q 1 they are multiples of each other, c 2 is a linear combination of q 1 and q 2. So, when I consider a linear combination of c 1 and c 2 that is going to be linear combination of q 1 and q 2 and so on. So, c 1 and q 2, so we have got this property, then c 1 is equal to r 1 1 q 1. So, take norm and then you are going to have modulus of r 1 1 to be equal to norm of c 1 to norm, then c 2 is r 1 2 q 1 plus r 2 2 q 2, take the inner product with q 1. So, inner product of c 2 with q 1 is equal to r 1 2 times inner product of q 1 with itself plus r 2 2 times inner product of q 2 with q 1, this is 0. So, r 1 2 is going to be inner product of c 2 with q 1, we have already determined r 1 1 q 1 1, now we have determined r 1 2. So, what remains to be determined to be determined are r 2 2 and q 2. When you consider r 1 1 q 1, then r 2 2 and q 2, this is going to be c 2 minus r 1 2 which is inner product of c 2 with q 1 multiplied by q 1. So, take norm of the both the sides, norm q 2 is 1, so modulus of r 2 2 is going to be Euclidean norm of this vector. So, once again for r 2 2 which is a real number, you have got choice, either you can choose it to be bigger than 0 or you can choose it to be less than 0. Next, a general case will be c j is equal to r 1 j q 1 plus r 2 j q 2 plus r j j q j, we would have determined q 1 q 2 q j minus 1, they are orthonormal vectors, they have already been determined. So, look at inner product of c j with q k, this is going to be summation r i j q i, i goes from 1 to j, this expression I am writing in the compact form as a summation q k by using linearity of the inner product in the first variable, you get summation r i j inner product of q i with q k, the only term will remain when i is equal to k. So, you will have r k j, so thus r 1 j r 2 j up to r j minus 1 j will be determined. Now, what remains to be determined is the coefficient r j j and vector q j, vectors q 1 q 2 q j minus 1 we have already determined. So, we have c j is equal to just now we saw that r 1 j is going to be inner product of c j with q 1 and r i j will be inner product of c j with q i, so I substitute plus r j j q j, all these things they are known. So, take them on the other side, so you are going to have c j minus this, take norm of both the sides norm of this is going to be equal to modulus of r j j, r j j is going to be either bigger than 0 or less than 0, we can decide once you determine r j j q j will be determined from this expression. So, we have a is equal to c 1 c 2 c n the columns of a, columns of q are q 1 q 2 q n, this is a orthogonal matrix r is equal to r i j upper triangular matrix, for the sake of definiteness choose r i i to be bigger than 0, so r 1 1 will be plus norm of c 1 2 and q 1 will be c 1 upon norm c 1, then for j is equal to 2 3 up to n r i j's are going to be inner product of c j with q i, let s j be c j minus this summation, then r j j will be 2 norm of this s j and q j is going to be equal to s j divided by its norm. Now, this is nothing but Gram-Schmidt orthonormalization process, we had considered Gram-Schmidt process, so if we have n linearly independent vectors, then we can consider the normalization process and we can construct a set of n orthonormal vectors which have the property that span of c 1 c 2 c j is same as span of q 1 q 2 q j. So, thus q r decomposition of a is nothing but Gram-Schmidt orthonormalization process applied to columns of a, whatever orthonormal vectors you get that is going to form our orthogonal matrix q and various coefficients r i j, they form our upper triangular matrix r. Now, we have seen that at each stage when we want to consider or when we want to determine the diagonal entries of r, we had choice to choose our diagonal entry to be bigger than 0 or it to be less than 0, which will mean that the q r decomposition of our matrix a is not unique, because for each diagonal entry I have a choice, I can choose it either to be greater than 0 or less than 0. So, any invertible matrix a can be written as q into r, the decomposition is not unique. If we want uniqueness, then you have to fix or you have to put some condition on diagonal entries of our matrix r. So, let us put the condition to be all the diagonal entries they should be bigger than 0, it is just one of the condition. I can as well put that all the diagonal entries should be less than 0, then such a decomposition will be unique or I can say that all even entries should be bigger than 0, all odd diagonal entries should be less than 0, it is just you have to put some conditions on the diagonal entries. So, let us prove uniqueness of q r decomposition using or with the condition that the diagonal entries of r they are bigger than 0. Now, let me recall earlier decompositions which we had considered, we had considered l u decomposition of a matrix. So, a matrix a we were writing as l into u where l is lower triangular, u is upper triangular. Now, not all matrix matrices have l u decomposition, we had to put additional condition that when you will consider the sub matrix which is formed by first k rows and first k columns. So, leading principal sub matrix, if it is determinant is not equal to 0, for k is equal to 1 2 up to n, then your matrix a you can write as l into u and the converse is true. If such a case is there, then the l u decomposition is unique, then from l u decomposition we went to what is known as l d v decomposition. So, you have got l to be lower triangular in the l u decomposition again for this uniqueness we needed that the diagonal entries of l should be equal to 1. So, it is a unit lower triangular matrix. In the l d v decomposition, we had l to be unit lower triangular, v to be unit upper triangular and d was diagonal. Then such l d v decomposition it is unique, from the l d v decomposition we considered Cholesky decomposition which is valid for positive definite matrices. So, a is equal to g g transpose where g is a lower triangular matrix, then g transpose will be upper triangular matrix and the uniqueness is obtained provided you put some condition on the diagonal entries of g. So, one of the condition is assume that all the diagonal entries of g they are bigger than 0, then the Cholesky decomposition is unique. Now, this uniqueness of Cholesky decomposition we are going to use to prove uniqueness of q r decomposition with the condition that q is orthogonal matrix and r is upper triangular matrix with diagonal entries to be bigger than 0. So, suppose a is equal to q 1 r 1 is equal to q 2 r 2. So, what we are going to do is, so we have a is equal to q 1 r 1 is equal to q 2 r 2. We have got q 1 transpose q 1 is equal to identity is equal to q 2 transpose q 2 r 1 and r 2 these are upper triangular and r 2. So, these diagonal entries are bigger than 0. So, if I look at a transpose, a transpose will be nothing but r 1 transpose q 1 transpose. So, if I look at a transpose a, this will be r 1 transpose q 1 transpose q 1 and r 1. Now, q 1 transpose q 1 is equal to q 2 r 1 is identity. So, we have got r 1 transpose r 1. Now, this r 1 transpose is going to be lower triangular because our r 1 is upper triangular. So, this is nothing but Cholesky decomposition. Our matrix a is invertible. Hence, a transpose a is going to be positive definite matrix. So, we have a transpose a to be equal to r 1 transpose r 1 and it will also be equal to r 2 transpose r 2. We are considering a is equal to q 1 r 1 is equal to q 2 r 2. Now, by uniqueness of the Cholesky decomposition, it will imply that r 1 has to be equal to r 2. Now, if r 1 is equal to r 2, then q 1 is going to be r 1 inverse a because a is invertible. r 1 has r 1 is upper triangular matrix with diagonal entries bigger than 0. So, it is going to be invertible. This is same as r 2 inverse a which is q 2. So, thus we have got r 2 inverse a. A is equal to q 1 r 1 is equal to q 2 r 2. Then, a transpose a is equal to r 1 transpose r 1 which is same as r 2 transpose r 2. A invertible will imply a transpose a to be positive definite with positive diagonal entries. So, Cholesky decomposition because it is unique you get r 1 is equal to r 2 and then you get q 1 is equal to q 2. So, now, we come to q r method. So, the q r method is easy to describe. What is difficult is to prove its convergence and to take there are many different methods. There are many computational points which are involved. So, we will not have time to consider all the intricacies of the q r method. As I had said before, q r method is one of the most popular method for calculating Eigen values of a matrix at present. This q r method it can be considered as or it can be interpreted as what is known as simultaneous iteration. In case of power method, we were looking at only one vector and then we were defining iterate. So, instead of that one can consider several vectors and then one can define the q r method or with one can interpret. That will that gives some idea as to why the q r method work. So, let me describe what is a q r method. So, we are going to start with a matrix A. We know that invertible matrix A can be written as q into r, where q is going to be orthogonal matrix and r is going to be upper triangular matrix. So, here is q r method. So, the first step is write your matrix A as q 0 into r 0. That means you calculate q r decomposition of your matrix A. So, this is the first step. In the next step, what you do is you define a new matrix. You have obtained q 0 and r 0. So, now multiply them in the reverse order. So, you consider r 0 into q 0. The matrix multiplication is not commutative. So, you are going to get a matrix A. So, you have a different matrix A 1. After you have found A 1, now you find its q r decomposition. So, you look at the columns of A 1 orthonormalize and that will give you columns of q 1. So, you have A 1 is equal to q 1 r 1. Once you find these factors q 1 and r 1, you multiply them in the reverse order and when you do that, you are going to get a new matrix A 2. So, this procedure you continue. So, when you consider A m, suppose you have come up to m th step. So, that matrix A m, find its q r decomposition, multiply them in the reverse order and then obtain A m plus 1. Now, under some conditions, this A m plus 1 it is going to converge to an upper triangular matrix. So, it looks like a magic that you start with a matrix A. Then you find its q r decomposition, multiply it in a reverse order, get a new matrix. Again, you calculate the q r decomposition of this new matrix. Once you get the factors, multiply them in the reverse order, get a new matrix. You continue. When you do that, you are going to get a sequence of matrices. So, you are going to have A 1, A 2, A m plus 1. This A m plus 1, it converges to A m plus 1 and upper triangular matrix. And what I am going to show is whatever matrices we are constructing in the process, all these matrices, they are going to be unitary equivalent because for Eigen values, it is very important that the transformations which we are doing are they preserve the Eigen values. So, at each stage, we are getting say A 1, A 2, A 3 and so on. We are going to show that they are all unitary equivalent. So, if they are unitary equivalent, that means they are also similar matrices. So, that will preserve the algebraic that will first of all, it will have the same set of Eigen values. Algebraic multiplicities will be preserved, geometric multiplicities will be preserved. So, if this sequence A m, if it is converging to upper triangular form or upper triangular matrix, the Eigen values of this upper triangular matrix which are nothing but the diagonal entries, they are going to be Eigen values of our matrix A. So, you construct a sequence A m, it converges to an upper triangular form, look at the diagonal entries of this upper triangular matrix, those are going to be Eigen values you are interested in. So, my plan is first show that this sequence is unitary equivalent, then I want to show or I want to describe an efficient way of calculating q r d composition, efficient that means we have to take into consideration the stability. So, as such we have got Grimesmith orthonormalization process. So, apply Grimesmith orthonormalization process and get q r d composition, but there is a better way of doing it and that is using reflectors. So, I first I am first going to show that all these matrices are unitary equivalent and then consider reflectors. So, we have A m is equal to q m r m, A m plus 1 is equal to r m plus 1 is equal to r m q m, because that is our definition. So, if I pre multiply A m by q m transpose. So, I am going to have and post multiply by q m, q m transpose A m q m is going to be equal to q m transpose q m r m q m, this q m transpose q m is going to be identity, because it is an orthogonal matrix. So, you get r m q m. So, this will be equal to that is nothing but our A m plus 1. So, A m plus 1 is equal to q m transpose A m q m, q m transpose is inverse of q m and hence A m plus 1 and A m they are going to have same Eigen values with algebraic and geometric multiplicities preserved. Now, this one can substitute for A m. So, A m is going to be q m minus 1 transpose A m minus 1, q m minus 1, when you continue you are going to have q m transpose q m minus 1 transpose q 0 transpose A 0 q 0 q 1 q m, A 0 is our original matrix of which we want to find Eigen values and thus A m plus 1 is equal to q 0 q 1 q m transpose A 0 q 0 q 1 q m. Now, this matrix is going to be orthogonal, because the product of two orthogonal matrix matrices is again orthogonal. So, suppose you have got q 0 transpose q 0 is equal to identity and q 1 transpose q 1 is also equal to identity. I consider q 0 q 1 multiplication and then q 0 q 1 transpose. So, this is going to be equal to when you take the transpose the order gets reversed. So, you have q 1 transpose q 0 transpose q 0 q 1, this is equal to identity. So, you have got q 1 transpose q 1 and q 1 transpose q 1 is identity. So, you get q 0 q 1 also to be orthogonal. So, thus when you look at A m plus 1, then this q 0 q 1 q m that is going to be an orthogonal matrix, its transpose is going to be its inverse. So, A 0 and A m plus 1 they have the same Eigen values and under appropriate conditions A m plus 1 converges to U which is an upper matrix. So, next we are going to look at reflectors. So, we have suppose you have got two vectors x and y. So, we are looking at real vectors and suppose they have got the same Euclidean norm. So, x and y are distinct vectors with same Euclidean norm. Then we are going to show that there exists an orthogonal matrix q such that q into x is equal to y. Now, if I do this how does it help me? So, we want to reduce our matrix A to an upper triangular form by using say similarity transformations. So, we have got matrix A. We will look at its first column. Now, this first column is a vector. So, it is a vector. We want to transform this to a vector say sigma 1 and then remaining entry is 0. When we reduce A to upper triangular form what we do is in the first column below the diagonal we introduce 0. So, the same thing we want to do, but the whatever transformations we are using we want them to be the unitary transformations. So, that the Eigen values are preserved. So, here is our m x and y are vectors in R n with Euclidean norm to be equal to 1 and we want to find an orthogonal matrix q such that q x is equal to y. So, I am going to explain for the case n is equal to 2 and then we will see that this gets generalized. So, you start with two unit vectors u and v which are perpendicular. So, inner product of u and v it is equal to 0. So, we have a vector unit vector v here, then there is a perpendicular vector u which is also unit vector. You look at line L in the direction of v and we want to consider a reflection in this line L. Now, this reflection will be characterized by we are considering reflection in the line L. So, q of v should be equal to v and when you consider reflection of u it should be equal to minus u. So, we want to find such a q with the property that q v is equal to v q u is equal to minus u. Now, look at p is equal to u u transpose. So, we have got our vectors are unit vectors norm u u 2 is equal to norm v 2 is equal to 1 inner product of u with v is equal to 0, u is a 2 by 1 vector. I look at p is equal to u u transpose. So, this is 2 by 1 u transpose will be 1 by 2. So, p is going to be 2 by 2 matrix when I consider p of u this is equal to u u transpose. So, this is 2 by 1 u transpose will be 1 by 2. So, u transpose u u transpose u is nothing but norm u square. So, it is going to be equal to 1. So, p of u is equal to u and p of v is going to be equal to u u transpose v, but this is going to be equal to u and then inner product of v is equal to u. So, this is 0 with u this is 0. So, what I wanted was q of v to be equal to v and q of u is equal to minus u this is what we wanted what we have got is p of u is equal to u p of v is equal to 0. So, now, what I am going to do is I am going to do some manipulation. So, if I look at q is equal to identity matrix minus 2 times p then what will happen is q of v will be equal to v because p of v is equal to 0 q of u is going to be equal to u minus 2 times p u, but p into u is equal to u. So, q of u is equal to minus u. So, we have got a reflector that q what we wanted was this is our unit vectors which are mutually perpendicular our aim is to look at the reflector reflection in the line l. So, q of v is equal to v q of u is equal to minus u. So, consider q is equal to i minus 2 p where p is equal to u u transpose then it satisfies the desired condition. Now, let us look at the properties of the v of p. So, p is equal to u u transpose when I consider p square it will be u u transpose u u transpose this is going to be equal to 1. So, p square is equal to p a projection then p transpose is equal to p range of p is going to be equal to span of u since q is equal to i minus 2 p q transpose is equal to q and when I consider q square it will be i minus 2 p multiplied by i minus 2 p. So, it is i minus 2 p minus 2 p plus 4 p square p square is equal to p. So, you get q square is equal to identity. So, we have got I started with something I said that x and y are two vectors which have got the same Euclidean norm and I want to find the orthogonal matrix q which will transform one vector x into y this is what I wanted and then we looked at something else we looked at a reflector. So, we obtained an expression for reflection in a line L. Now, how does it help me in getting orthogonal matrix q which will transform vector x into vector y. So, there what we are going to do is we have got two vectors x and y they have got the same Euclidean length. Now, you consider a parallelogram with one side as vector x and another side as vector y. So, we have got say. So, here is my vector x. So, vector vector x vector y having the same length. Now, I look at a complete the parallelogram when I do that this vector is going to be vector x plus y. Now, consider reflection in this line in the direction of x plus y. So, the reflection of x is going to be precisely our vector y. So, if we want to say consider the earlier notation the v was unit vector in this direction. So, our v is going to be x plus y divided by norm of x plus y to norm. Our u was a vector which is perpendicular to our line. Now, the diagonal of this parallelogram they are going to intersect perpendicular or they are perpendicular. Now, this diagonal this was one diagonal x plus y. Another diagonal is going to be x minus y. So, your u is going to be x minus y divided by norm of x plus y. So, this is going to be x minus y divided by norm of x minus y to norm and then if you consider q is equal to i minus 2 times u u transpose then this will transform our x to y. So, this is our claim. So, let us work out the details of this. So, we have got norm x is equal to norm y x not equal to y. We want to find an orthogonal matrix q such that q x is equal to y. So, here is the parallelogram v is x plus y divided by its Euclidean norm. So, this is the parallelogram v is x plus y divided by norm of x minus y because I am dividing by its norm. We are going to get unit vectors. We assume that x is not equal to y. So, your u is going to be non-zero vector and now we are going to look at q is equal to identity minus 2 u u transpose because when we wanted to look at the reflection in the line L that is what we have obtained that look at q is equal to i minus 2 u u transpose. Such a matrix was orthogonal matrix. Now, u is equal to x minus y upon norm of x minus y v is equal to x plus y upon norm of x plus y p is u u transpose and q is i minus 2 p q of v is equal to v q of u is equal to minus u. So, I look at q x. This x I write as x plus y by 2 plus x minus y by 2, y by 2 will get cancel. So, I am adding and subtracting vector y by 2 x plus y by 2. It is going to be multiple of our vector v. So, if q of v is equal to v then q of alpha v also will be equal to alpha v. So, that is why q of x plus y by 2 is going to be x plus y by 2 x minus y by 2 is a multiple of u q of u is equal to minus u. So, q of x minus y by 2 is a multiple of u q of x minus y by 2 will be minus x minus y by 2. Now, x will get cancelled or the other x by 2 will get cancelled and you will get q x is equal to y. So, we have got a way to construct q. Once the vectors x and y are given to us, what you have to do is construct these unit vectors and then look at q is equal to i minus 2 times u u transpose. Such a orthogonal matrix will transform the vector x to vector y. Now, let us use this to construct order to rather what we are trying to do is writing a is equal to q into r. So, I am now going to look at the first column of a. It is a invertible matrix. So, the column will be non-zero. Now, this column I want to transform in a column vector whose first entry is non-zero and all the other entries they are 0. So, if I do this I do not have much choice for this vector y. The first column is given to us. So, that is given. Now, for the second vector it has only one non-zero entry and the Euclidean norm both of them they have to be the same. So, we have this is our first column a 1 1 a 2 1 a n 1. This is my vector x vector y will be of the form sigma 1 0 0 0. Since, x and y should have same Euclidean norm that is a important condition. This sigma 1 will be nothing but summation a i 1 square i going from 1 to n and its square root. For this sigma 1 the choice is either it should be positive square root or negative square root and that is it. u is going to be x minus y divided by its Euclidean norm. So, this is our x, this is our y then you look at u and then your q 1 is going to be identity minus 2 u u transpose. So, you have x minus y this is going to be a 1 1 minus sigma 1 a 2 1 a n 1 sigma 1 square is summation a i 1 square i goes from 1 to n because we want to have y and x to have the same Euclidean norm. When I calculate norm of x minus y to norm square it will be summation a i 1 square minus 2 sigma 1 a 1 1 plus sigma 1 square this is sigma 1 square. So, it becomes 2 sigma 1 into sigma 1 minus a 1 1 and that is minus 2 sigma 1 u 1. We can transform the first column into a column which has only 1 non-zero entry and that is the first entry. So, in my next lecture we will see how this when you continue this procedure how we get q r decomposition of our matrix and then we are going to consider least square problem. So, thank you.