 In the last lecture we have been reviewing several of the properties of matrices, special matrices, operations and matrices. We are going to continue the coverage of other properties of matrices that are critical to our analysis. The first of the topic in that direction are going to be the notion of eigenvalues and eigenvectors of any real matrix. Let A be a n by n real matrix. If there exists a vector V belonging to Rn at a constant lambda, a real or a complex constant R such that A v is equal to lambda v then lambda is called eigenvalue and V is called eigenvector of A. From the definition it follows that lambda v the pair the constant lambda and the vector v is a solution of the homogeneous system that can be obtained from A v is equal to lambda v. For V to be a non if V is 0 this equation is trivially satisfied V is equal to 0 is called a trivial solution. We are seeking non-trivial vector that mean a non-zero vector. For a non-zero vector to solve this equation it is necessary that the determinant of the matrix A minus lambda I must be 0. We have earlier seen one of the conditions necessary for the existence of solution of homogeneous system is the system must be singular. Here the system matrix is A minus lambda I. The determinant of A minus lambda I must be 0. The determinant of A minus lambda I elements of A are known elements of I are known lambda is a variable. So it becomes a polynomial of degree n. This polynomial P lambda which is the determinant of A minus lambda I is called the characteristic polynomial. An nth degree polynomial has n roots. Let lambda 1 lambda 2 lambda n be the n roots of P lambda is equal to 0. From fundamental theorem of arithmetic we all know that lambdas can be either real or complex. Complex roots always come in conjugate pairs. The reason complex root come in conjugate pairs is that the elements of the matrix A are real. This implies the coefficients of the polynomial P lambda are real and when you are trying to solve a polynomial with real coefficients the roots if it is complex it has to be complex conjugate. That is for any general matrix. For a special class of matrices when A is symmetric lambdas are real. When A is symmetric and positive definite positive symmetric matrices are called SPD, S for symmetric PD for positive definiteness lambdas are real and positive. This means that for a general matrix the Eigen values lie in a complex plane. This is the real axis. This is the imaginary axis. So for a general matrix the Eigen value can be anywhere. If it is complex it might occur in conjugate pairs. It could be real. It could be positive. It could be here. So that is a general distribution of Eigen value for any general matrix. For symmetric matrices the Eigen values are always real. The Eigen values are real. This is for symmetric matrix. For a positive definite matrix the Eigen values are always real and positive. So you can see the restriction how it constrains the distribution of Eigen values. It could be anywhere in the two dimensional complex plane for a general matrix. It is along the real line for symmetric matrices. It is in the positive half of the real line for symmetric positive definite matrices. We will have lot more occasions to talk about symmetric positive definite matrices. So this Eigen structure of symmetric positive definite matrices is an important property that we need to keep in mind. We are going to illustrate the computations of Eigen values and Eigen vectors that A, B is symmetric matrix. By the previous claim the Eigen values must be real. Yes 9 and 4 they are real. By solving the equation A v is equal to lambda v. A v1 lambda 1 v1 A v2 equal to lambda 2 v2. These are two equations corresponding to two distinct Eigen values. If you solve these linear equations it can be found that v1 is one Eigen vector v2 is another Eigen vector. The Eigen vector we are interested only in the direction of the Eigen vectors. So v1 is a normalized Eigen vector v2 is a normalized Eigen vector. It can be shown v1 is this is not right it is a perpendicular sign v1 and v2 are orthogonal v1 v and v2 v1 orthogonal to v2 orthogonal to v2. So I would very much encourage the reader to be able to verify these computations. Now I am going to generalize this. Let A be a symmetric positive definite matrix. That lambda IVI be such that AVI is equal to lambda IVI for each I running from 1 to n there are n such equations. So we have a collection of vectors Eigen vectors without loss of generality as we mentioned Eigen vectors are going to be normalized. So v1 v2 vn is a collection of mutually orthogonal and normalized Eigen vectors. So it constitutes an orthonormal system. We have already seen the notion of orthonormality in the last class. Now I am going to construct a matrix v which consists of n columns. The first column is the first Eigen vector second column is second Eigen vector the nth column is the nth Eigen vector. This is a matrix there is a correction here. This is the matrix is n by n. This matrix is orthogonal so its transpose is equal to inverse. So from the basic definition A v is equal to v lambda. This essentially tells you simultaneously all the equations that are summarized one for each I. So this equation A v is equal to v lambda where lambda is a diagonal matrix. So you can readily see A is the given matrix v is the matrix of n Eigen vectors lambda is a diagonal matrix of n corresponding n Eigen values. Look at the order lambda 1 lambda 2 lambda n v1 v2 vn they are correspondence with each other. Since v transpose is equal to v inverse I can multiply on the right side by v transpose. So A v equal to v lambda. So we can multiply A v v transpose is equal to v lambda v transpose but v v transpose is equal to A is equal to I v v transpose is equal to I identity matrix. So A is equal to v lambda v transpose. This is called the Eigen decomposition of A. This Eigen decomposition of A can be expressed in element form. So this is simply the sum of the product outer products of VI and VI transpose. So VI VI transpose is a matrix each of these matrices are weighted by lambda I. So A can be expressed as the weighted sum of rank one matrices each rank one matrix corresponds to an Eigen vector. Now we come to another important concept associated with this called spectral radius denoted by rho of A. Spectral radius is equal to the maximum of the absolute value of the lambda s. So if A is a symmetric matrix lambda s are real. If A is a symmetric and positive of lambda s are real and positive. So the spectral radius of a symmetric matrix is given by the maximum of the absolute value of Eigen values. Now we are going to introduce another related concept called singular values of A. Let A be a non-singular matrix. The Gramian A transpose A and A transpose are then symmetric positive definite. In fact there is a result here I would like you to think about the Gramian must be capital J because it is the name of the person capital J. So A is non-singular A transpose A and A transpose are symmetric matrix. If A is non-singular it is said to be full rank. If A is non-singular and full rank then A A transpose A transpose A are both symmetric and positive definite. This is a very fundamental result with respect to the symmetric positive definite matrices and its relation to Gramian. So if A is non-singular A transpose A is symmetric. Therefore I can do a symmetric Eigen value analysis. A transpose A V i is equal to lambda i V i. This is the same as we have done for A. Now what we did for A I am redoing for A transpose A. Here lambda 1 lambda 2 lambda n are the Eigen values because A transpose A is positive even the least Eigen value is positive. We are going to order the Eigen values the largest is called lambda 1 the next largest is called lambda 2 the least largest is called lambda n in the least largest is also positive that means everybody else is positive. Now I would like to relate the Eigen values Eigen vectors of A. So given a matrix A there are two Gramians A transpose A A A transpose both are symmetric and positive definite. I am now going to argue if you know the Eigen values and Eigen vectors of one of the Gramians we also can infer the Eigen value and Eigen vectors of the other Gramian. To that end I am giving it a homework problem to verify it is very simple A transpose A times U i is equal to lambda i U i where U i is defined by 1 over square root of lambda i A V i. So if you for if I know A I know A transpose A if I know A transpose A I know lambda i V i. So I know A I know V i I know lambda i so using A V and lambda you define a new vector U i. So new vector U i is simply a linear transformation of the vector V i scaled by 1 over square root of the Eigen value. So this if I define U i this way it can be verified that A transpose A A transpose U i is equal to lambda i U i. So this essentially tells you lambda i is simultaneously Eigen value of A A transpose as well as A transpose A they both share the same Eigen value. The Eigen vectors V i and U i are related by this. So here is a summary A transpose A and A A transpose share the same Eigen values and then Eigen vectors are also related you can essentially see U i is related to V i. Now if I define sigma i to be square root of lambda i now please remember lambda i's or the Eigen values are symmetric positive root matrix they are all positive. So square root of that exists and square root of that is real. So I am now going to define the positive square root of lambda i equal to sigma i and the sigma so for each lambda i there is a sigma i there are n such sigma i's sigma i's by definition they are called singular values of A. So the Eigen values of A transpose A are called the square root of the Eigen values of A transpose A are called the singular values of A. So singular value decomposition singular values Eigen values Eigen decomposition these are all the related concept that we are seeing in this part of the talk. Now we move on to another interesting concept relating to matrices just like vectors have a size just like the size of a vector is captured by the notion of a norm of a vector matrix is also an object every object can be endowed with the definition of its size the size of a vector is measured by the norm of a vector the size of a matrix is also going to be defined by a norm of a matrix. So I am now going to define the notion of what is called norm of a matrix A it is a measure of the size of A just like in the vector case we had various norms 2 norm 1 norm infinity norm Inkovsky's norm the energy norm in the case of matrices also we have quite a variety of norms to talk about I am not going to talk about all the possible norms I am going to talk about some of the simple norms which are often used in analysis. The first of the norms is called the Frobenius norm Frobenius norm of A is simply an extension of the Euclidean norm for the matrix A the Frobenius norm is denoted by this symbol the norm sign with the subscript F and what is it you take the sum of the squares of all the elements of the matrix take the square root of A this is exactly the way we had defined the Euclidean norm the Euclidean norm of a vector is equal to the square root of the sum of the squares here it is a square root of the sum of the squares of all the n square elements of the matrix and that is one measure of the size of the norm that is another norm called induced norm these induced norm are defined using the notion of an operator. So let A be a matrix that corresponds to a linear operator or a linear transformation the pth norm of A defined by the norm symbol A with a subscript P that is defined by the supremum taken over all x that is not 0 of the ratio A x P norm divided by x P norm. So you can essentially see the following given A given A pick any arbitrary vector x A x is a vector computes its P norm x has also its P norm compute this ratio this ratio varies A is fixed x varies you vary x x belongs to R of n there are infinitely many x's as you vary x this ratio varies as this ratio varies I am interested in the maximum value. So supremum is a you can think of supremum is a very technical term I do not want to get into the technicality for practical purposes you can assume it is a maximum value of the ratio of the P norm of A x to the P norm of x. So what does this tell you this tells you the following a two dimensional analogy is like this here is a vector x here is a vector A of x the A of x has so the numerator tells you the pth norm of A of x the denominator tells you the pth norm of x if this ratio is larger than 1 A x is larger than x that means there is a magnification if this A x if the numerator is less than the denominator then there is a shrink. So a linear operator can either elongate a vector or a shrink a vector the maximum of this ratio the magnification factor is called the pth norm of the operator A or a linear transformation A. Equalently we can also compute the pth norm of the facts where x is constrained by this relation in other words you can consider all those back vectors that whose pth norm is 1. So you reduce the range of values of n which is equivalent to this definition. So this is how you define the pth norm of A matrix by setting p is equal to by setting it should be lower case p by setting p is equal to 1 to infinity we get various matrix norm you can get one norm two norm infinity norm and so on. Given the matrix now that we have a matrix norm we have a vector norm there are standard inequalities which are of great interest in proving several results in analysis. So the norm of a transformed vector so x is a vector A x is another vector A x is a transformation of x by A. So what does the left hand side say in the inequality 1 the norm of the transformed vector is less than or equal to the product of the norm of the operator A and the norm of the vector x likewise the norm of the product of two matrices A and B is less than the product of the norm of the operator A and the operator B. These are two fundamental inequalities now please realize in this inequality I did not specify the nature of the norm these inequalities true for any and every type of norm you can pick a two norm one norm infinity norm or any other norm for all of these norms these inequalities whole good these are fundamental inequalities and these inequalities are very similar to several inequalities we have seen for the vectors. Now we have defined the norm but the whole question is how do I compute these p norms how do you compute in other words how do I compute these various norms for matrices here is an example of the computation if A is a matrix the one norm of A it can be proven that it is equal to the maximum over J of summation I is equal to one turn A ij so let us talk about this now I have a matrix A I have a matrix A A has different columns so let us consider the jth column of A the elements of the jth column are going to be A1j A2j and Anj so what is that we are now going to be looking for we are going to be looking for the absolute value of each of these and I am going to take this sum of the absolute value this must be absolute value sum of the absolute value of the elements and take the maximum over J so one is called the column norm another is called the row norm so the maximum is taken over J for the column norm because J is the column index I is the row index so now look at this now so for one you sum along the row for another one you sum along the column so one is called the first one the one norm is the column norm the infinity norm is called the row norm it can be shown that one norm can be easily computed by this infinity norm can be easily computed by this these are computational algorithms for quantifying the values of these norms the two norm of a matrix is very is can be simply stated as sigma 1 where sigma 1 square is the is the maximum eigen value this must be the maximum eigen value the maximum eigen value of A transpose A sigma 1 is also called the largest singular value we simply introduce the notion of a singular the singular value in the in the in the last couple of slides so given A you compute A transpose A A transpose A symmetric and positive definite if A is is is not singular it is symmetric and positive definite so all the eigen values are are are are real and positive the square root of these eigen values are called the singular values the maximum of those singular values is called the two norm of is not is called the two norm of A when A symmetric A transpose is A so A transpose A is A square A square x is equal to lambda square x if A is equal to lambda x why is that A square x is equal to A times A of x this is A times lambda of x this is equal to lambda times A of x that is equal to lambda times lambda of x that is equal to lambda square of x. Therefore if lambda is an eigen value of A lambda square is an eigen value of A square if lambda is an eigen value of A lambda to the power k is the eigen value of A to the power k. So that is how the eigenvalue square itself when you square the matrices. Therefore by combining 3 and 4 we can see the 2 norm of A is simply the maximum of the eigenvalue. I do not have to even put the absolute value sign because A transpose A is positive symmetric and positive definite so sigma 1 is always positive but for safety sake one can introduce without loss of generality and lambda x is the maximum eigenvalue. And we can also recall that the maximum eigenvalue is called the spectral radius. Therefore we can conclude the 2 norm of a symmetric positive definite matrix of a symmetric positive definite matrix is given by the spectral radius. So what is it what a spectral radius means if you consider a circle with center origin and diameter as I am sorry the radius as rho of A all the eigenvalues of the matrix A lie within that circle. So that is the notion of the spectral radius of this matrix A. So we talked about matrices there are norms we studied various properties of norms these are the computational procedures for computing the values of different norms of interest in analysis. Just like we have talked about equivalence between the 1 norm, 2 norm, infinity norm for vectors here also I have a set of inequalities that relate to the behavior of various norms. So you can show given a matrix A the 2 norm is less than or equal to the product of the square root of the product of 1 norm and infinity norm. The infinity norm and the 2 norm I can sandwich the 2 norm using the infinity norm I can sandwich the 2 norm by 1 norm I can sandwich the Forbenius norm by 2 norm. We also know that another result which is fundamental important the spectral radius is less than I am sorry the spectral radius I want to highlight this. The spectral radius of a matrix is less than any matrix norm equality happens when the matrix is symmetric. So these are some of the interrelations between the 2 norm the 1 norm the infinity norm the Forbenius norm of matrices in the case of matrices the Eigen values play a definitive role in the definition of norms especially for the 2 norm and this is a very nice summary of the various properties of norms of matrices. Why do we do norms? For 2 reasons one is to be able to measure the size of the norm secondly the notion of a 2 norm is very useful in trying to quantify certain properties of matrices. We say a matrix is singular we say a matrix is non-singular we say a matrix is well conditioned we say a matrix is ill conditioned. One of the conditions for the solution of ax is equal to b we all know if I want to be able to solve ax is equal to b we would like to be able to make sure a is non-singular. We also know when so we have singular and non-singular yes or no day and night but in practice some matrices may be very close to being singular without being singular. So such matrices are said to be ill conditioned so I need to be able to characterize the degree of non-singularity how do you measure the degree of non-singularity one way to be able to measure the degree of non-singularity is through the notion of what is called a condition number of a matrix. So let a be n by n matrix this is the definition the condition number of a matrix is denoted by the symbol kappa of a the condition number is dependent on the definition of a norm. So this is the condition number using P norm of a matrix the condition number of a P norm of a matrix is simply the product of the P norm of a times the P norm of a inverse. But you can see the definition of the condition number is norm dependent. So I can have norm 1 conditioning, norm 2 conditioning, norm infinity conditioning so on and so forth. This in general if you cannot solve the equation Ax is equal to b we throw the word oh the matrix is ill conditioned. So if something is ill conditioned then there must be a concept of well conditioned. If something is singular non-singular very nearly singular these are all fuzzy characterizations of properties of matrices. We would like to be able to quantify this fuzziness using certain measure of the properties of these matrices that is where the condition number comes into play. How the condition number is related to the well conditioning ill condition of the matrices that is what we are going to be talking about presently. Recall the standard identity i is equal to a a inverse the P norm of i is 1 for every P 1 to infinity. So but we know so if i is equal to a a inverse if I took the norm of i the norm of i must be less than or equal to the norm of a times norm of a inverse this is inequality that we saw in couple of three slides but the norm of identity matrix is 1. Therefore I get this inequality 1 is less than the product of the P norm of a P and a P inverse by this definition the product of a P a P inverse is the condition number. So you can readily see the condition number of a is always greater than equal to 1 is always greater than equal to 1. So condition number is greater than equal to 1 condition number is a positive number it can be very large. So the range of the values of the condition number is 1 P infinity. So in this scale when the condition number is closer to 1 we say it is well conditioned when the condition number is very large is ill conditioned. Again how large is large we will talk about that in a minute how large is large depends on the computer machine position in a 32 bit arithmetic that is only a largest value you can measure. Therefore if a condition number kappa of a matrix let us say is 10 to the power of 20 or 10 to the power of 50 a matrix was 10 to the power of 50 is said to be more ill conditioned than a matrix was 10 to the power of 20 which is more ill conditioned than a matrix with 10 to the power of 3. So this ranking of the condition number helps to in some sense quantify the degree of ill conditioned associated with the matrix. So now let us you I use the P norm I am now going to specialize the discussion of the norm for a spectral condition number. So let a be a symmetric matrix spectral condition number is related to the maximum Eigen value. We also know the following if lambda is an Eigen value of a lambda inverse is an Eigen value of a inverse therefore the 2 norm of a is the maximum Eigen value of a the 2 norm of a inverse is the minimum Eigen value of a. So condition number for symmetric matrices is simply given by the ratio of the maximum Eigen value to the minimum Eigen values. Therefore the spectral condition number so for a symmetric matrix with this for a general non symmetric but non singular matrices the condition number is simply given by sigma 1 by sigma 2 where sigma 1 is the largest I am sorry this must be sigma n sorry this must be sigma n this is the ratio of the largest to the smallest singular values where sigma i is the ith the singular value and sigma the singular values are counted like sigma 1 is greater than equal to sigma 2 greater than equal to sigma n and sigma n is positive. Therefore this slide provides you a new concept of associating a number with a matrix called the condition number the value of the condition number is very indicative of the difficulties that one will have in computational process. Before going further I also want to be able to relate the various properties of condition numbers again matrices are related the Eigen values of the matrices the singular values are related the norms are related the so there is a relation between all the condition numbers themselves because condition numbers are defined in terms of norms if norms are related if condition numbers are related to norms condition numbers also must hold certain relations among themselves. So the 2 condition number 1 condition number infinity condition and 2 condition number infinity condition number and 1 condition number you can say they are all interpause so what does this mean if a matrix is well conditioned in 1am it is well conditioned in every norm if a matrix is ill conditioned one amp it is ill conditioned in every norm. So what does this tell you you can pick any norm that suits you computationally and do the analysis without having to worry about the choice of the norms so that gives you that provides you a lot of freedom. But among all the norms the one norm and the infinity norm are easily computed one is the column norm another is the row norm therefore from a computational perspective one may want to be able to use one norm or infinity norm. But in mathematical analysis theoretical analysis they generally often use the two condition number two norm because two condition number two norm is intimately associated with the Eigen structure spectral radius and so on that is a very appealing property. In the first course in linear algebra we are generally told the ill condition of the matrix is decided by the value of the determinant. But I am going to give you a counter example to show it is not the case in other words what we are told in a first course in linear algebra if I have difficulty in solving Ax is equal to b if I have difficulty in solving Ax is equal to b if the determinant of a is very large or very small then they will simply tell that you will have numerical difficulty yes you may have numerical difficulty but the ill conditioning or the well conditioning of a matrix is not determined by the determinant of a matrix as might often be given to understand in the first course. So here there are a couple of very good examples let a be a diagonal matrix of all halves the determinant of a is 1 over 2 to the power of n you can readily see the determinant of a goes to 0 as n goes to infinity but the condition number of a is 1 for all p. So the determinant and condition number they do not have much of a relation as another example let b be a n by n matrix consider an upper triangular matrix given by this you can readily see the determinant of b is 1 but the condition number of a infinite condition number is n and that goes to infinity as n goes to infinity. So what does it mean I can have matrices where the determinant goes to 0 but the condition number remains constant I can have matrices where the determinant remains constant but the condition number can go to infinity. So this essentially tells you there is no intrinsic correlation between determinant and condition number even though we simply say a matrix must be non-singular that means the determinant should not vanish for being able to solve Ax is equal to b the appropriate way to describe the properties of solution one obtains from solving a linear equation one has to relate it to the condition number of a kappa. So kappa is much more important than the determinant why kappa is more important now I am going to give you another result that will force the importance of kappa the condition number within the context of solving linear systems. Let us suppose I want to solve Ax is equal to b now let us think of the possible way suppose you want to enter the number 1 over 3 and you press the key 1 over 3. So 1 over 3 is supposed to be stored in your machine but 1 over 3 can never be stored correctly is 0.3333 what is the problem 1 over 3 does not have a terminating decimal expansion only numbers that have terminating decimal expansion will be able to one one can hope to be able to represent them correctly. So 1 over 3 1 over 7 these numbers once you store them you start with there is an error only rational numbers have terminating fraction a general real numbers may not have terminating fraction when you do arithmetic you cannot confine yourself simply to rational arithmetic we are supposed to have real arithmetic. So when you try to store a real number in a finite precision machine there is always error in representation that means you start with your left foot. So when you think you are solving Ax is equal to b you are not actually solving Ax is equal to b you are solving a plus epsilon by equal to b plus epsilon f. So what does it mean epsilon b is the error in the matrix a epsilon f is the error in b there are two kinds of errors a may be obtained from experiment that could be an inherent error in the experimental measurements a the numbers you store them storage error. So epsilon b in this case I am simply I am not worrying about other errors errors that arise out of finite precision arithmetic. So epsilon b b is a matrix epsilon is a small number so if I am thinking I am storing a you are not storing a you actually are storing a plus epsilon b you do not know what epsilon b but you know that there is an error epsilon f is likewise an error. So y is the system you are solved y is the solution system you are solving and you are pretending y is Ax this is the game we all play that nature of the business. So epsilon b and epsilon f are the perturbations of the matrix and the vectors respectively but we are epsilon is greater than 0 but small. So if y is not equal to x there is an error I am now going to consider the relative error in y. So y is a vector x is the true solution y minus x is the error vector in the solution I am going to take the norm of the error divided by the norm of the true solution. So what is that called the transfer it is called the real I am sorry relative error in the computed solution I am not going to show the derivation the derivation I generally do it in my class but it will take us too much into the outside of these scope of these lectures. It can be shown that this relative error is bounded about by the product of condition number times epsilon divided by times b by a plus epsilon times f by b. Now let us talk about b what is b? b is the error matrix that corrupts a a is the real matrix. So this is the relative error in a this is the relative error in b. Epsilon are the multiplying factors the same epsilon in here. So the right hand side is the constant multiple of the epsilon times the sum of the relative errors in the matrix and the right hand side. Now the computer precision decides what b is the computer precision decides what f is epsilon is decided by the smallest value the computer can store. So all these factors are decided by the computer architecture who depends so what else so your relative error is bounded by can be magnified by the product kappa a times the sum of the relative errors. So if kappa is large your solution could be much more erroneous your kappa is small your solution could be much more precise. Therefore this is the reason why we call kappa the condition number it is a conditioning the matrix that relates to the quality of the solution obtained by any method that you use to solve x is equal to b. Now what is any method so what are the methods we know how to solve x is equal to b we know ton of methods no matter what are the method you use this you are bound by this inequality. So if kappa is so if kappa is 10 to the power of 20 means what your relative error can blow up to 10 to the power of 20 if the relative error can go up by 20 to the power of 20 what does it mean you have spent the money but the result is not worth the paper written in it. So that is the importance of the notion of condition number why is this important people in meteorology will say I am using a 3D war I am using a 40 war I am using this I am using that yes all those algorithms are very well understood very well known but you need to be cognizant to the fact that the solution that these algorithms output the quality of it is decided by the nature and properties of the matrix that go into the computational process. So since kappa is greater than 1 errors in a and b are amplified so this is the keyword amplified the larger kappa more sensitive the system to the round off errors round off errors comes because of finite precision arithmetic. So here is a beautiful idea now I have a problem to solve I have an algorithm to solve the problem I have a computer architecture on which the algorithm is implemented here we talk about the effect of computer architecture the finite precision arithmetic could have on the quality of the solution that you are going to so it is a beautiful combination of algorithms and architecture how they are melded together to give a solution whose quality can be quantified like this with that we end the coverage of review of matrices I am going to suggest several exercises and I they are given in these problems there are about 12 problems in here and I would strongly encourage students to use a pencil paperwork do not go do not write a program you should know it first to be able to do with hand before you do with computers. So all these problems are very simple and fundamental to understanding many of the concept we covered if you want proves of many of the things that we have done in this lecture you can refer to the three times standard textbooks these are my favorite one Golobin Van Laan Mayor hon and Johnson so with this we conclude our coverage of overview of many results from matrices you can see we have reviewed a ton of results you may wonder do we need all of them you will soon see we will use almost all of them in our analysis of algorithms