 So, in the previous lecture we have seen this interesting result with respect to eigenvalues and eigenvectors where we have seen that if your matrix because now I am not going to talk about operators in general, we just going to talk about the matrix representations of those operators. Since we shall be dealing with finite dimensional vector spaces and subject to an ordered basis you can always represent things as like things like operators as matrices square matrices. But also any linear transformation can be represented as you know rectangular matrices in general right. So, what we have seen is that if you have a square matrix that represents a self adjoint operator and in the parlance of matrices we will say that the matrices over complex field there will be Hermitian that is there will be equal to the transpose of the complex conjugate. So, if you take the complex conjugate of each entry of the matrix and take the transpose of that matrix that will equal the original matrix. So, in such a case we have seen that the eigenvalues firstly turn out to be all real they can be repeated but they will all be real. Secondly does not matter whether they are repeated or not you will always be able to diagonalize such a matrix through a you know similarity transformation or a change of basis. Thirdly this basis also happens to be an orthogonal or you can make it orthonormal right. We saw a proof for that are there any questions about the proof I felt maybe we had hurried through the proof. So, I can quickly recapitulate if you have any doubts about the proof that we finished in the previous lecture. Any queries you have about the proof if you have gone through the proof that is yes. So, how are we proving it is an orthogonal basis. So, what we did was first we chose some arbitrary lambda then we expanded this if you remember the steps I am going to quickly go through run you through the steps again. So, what we did was we chose an orthogonal basis V 2 till what was it what was the size we had chosen V k. So, this was k plus 1 was it. So, then this maybe belongs to k plus 1 cross k plus 1 ok. So, that was a running variable the induction. So, we saw that for every Hermitian matrix of size k cross k as a base step of the induction we assume that it is orthogonally transformable through a set of series. So, at least any Hermitian matrix of size k cross k has an orthonormal basis as it is you know the eigenvectors they form an orthonormal basis that we have assumed. Now, we have to show that for the k plus 1 cross k plus 1 also it is true. So, what did we do we added this and then we saw that this led to what. So, the first fellow will become lambda V the rest of it is just a V 1 through till a V k like so all right. Now, we hit this one with V Hermitian which are basically row vectors now right. So, we hit this side also with the same thing which is V Hermitian V sorry V 1 Hermitian until V k Hermitian like so and what did we have as a result of this because this by my choice formed an orthonormal basis yeah. So, every one of these inner products with the first column at least yeah what does it turn out to be 0's right. So, the right hand side we saw then turn out to be just lambda here and all 0's here right and these I really do not care too much about it and this is sitting here is a k cross k matrix here right it has we still do not know we will infer about the structure of this one this is some k cross k right because this is our size k plus 1 cross k plus 1 this first the starred entries are essentially V Hermitian A V 1 V Hermitian A V 2 till V Hermitian A V k that is what these entries are I have written them as stars because at the moment I have not inferred anything about them. Now, once I have done this I notice that A happens to be A Hermitian now if you look at this this is V Hermitian this is A this is V what can you say about this matrix if you take V Hermitian A V the whole Hermitian the order gets reversed remember. So, this becomes V Hermitian A Hermitian V but because A Hermitian is equal to A you further write this as V Hermitian A V which means that this matrix is also itself Hermitian because when you take its Hermitian it returns the same thing which means that this right hand side fellow must also be Hermitian is also Hermitian right. Now, this side is also Hermitian what does it mean if this entry is 0 so must this entry be 0 if this entry is 0 so must this entry be 0 in other words now I can say that this must also be 0 furthermore if this entire fellow has to be Hermitian this K by K block sitting here must also be Hermitian otherwise this entire thing cannot be Hermitian. So, this entire thing is Hermitian for the base step of the induction what have I assumed any Hermitian matrix of size up to K can be orthogonally diagonalized right through an orthonormal set of vectors. So, this one now let me write this as let us say yeah V sigma let us not use sigma let us use lambda hat may be V tilde V tilde lambda hat V tilde Hermitian this is orthonormally diagonalizable. So, this is exactly this V tilde is exactly that vector which orthonormally diagonalizes this where this fellow is that diagonal representation of this fellow. So, this I can write as lambda 0 0 and V tilde this big lambda which is K cross K diagonal and V tilde Hermitian like so. So, let us start with this step now take V Hermitian A V is equal to lambda 0 0 V tilde lambda hat V tilde Hermitian you agree with this yeah. So, from this can I erase this now I suppose I can write. So, from this we may now deduce that V rather let me hit it with V on the left and V Hermitian on the right. So, what I have is A is equal to V I will do something I will just take 1 0 and V tilde right then I will write this and I ask you to convince yourself that this is in lead true this is going to be 1 0 0 V tilde Hermitian and V Hermitian right. Now what it means is that this fellow can be diagonalized by this fellow acting on the left and this fellow acting on the right. You may readily check that one of them is the Hermitian of the other complex conjugate transpose of the other the order obviously gets reversed. All that I now need to show is that the product of these two fellows is identity and I would have shown you that there is this orthogonal or orthonormal set of vectors that indeed diagonalizes A. So, what I have to show is that when you take the product of this and this it devolves to the identity. So, that is I would put it to you quite straight forward just do a multiplication like. So, this is V tilde and then you have 1 0 0 V tilde Hermitian V Hermitian what do you think is this going to result in well because this is of course an orthonormal vector. So, this product with this leads to identity of size k cross k. So, this in between you will have 1 and block diagonal identity of size k cross k which means this entire thing in the middle is just identity of size k plus 1 cross k plus 1 and then you will be left with V V Hermitian right. What about V V Hermitian by my very choice I had made this I started with an Eigen vector V and the rest of the vectors were chosen to extend it to an orthogonal basis it is an orthonormal basis for the k plus 1 dimensional n doubles right. So, this is also going to be identity which means that this indeed this fellow or this fellow they are what you call unitary matrices such matrices are called unitary matrices. So, this is what you call the unitary matrix and now check that if you hit with V Hermitian here and V here or you can hit it with V on the right this fellow gets pulverized. So, in fact this V and this this entire object let us give it a better name V hat. So, in fact this V hat is actually going to be a set of Eigen vectors yeah straight forward no this is this is what V hat Hermitian V hat Hermitian. So, hit A with V hat on the right what you will have is A V hat what happens with V hat Hermitian times V on the right here identity. So, you will have V hat, but what is that that is a classic definition of the Eigen vectors and Eigen value equation is it not it is just decoupled I mean if I just write it like that this is A let us say this is let me give it V 1 hat V 2 hat till V k plus 1 hat is equal to V 1 hat V 2 hat V k plus 1 hat times lambda 1 lambda 2 till lambda k plus 1 equate each column on either side and you have indeed the Eigen value Eigen vector equations. So, each of those V 1 hat V 2 hat till V k plus 1 hat are indeed Eigen vectors. So, that establishes that I mean whenever you say that it is diagonalizable using a unitary matrix it means the Eigen vectors are orthogonal right because that is how we diagonalize we diagonalize using the Eigen vectors. Now, if it is unitary diagonalizable it means that it is basically the Eigen vectors that are forming an orthogonal or you can normalize them and form an orthonormal basis in this case right. So, this is a great property we will readily see an application of this may be something that you have learnt in high school, but now you will see a very elegant application of this idea to such a problem you already know how to solve it I am guessing, but let us look at this nonetheless because this is a very special class of matrices now we are dealing with you see we started with this Eigen value Eigen vector problem for any arbitrary matrix n cross n we did not impose any structure on it, but now we have suddenly moved on to symmetric matrices why are we so interested in symmetric matrices after all or Hermitian matrices for complex case right because these have very practical applications that is what motivates us. So, we will try to invest good deal of time probably the entirety of this lecture on seeing some applications of this and let us see how far we can get with that. So, you remember when you learnt about conic sections maybe in your plus two levels right you had a parabola which you sketched like this yeah. So, y squared is equal to 4 a x kind of thing then you had the ellipse yeah x squared by a squared plus y squared by b squared is equal to 1 then you also had hyperbolas like this which is also obviously x squared by a squared minus y squared by b squared is equal to 1 and these are already you know straight forward prototypes of these curves, but things start to get complicated when you have a quadratic expression in x and y and you have cross terms. I mean you can probably guess that if you have something like a x squared plus b y squared plus c x plus d y plus e is equal to 0 this is not that difficult to deal with right y just complete squares. So, it is basically a translation of the origin this is not difficult to deal with you just complete squares here take a common so it is x squared plus c by a x plus something something squared and that tells you exactly how much shift you are giving about the origin, but the shape remains undistorted whether it is this whether it is this or whether it is this hyperbola it just causes a translation but the real ugly stuff starts to happen if instead you also have plus 2 h x y kind of term what is that generally correspond to from your experience or from whatever you remember from plus 2 level rotation and perhaps distortion also. So, you tend to get ellipses that no longer look like this nice form where the principal axis the major and the minor axis are just along the x and the y directions you can start to get ellipses such as this and heaven knows what else right, but wouldn't it be nice if by looking at this equation of course there is some formula probably if you have in your if you have seen in your coordinate geometry books you would see that they give you some mystical some formulas right check this h squared minus this this this this and if this is true then it is a hyperbola if this is true it is an ellipse and you know you have to memorize all these sort of things, but it would be better if we had a better understanding of what is going on here. So, here is the idea any time you are given an equation such as this which is obviously the equation of a general conic section shifted and rotated in some arbitrary manner and if you have to find out whether this corresponds to a parabola and ellipse or a hyperbola what do you do apart from remembering those nasty formulae right any time in your life forget about formulae that is one of the things we hate to do right. So, what do we do here is what we do focus on this part because this part as I said is just a translation not difficult to figure out. If I focus on this equation can I not write this in the following manner just notice yeah doesn't this part have a representation like so what is so interesting about this the representation involves a matrix which is symmetric oh b it should be b thank you yep it should be b. So, it is a symmetric matrix nonetheless right now if it is a symmetric matrix I will tell you that we are back in business with whatever we have learnt now how so what do we know about this symmetric matrix it is a special class of Hermitian matrices when the entries are just real Hermitian is for general complex matrices here the complex conjugation need not happen just transposition suffices. So, what can we say about this it is diagonalizable by a I mean unitary is for complex matrices you just say orthogonal matrix real orthogonal matrix or an orthonormal matrix you can say right. So, this part then turns out to be what exactly yeah if I call this the a matrix that I have raised it that I was dealing with up until this point what does that a lead to v Hermitian a v is equal to some diagonal matrix which means that a is equal to v diagonal matrix times v Hermitian in this case v transposed and v and v transposed are inverses of one another that is a beautiful part about it right because it is diagonalizable by an orthogonal matrix. So, it is v diagonal v inverse which happens to be the v transpose in such cases that diagonalize I mean the inversion process becomes very simple know if it is an orthogonal matrix is just the transposed. So, I am going to take up this term and I am going to write this in the following manner x y and v some diagonal matrix times v transposed times x y ok. And now I am going to say that let x y equal to v x bar y bar then what happens what does this part then become what can you say about this part now look at this v transposed x y is what now it is just x bar y bar is it not if x y is equal to v x bar y bar then we did with v transposed on both sides then what do you have x bar y bar is equal to v transposed x y which is nothing, but this fellow sitting on the right. So, this is equal to this diagonal times x bar y bar and pre multiplied by also x bar y bar check is it not the case it means that now you have transformed your basis you have chosen a new basis originally you had the x y. So, this is x this is y this is x this is y this is x this is y and now through this transformation you have gotten rid of this pesky cross coupling term here now because you know the standard forms of these equations if you look at the equation in terms of x bar and y bar it is just a change of basis the nature of the curve will not change due to that right. So, if you know what the curve looks like in terms of x bar y bar inherently you will answer the question about whether it is an ellipse or a parabola or a hyperbola you can go ahead and apply the same transformation to this part as well this part of course e remains constant, but this part as well you can apply and then you can see what sort of a shift it is in terms of x bar y bar. So, in the x bar y bar frame of reference it is basically being decoupled what was coupled in the x y coordinate system has now been decoupled in the x bar y bar coordinate frame. So, forget about all those formulae all those determinants and writing up all those things right. In fact, you can just go back to your plus two level maths books if you still have them and go back to the equations that you have and check that this is indeed what is happening you can take that as an interesting exercise that formula using some determinants probably you had right check that it is sign of the determinant you can check that it exactly corresponds to something that we have now seen here right. So, when you have this decoupled you basically mean that when you are saying that it is going to be an ellipse you need both the diagonals to be positive when you need a hyperbola you need them to be of opposite signs right and when you are talking about a parabola you need one of them to be 0 and at least one of these fellows to exist the other fellow. In fact, the one that is 0 the other fellow must exist yeah at least right then you can represent it as a standard parabola right. So, the point is that just through an application of this elegant idea that any symmetric matrix can be orthogonally diagonalized you can just check for the kind of conic section that a quadratic form like this represents and if that does not sound very impressive because you all know the formulae for all of this let us just jack it up a bit just crank up the level of difficulty a bit and hopefully this is something you have not had too much experience with which is when we talk about quadric surfaces in 3D yeah we talk about ellipsoids and paraboloids and hyperboloids and stuff might have had some experience, but I am pretty sure you have not dealt too much with them mostly we deal with 2D Cartesian geometry in our day to day lives. So, let us just give you a quick glimpse of what lies in store see the idea is still the same from a 2 cross T to symmetric matrix it will change to a 3 cross 3 symmetric matrix when this yes you have a question right. So, you can check what is the relation between X bar Y bar and X Y through this relation this V is just an orthogonal matrix that is a very interesting question thank you basically what is the relation between X Y and V X bar Y bar if you look at the norm of this and the norm of this because this is a unitary or a Hermitian or a you know not Hermitian sorry it is a unitary or an orthogonal matrix it is a norm preserving matrix it is a norm preserving transformation you can just check I mean X Y times X Y on the one hand this is X squared plus Y squared and you can check over there V X bar Y bar and here you have X bar Y bar V transposed. So, this V transpose V from the middle it vanishes yeah. So, this is also going to be X bar squared plus Y bar squared. So, such transformations just call a rotation just the right sort of rotation that gives you another orthogonal basis on the basis on the in terms of which you can actually represent these curves in the standard form. So, it just rotates them sufficiently the X and the Y direction it causes just a sufficient amount of rotation they are orthogonal yeah they will be orthogonal yeah of the based on this curve. So, now you are representing the curve in terms of X bar and Y bar. So, any linear transformation can be captured by what it does to a basis. So, you take this let it act on X 0 you take it let it act on 0 Y and see what sort of a rotation it causes to X and Y you will see that at the end of it the rotation that it caused to X and to Y to lead to X bar and Y bar that actually leads to a final configuration of X bar and Y bar. So, that they lead to an orthogonal basis right that is the beauty of this transformation right. How is it? This is V transpose V this is just identity. So, if it is identity you whether you write it or not does not matter. So, it is non-preserving that is true of any size anything if you have an orthogonal matrix or an Hermitian matrix it preserves the norm. All it does is rotate cause a change in the direction of vectors. When orthogonal matrices act on vectors they just cause a pure rotation to every vector. Every vector is not rotated by an equal angle, but if you take a enough number of fellows which is to say the number of fellows that constitute a basis and if you see the actions then you know what it does to any other vector, but it will never cause a change in the norm because of its nature it is non-preserving right. So, now let us crank up the level of difficulty a bit and see that when you are faced with curves that contain much more than just X and Y, but also Z. So, maybe you have F Z plus let me just rewrite this equation again I mean I do not want to delegate X to be I mean Z to be unimportant. So, it is A X squared plus B Y squared plus C Z squared plus 2 H X Y plus 2 G Y Z plus 2 F X Z plus whatever it is. I am running out of variables now. E X plus K Y plus L Z plus M is equal to 0. So, many terms right, but we still need to focus on only the part that is quadratic which is just this is it not right. So, what is the representation? Again needless to say it is going to be a symmetric matrix A B and C H H what is it Y Z. So, that would be G and X Z is F F. So, this is at the heart of it this is what the matrix is and now I am going to just you know I am not going to just derive this and all I am just going to give you a few sketches of the standard quadric surfaces. So, of course a standard ellipsoid has an equation like so that is an ellipsoid. What does it look like? Well of course I cannot do a very good job of drawing it, but you have to imagine that this is what an ellipsoid looks like. Yeah. So, this is a standard ellipsoid. If you have a hyperboloid there can be actually two kinds of hyperboloids. So, one kind of hyperboloid would be X squared by alpha squared plus Y squared by beta squared minus Z squared by gamma squared is equal to 1. So, this kind of hyperboloid looks something like this. Let me take the Z here the X here yeah oh wait maybe I should X Y Z yeah X Y Z. So, this hyperboloid you will have to imagine this by the way I cannot draw a 3D object faithfully on a blackboard on a two-dimensional board yeah that is one kind of hyperboloid. There can be another type of hyperboloid which has no projection on the X Y axis what we call as a trace of a curve on an X Y axis. So, the second kind of hyperboloid would have a representation like so Z squared by gamma squared minus X squared by alpha squared minus Y squared by beta squared is equal to 1 which would look essentially like two bowls. So, this is Y this is X this is Z. So, it looks something like a bowl here that opens up here and like another bowl that opens up here. So, it never touches the X Y plane and you can see why it cannot if you put Z is equal to 0 there is no X Y that satisfies this equation. So, this fellow has no trace on the X Y plane right. So, that is a second kind of hyperboloid. So, this is a hyperboloid this is an ellipsoid this is a hyperboloid of kind 1 this is a hyperboloid of kind 2 of course, you can just stretch your imagination and think that this can be drawn along various principle axis just several permutations maybe I am biased against Z. So, I am doing it like this yeah essentially it makes no difference otherwise. That is three types of quadric surfaces you could you could also you could also end up with a so called elliptic cone which has an equation that looks like this. So, this elliptic cone this is again the Z axis where is it X axis the Y axis. So, this elliptic cone looks something like this where if you take cross sections of this it is an ellipse and you can see that if you put Z is equal to 0 then only solution is the origin. So, it has no trace other than the origin on the X Y plane yeah. So, that is a standard. So, this is elliptic cone you can have an elliptic paraboloid which is like Z is equal to X squared by alpha squared plus Y squared by beta squared. So, this is an elliptic paraboloid okay. So, an elliptic paraboloid obviously this is Y this is X this is Z this is X. So, obviously this would have no existence for negative Z. So, the only thing that can happen is yeah it touches the origin and you can just I leave it to you as an exercise to check through your imagination. So, if you slice it like this it looks like parabolas, but if you slice it like this parallel to the X Y plane it will result in ellipses right. So, this is an elliptic paraboloid. So, we have one kind of 1 2 3 4 5 and finally maybe I will just get rid of this now since we have the expression for this quadratic part at least and finally we will have a hyperbolic paraboloid which is probably the most complicated of all of this. So, we have if I am not mistaken Z squared is equal to Y squared by beta squared minus X squared by alpha squared I would rather not draw this except because I will make a mess of it it is too complicated it looks like that saddle that we talked about by the way. So, you can just check what happens if you equate Y is equal to 0 which is when you are talking about the ZX plane it will be a hyperbola same with if you put X is equal to 0 yeah, but on the other hand if you put any if you put Z is equal to 0 then this is basically pair of straight lines, but if you put Z at some positive number then what happens no Y is equal to 0 will be probably a pair of oh wait is this Z or Z squared I seem to have forgotten that, but anyway this is going to be an hyperbolic paraboloid sorry yeah it should be a pair of straight lines it seems yeah yeah if you put Z is equal to 0 it will be just a pair of straight lines no in the elliptic cone also these are straight lines. So, if you put any one of these as 0 if you if you slice them along X is equal to 0 or Y is equal to 0. So, if you take along the ZX plane or the ZY plane. So, visualize it here ZX plane and the ZY plane you will just end up with a pair of straight lines yeah it probably should be Z I think yeah I think it should be Z yeah yeah it would not have symmetry this one does not have symmetry. So, it should be Z yeah because this one now if you put Z is equal to 0 this just becomes a pair of straight lines right if you put Z as some positive number then you can just divide it out by the positive number and you get one kind of hyperbola if you put Z as some negative number you can divide by this and you get the different kind of a hyperbola. So, that is why the shape is so warped I would rather not draw this yeah no donuts would have some punctured in between those are like you are talking about toroids yeah those will have some punctured in between. So, donuts are similar to your coffee cup by the way weird as it may sound, but yeah. So, this is the hyperbolic parable now you can imagine that you have all these varieties right different types of curves and now if I give you that general equation like this I mean and with this cross terms it is a difficult task I put it to you to determine as to what kind of a quadric surface you are you are representing through that equation, but again if you apply the same idea and look for the particular orthogonal matrix that diagonalizes this then you have transformed it to some x bar y bar z bar new set of coordinates in terms of which there is no cross coupling and just by looking at the coefficients of these fellows and comparing them with the standard forms that I have outlined there you can tell what kind of a quadric surface you are dealing with right. So, its utility stretches beyond just the 2D that you are already familiar with in conic dealing with conic sections, but also in in three dimensions when you are dealing with these kind of quadric surfaces you can just ascertain. So, that is a I think a useful application of the idea of this knowledge about this spectrum of a symmetric matrix that brings with it right ok. We will show another result and probably that will take up much of our lecture today. I will see how far we can get with the proof, but the main idea is not just the proof, but to give you an application of that idea and that is our next point which is singular value decomposition ok.