 So the next thing to talk about is induced norms. Basically, the idea is that we can associate matrix norms with vector norms through this notion of induced norms. So I will define it here. So let be a vector norm on c to the n. So I am taking complex, like I said a lot of the results here are applicable to complex as well. So I may go back and forth between these two and where it is really required to make a distinction, I will tell you the difference. Define c to the n cross n as, okay, so you take that norm and if I compute that norm on the vector ax and take its maximum over all vectors which have a unit norm according to this vector norm, I will define that to be this with three bars, this function with three bars on the matrix A, okay. And so this is defined as the induced norm. So this is the, so I will just put it in double quotes because I haven't really formally stated this yet. So this is the matrix norm induced by the vector norm. So this is the basic definition of an induced norm, okay. So one thing you can immediately notice that if I scale x by some constant c then the norm ax scales by the constant c and the norm of x also scales by the same constant c. Another thing is that I am trying to maximize this norm ax over all x such that norm x equals 1. I can as well maximize this over all x such that norm x is less than or equal to 1 because if there is some vector that solves, so let me write that here so that I can explain more clearly. Note it is equal to max over all x such that x less than or equal to 1 of norm ax. That's because suppose you solve this problem and it gives you a solution for which norm of x is strictly less than 1. Then what I can do is I can scale that x by a factor which is greater than 1 then the norm will be may become equal to 1. But then when I scale that x by some factor greater than 1 then this norm of ax will also scale by a factor greater than 1 and that will only increase its value. So basically what that means is that the solution to this optimization problem will always occur at an x such that norm of x equals 1. So there is no loss or no harm if I include a whole bunch of other points which are inside the unit circle according to this norm because those points will never be the solution to this problem. And this in turn can be written as the max over all x not equal to 0 of norm ax over norm x. Okay this quantity itself doesn't change if you scale x so I can as well maximize norm ax over norm x over all x not equal to 0. And finally I can also instead of maximizing over all x not equal to 0 I can also maximize over all x such that norm of x according to some notion of norm I'll call it x alpha equal to 1 of norm ax over norm x. So different ways of writing this is any vector norm. So there are different ways of writing it but they all give you the same answer which is what we are calling norm of a. So I've been calling it the norm of a but actually we need to show that so it's the theorem is that the function defined above is a matrix norm okay c to the n cross n and norm ax. So this is a vector so this norm here is the vector norm that induced this quantity here with three bars. This is less than or equal to the norm of a times the vector norm of x for every a and x in c to the n. So pick any vector this norm of a is actually an upper bound on norm of ax divided by norm of x. If x equals 0 this is 0 and this is also 0 so it holds when x equals 0 as well. Also the norm of the identity matrix equals 1. So remember previously we had said that the norm of an identity of the identity matrix is greater than or equal to 1 for any vector norm any matrix norm that's because the norm of i squared is less than or equal to norm of i times the norm of i and therefore but i squared equals i and therefore norm of i is greater than or equal to 1 for the identity matrix i. For these kind of norms induced norms norm of i equals 1. This is also known as say so this is called induced norm or operator norm so I'll just write that here. Sir the definition of induced norm is valid for all type of vector norm. Yes and sir can you please explain that node part again I could not understand that why this max of norm of x less than equal to 1 should be equal to norm of x. So the so the by definition this is the max over all norm x all vectors x such that norm x equals 1 okay and I have to find this norm a x I search over all vectors which have unit norm and find the vector that maximizes the norm of a x. Now instead of searching over all vectors x such that norm of x equals 1 suppose I search over all vectors for which norm of x is less than or equal to 1 okay that includes so I guess first of all a very intuitive and simple way of thinking about it is suppose I consider the this norm to be the l2 norm then basically when I say norm x equals 1 I'm asking you to search over points relying on this circle okay and a is some matrix and among all the points on this circle I'm asking you which point maximizes norm a x then basically as you go further and further out on this circle this norm of a x will also increase so if I take a point x here versus a point x here the norm of a x over here will be bigger than the norm of a x over here if these are on the same line the same direction so if for example when you solve this problem if you get an if you if you say my solution is some particular x hash such that norm of x hash is equal to some value say b which is less than 1 okay so then this the maximum is happening at an interior point then then what I can do is I can propose a better solution I can say my solution is x dagger which is equal to your 1 over b times x hash if I do this then if I look at what happens to basically norm of a x this is equal to 1 over b times norm of a x hash so whatever maximum you found I'm able to find a an x x dagger notice that this also satisfies my constraint is actually equal to 1 over b times the norm of x hash which is equal to b and so this is equal to 1 okay so this satisfies the constraint that norm of x should be less than or equal to 1 but the value of the objective function a x dagger is going to be strictly greater than a x dagger itself so whatever solution you found I am able to find a solution that beats your solution it achieves a higher maximum and as a consequence of that the solution to this problem will always occur at a point where norm of x equals 1 and so you don't lose anything by saying I'll maximize over all x such that norm x is less than or equal to 1 it will always give you the same solution as the earlier optimization problem yes thank you sir yeah I don't have to restrict to x less than or equal to 1 I can maximize over all x not equal to 0 by simply including norm x and the denominator of this objective function that is the idea okay so this is the theorem so basically whatever we defined earlier is in fact a matrix norm okay so let's let's let's check this so again we need to verify non-negativity, positivity, homogeneity, triangle inequality and sub multiplicativity okay now obviously sir yeah sir some of these matrix norm is with double bars and some places with triple bars so will you explain that it's always with triple bars except when I was giving a couple of example here okay and I somebody else already asked the question I suppose you weren't paying attention and I explained that I'm going to use a different I'm going to define the l2 norm of a with three bars in a different way and so I'm saving the triple bar here for that purpose and so here is a l1 and a l2 I have defined with two bars because I'm going to use a different I'm going to define a different norm for the l1 norm and l2 norm okay otherwise matrix norms are with triple bars okay so now these are the four properties so first we'll take non-negativity so obviously by definition the norm of a is the maximum of a non-negative quantity this norm of ax this is a vector norm and it's a non-negative quantity so when I take the maximum of a non-negative quantity I cannot suddenly get a negative quantity so it is non-negative the positivity is because the this norm of a will be equal to zero if and only if ax equals zero for all x x not equal to zero so for that if you the simplest way is to consider this version of writing out norm a is the maximum over all x not equal to zero of norm a ax over norm x so for this maximum value to be equal to zero the value itself must be equal to zero for all x now if if norm a has to be if if not if if norm of ax is zero for all x it means that ax itself this is a vector norm it means ax equals zero for all x so and this is true if and only if a is equal to zero why is that true why should ax equals zero for all x mean that a is equal to zero positivity property of the vector norm no there is no norm here I'm just saying that a matrix a is such that if I find ax it gives me the zero vector for all x and I'm saying we can express the columns of the you can express the columns of the matrix as the transformation of the basis elements I don't follow that any matrix if we add the matrix on the basis elements then we'll get the columns of the matrix and if all of them are zero then the matrix will be zero so you're saying that you want you want to think of a basis in say c to the n and you want to project the columns of the matrix on this basis and from that somehow conclude that ax equals zero means a equals zero yes sir well I can think of a simpler way of saying that so suppose I range space of a zero means a zero always yeah that's so I'm asking why is that true and so one very simple way to see that is that if I take x equals the vector 1 0 0 0 0 then what that will do is to pull out the first column of a since ax equals zero for all x it's also true for this vector 1 0 0 0 0 that means that the first column of a must be zero next if I consider x equals 0 1 0 0 0 that that ax for such a vector will be equal to the column a2 but that's also equal to zero so the second column is also equal to zero and so on so if ax equals zero for all x then a is equal to zero similarly I mean on the contrary is trivial because if a is equal to zero then of course ax is equal to zero for all x yes sir I mean the same same proof yeah so the next property is homogeneity so we need to show that so if if I consider the norm of c times this matrix a this is equal to the max of so I won't write the constraint here because it's just repetitive it's the same constraint x norm of x equals 1 so I won't write that c ax and this is a vector norm so I can pull out the c and write this as mod c and this mod c doesn't depend on x so I can in fact pull it out of the max also times the max of ax and this is actually equal to mod c times norm of x a so it it does satisfy the homogeneity property this by definition is the norm of it then triangle inequality if I take a plus b this is equal to the max of norm a plus b times x which is equal to okay so this is the norm of ax plus bx and if I use the triangle inequality this is less than or equal to the max of norm ax plus norm bx in turn this is less than or equal to so instead of maximizing the sum of these two quantities I can individually maximize these two terms and add them up that can only increase the value so that is max of norm ax plus max of norm bx and this by definition is norm of a plus norm of b so it satisfies the triangle inequality and sub multiplicativity so if I consider norm of a b this is equal to max of I will consider the the other form so I will write it like this a b x over norm x over all x not equal to 0 so this is equal to so for a moment let us say bx is not equal to 0 then I can multiply and divide by bx by norm of bx and now I have the max of the product of two terms and what I can do is I can I can individually maximize these two terms they are non-negative terms so instead of maximizing the product of these two terms I can individually maximize these two terms and then take the product of the two maxima that will only be greater than or equal to this term so I can write it like this and of course I can consider bx to be some other vector y and maximize over all y not equal to 0 so max of norm ay over norm y times max of bx over norm x and this is equal to a but of course if bx is equal to 0 then a b x is also equal to 0 and so the this quantity you are trying to maximize this will be 0 and so such kinds of x's can be discarded safely from this maximization problem they won't be the maximum in fact they are achieving the minimum value that this can achieve which is 0 okay and the final part is that there is this the statement of the theorem also says one inequality which is that the norm of ax is less than or equal to the matrix norm of a times the vector norm of x for every a and x we need to show that of course that is trivial because if you consider norm of ax over norm x this is norm of ax divided by norm x and this is less than or equal to the matrix norm of a because by definition by definition the right hand side is the maximum value of the left hand side over all x not equal to 0 and so for any a and x this is true and so I will write that just for completeness the rhs is the max of the left hand side by definition so the maximum of a certain function is always greater than or equal to the value attained by it at any particular point so this of course assumes that x is not equal to 0 but clearly this inequality is valid when x equals 0 also clearly this ax less than or equal to norm a norm x holds when x equals 0 also so basically this in turn I can simply take this norm of x to the other side and hence norm ax is less than or equal to the matrix norm of a times the vector norm of x and finally if I consider the norm of the identity matrix this is equal to the max over all x such that norm of x equals 1 of the norm of identity times x but this is just norm of x itself so the constraint set says norm of x I mean you are maximizing over all x such that norm x equals 1 of norm x itself and so this is equal to 1 so that concludes this any questions okay so this this matrix norm is said to be induced by the vector norm so it's also called the operator norm or it's also called the least upper bound norm okay this is basically it says the least upper bound norm this name comes from the fact that the by definition this norm is the maximum over all x equal to 1 of the norm of ax so in other words it's the smallest upper bound you can place on norm of ax over all x satisfying norm x equals 1 or if you look at this definition here it's the smallest upper bound you can place on norm ax over norm x over all x not equal to 0 so that's why it's called also called the least upper bound norm okay so basically the fact that the operator norm or the induced norm is a matrix norm is a general property and it's true for all vector norms and so one way to show that some particular definition of a norm is indeed a matrix norm is to show that it is in fact induced by some vector norm and so so so now we have another way to show that so one way to show something is a matrix norm is to show that it is induced by some vector norm so here's an example so now I'm going to introduce the l1 norm with three bars so this is called the max column sum norm so I'm going to denote this by with three bars and one next to it so it's defined as max one less than or equal to j less than or equal to n of sigma i equal to 1 to n mod aij so this so what I'm doing here is I am taking the sum of the magnitude of entries in each column of the matrix so when j equals 1 it's the first column I'm taking this l1 norm of the first column and I'm looking across all columns first second third up to the nth column I'm asking I'm asking which column has the maximum l1 norm and I'm defining that to be the matrix l1 norm of this matrix a so this is induced by the vector norm l1 so to quickly see that if I have a equal to a1 an then this norm a1 is equal to the max one less than or equal to i less than or equal to n of the l1 norm of ai as I just mentioned the this norm here is taking the max column max of the l1 norms of the columns of a the largest l1 norm among the columns of a is this matrix norm now if we let x to be equal to this vector x1 through xn then if I look at ax l1 this is equal to the l1 norm of x1 a1 plus plus xn an the entries of this vector x multiply the corresponding columns of a and this in turn is less than or equal to I'm using triangle inequality sigma i equal to 1 to n l1 norm of xi ai I've just taken the norm inside the summation that's triangle inequality which is equal to now xi is a scalar here so I can pull that out sigma i equal to 1 to n mod xi times the l1 norm of ai which in turn is less than or equal to so I can replace all these by the largest column norm and that will only increase the value so sigma i equal to 1 to n mod xi times the max one less than or equal to k less than or equal to n norm of ak and this by definition is the l1 norm as defined here the max column sum and so this is equal to the l1 norm of x the sum of all the entries it doesn't depend on i anymore this can come out of the summation times the matrix norm of a so what this means is that for any x the l1 norm of ax is less than or equal to so I'll write that this implies this is true for any x the l1 norm of ax is less than or equal to the norm of x the l1 norm of x times the matrix norm of a and so if I it is also true that if I take since it's true for every x if I take the max over all x such that x l1 equals 1 of the norm of ax this is going to be less than or equal to the matrix norm of a but I want to show that these are actually equal that this is this definition that I have here is in fact induced by this for that I need to do a little bit more work so now let x is equal to ek k equal to 1 2 up to n so this ek is the kth column of i so then what I have is that max norm x equals 1 of ax is at least equal to the value it takes at one of these an example vector which is for example e1 if I take e1 that satisfies l1 norm of e1 equals 1 and I can that will give me the l1 norm of the first column of a so if I in general if I take the kth column then it will be ak l1 okay so this is also true and so so then this is true for all of these k k equal to 1 2 up to n which implies that this maximum of this is greater than or equal to the max 1 less than or equal to k less than or equal to n of norm ak l1 which by definition is the matrix norm of a so I have shown that this max of this is less than or equal to norm of a l1 I have also shown that this max of this is greater than or equal to norm of a l1 so if I call this step a and this step d these two together imply max 1 ax l1 equal to so this shows that this the max column some norm as I defined it is in fact induced by the l1 norm okay so it is an exercise for you to show that this is a matrix norm from by showing that it satisfies the four properties for a matrix norm okay so that's all I have time for today I'm actually a little over time