 So last time we were discussing norms and matrix norms in particular, today we will continue our discussion on matrix norms. Just to recall, matrix norm is a mapping from the space of square matrices r to the n cross n to the real number line r and we say that this matrix norm which is denoted by this symbol with three lines around it. This is a matrix norm if for any a and b in r to the n cross n, it satisfies these four properties. The first is that it is non-negative. The second is that it's positive, meaning that it can only be equal to zero if the matrix itself is equal to zero. The third is that it's homogenous and the fourth is that it satisfies triangle inequality. So basically any vector norm I start with will satisfy all four of these properties. So if I think of an n cross n matrix as a big vector containing n squared entries in it and I define a vector norm on that, that will certainly satisfy these four properties. The only question mark is whether it satisfies this last property which is called the submultiplicativity property or not. So this is what we will examine today. So some vector norms are in fact matrix norms when you apply them on r to the n cross n whereas others are not. So we'll start with the L1 norm. So remember that the L1 norm of a vector is the sum of the absolute value of all of its entries. If I simply extend that to an n cross n square matrix, then I would define the L1 norm of a matrix A to be the summation of all of its entries in magnitude. So the question is, is this a matrix norm or not? So the answer is yes. Of course, this norm as defined here does satisfy the non-negativity, positivity, homogeneity and triangle inequality by virtue of the fact that it is an L1 norm in the vector space. So it directly satisfies those four properties. And so we only need to check whether it satisfies the submultiplicativity property or not. So I'll just write that here. 1, 2, 3, hold because it's already a vector norm. So now about submultiplicativity. So what I need to show is that if I take any two matrices and I find AB, L1, this is going to be less than or equal to the L1 norm of A times the L1 norm of B. That's what I need to show. Now this quantity AB, L1 is the sum of the magnitude of the entries of AB, which is equal to sigma ij equal to 1 to n. So this is the sum over all the entries of this product matrix. But each entry in the product is the inner product between a row and a column of A and B respectively. So I'll write that like this. So if you remember, we discussed this different ways of writing out a matrix product. This is the way by which we write out every entry of the product matrix, Aik times Bkj. This is the i, kth entry of this product matrix AB. Now this quantity itself, I can upper bound by taking the magnitude inside the summation. This is less than or equal to summation. Now because the summation will go over ij and k, I'll just write them together, ijk equal to 1 to n mod of Aik Bkj. This in turn is less than or equal to, what I'll do is, see here it's index k and then there's k repeating here. I'll just replace this k with an l and take the summation l going from 1 to n as well. What I'm doing there is I'm introducing a whole bunch of non-negative terms into this summation and so that can only increase its value, it cannot decrease it. So I'll write this as sigma ijkl equal to 1 to n mod of Ail mod of Bkj. So this is a double summation over two indices, just make this a little meter, ijkl equal to 1 to n. Now this is just as, so notice that in the first term it depends on i and l, it has no j and k in it, the next term has jk in it but no i or l. So this is actually equal to the product of these two terms sigma il equal to 1 to n mod of Ail times sigma jk equal to 1 to n mod of Bkj and this is nothing but by our definition here, this is nothing but the l1 norm of A times the l1 norm of B and thus ABl1 is less than or equal to the l1 norm of A times the l1 norm of B. So it satisfies some multiplicativity and hence this AL1 as defined here is indeed matrix norm. So we can go to the next possibility. So we did l1, now we can look at l2. Sir. Yeah. The notation for matrix norm will have three vertical bars, right? Why have you ignored the one, one of the bars? Yeah. So it turns out that I am going to define the l1 matrix norm a little differently in a few minutes once we discuss something called induced norms. So it turns out that this is not quite the definition of the matrix l1 norm that I am interested in and so I have used a different notation. So for these norms that I am discussing here I will use only two bars to distinguish this norm from another notion of l1 norm on matrices that I am going to define momentarily. Okay. Okay sir. So the l2 norm also I am going to denote it with two bars because again this is not quite the l2 norm on matrices that I am going to later be interested in but I will define this to be. Sir. Is there another question? Yeah. Sir. Sir, in the previous one you have written like Aik, Bik and you have introduced Ail right? So what does it mean? I mean for every Aik you are replacing it with bunch of Ails right? I mean you are adding lot of Ais. Yeah. Okay. Sir. So you are taking Aik and you are adding some other like Ais right? A1. Yeah. Is it? Yeah. Okay. There are many, many more. I am adding n into n minus 1 terms into the summation which will only increase its value but they are all in magnitude so none of them is negative. So it can only increase the value. Okay. Thank you. So the l2 norm I have defined like this. So it is so just like the vector l by l2 norm which is the square root of the sum of the squares of all the entries I define the l2 norm to be the sum of the squares of all the entries in the matrix and then you take the square root. So is this a matrix norm? So once again by virtue of the fact that this is the l2 norm of the vectorized version of the matrix, this will satisfy the non-negativity, positivity, homogeneity and triangle inequality. Again the only thing we need to look at is whether it satisfies the sub multiplicativity property or not. And again for this norm it turns out the answer is yes. So we will see that very quickly. So once again if I take the l2 norm of a product AB this is equal to sigma ij equal to 1 to n. So I will consider the square so that I do not have to keep writing square roots. So this AB squared if I show that this is less than or equal to l2 norm of A squared times l2 norm of B squared that is also good enough for me. So this is ij equal to 1 to n. I have to take mod square of this term here. So I will just write the same term here sigma k equal to 1 to n aikvkj square. And now this is actually the square of the inner product between the ith row of A and the jth column of B and I can use Cauchy Schwarz inequality here and write this as less than or equal to sigma i equal to ij equal to 1 to n sigma say k equal to 1 to n mod aik square times sigma just to just for convenience I will replace k with m and write m equal to 1 to n mod of Bmj square. This is just from Cauchy Schwarz inequality. Then notice that when I take this if I take this the the if I look at these two terms this term has a summation over k but it has no dependence on j and this term has a summation over m but it has no dependence on i. So I can write this as this is equal to sigma i k equal to 1 equal to 1 to n mod aik square times sigma j m equal to 1 to n mod Bmj square. These two are exactly equal just another way of writing this double summation with the summation inside of it and so this is equal to A2 square times B2 square. So it does satisfy this sub multiplicativity property. This particular matrix norm is actually called the Frobenius norm. So I will maybe mention just one property of this norm. The property is that this particular norm is invariant to left or right multiplication by a unitary matrix. So one useful trick in this direction is that this norm A l2 square is actually equal to the trace of A transpose A. So if you think about the entries of A transpose A along the diagonals you will get the sum of the squares of each column the entries of each column of A. So because we are doing this online I will just maybe show you that very easily very quickly. If I have A11, A12, A21, A22 and if I take its transpose and left multiply it that would be A11, A21, A12, A22 and then if I look at the diagonal entries they will be the first diagonal entry will be A11 square plus A21 square. The second diagonal entry will be A12 square plus A22 square. And so if I take the trace of this matrix this is A transpose A. So if I take the trace of this, this is equal to A11 square plus A12 square plus A21 square plus A22 square and so it is exactly this Frobenius norm as we defined it here. So this is one useful formula. Now what I wanted to say though is that if you have a matrix A and I write out its columns A2 up to An then if I look at what this A2 square is this is actually equal to the sum of the squares of all the entries in this matrix but I can also write that as A1 L2 norm square which is the sum of the squares of the entries in the first column of A plus A2 L2 norm square plus plus An L2 norm square. This is another way of writing it. Now the L2 norm by itself is unitary invariant meaning that if I have if U is unitary then Ux L2 norm is equal to the L2 norm of x itself. The way to see this immediately is that recall again that the L2 norm square in the vectors for vectors the L2 norm square is equal to x transpose x. So if I take the L2 norm of Ux that is equal to x transpose U transpose Ux and U transpose U is the identity matrix so that is the same as x transpose x. So this is true for any unitary matrix and for the L2 norm. So we have that if I take the L2 norm of U times A square then that is equal to by using this formula here it is equal to U A1 L2 norm square plus U A2 L2 norm square plus etcetera plus U An L2 norm square. So now I am using the column view of matrix multiplication. When I multiply U with a matrix A the columns of the product are equal to the multiplication of U with each of the individual columns of A and so this is because this is equal to A1 square plus this is equal to a L2 norm of A1 square this is equal to the L2 norm of A2 square plus etcetera plus the L2 norm of An square which is equal to the L2 norm of A square. So this Frobenius norm is invariant to left multiplication by a unitary matrix. You can similarly show that if I have if I take U A V L2 norm square this is equal to A L2 norm square when or I will say if U V are unitary. So basically this Frobenius norm is invariant to left or right multiplication by a unitary matrix. So the next thing I want to consider is the L infinity norm. We looked at L1, L2 and L infinity so we are basically covering most of these LP norms. For P not equal to 1, 2 or infinity it's a little more difficult to figure out what's going on so we don't and usually in applications you don't encounter those norms so we don't worry about too much about those norms. So this L infinity norm if I simply extend the definition it would be the max 1 less than or equal to ij less than or equal to n mod aij. The L infinity norm of a vector is the max magnitude entry and so I am extending that and saying the L infinity norm of a matrix is the max magnitude entry across the matrix. Is this the matrix norm? So for this particular example it turns out that the answer is no. So once again when you want to show that it's not a matrix norm all you need to do is to provide one counter example where it does not work and that is enough. So if I let the matrix so if I consider j equal to the all ones 2 cross 2 matrix then j square what is j square equal to it's the all 2's matrix which is equal to 2 times j. So if I look at the L infinity norm of j as per this definition it's the max magnitude entry it is 1 but if I look at the L infinity norm of j square it is 2 which is not less than or equal to the L infinity norm of j square. This was one of the requirements in this see if some multiplicativity were to hold then the L infinity norm of j square must be less than or equal to the L infinity norm of j times the L infinity norm of j which is L infinity norm of j square but that is equal to 1 here is not bigger than 2 and so this is as as written here this is not a matrix norm. However a slight modification to this norm if I define now I am using 3 bars to distinguish it from what I wrote about if I write it to be n times a infinity n is the dimension of a it a is an n cross n matrix this is a matrix norm. So if I consider so once again because it's just a scaled version of a vector norm the first four properties namely non negativity positivity homogeneity and triangle inequality are naturally satisfied by this. So the only property we need to check is the sub multiplicativity property. So if I consider a b infinity then this is equal to n times the maximum entry in magnitude of the ijth entry of a b and as we wrote above this is equal to sigma k equal to 1 to n a i k b k j. Now first of all I can if I take the modulus inside the summation I cannot decrease the value of the summation so it is less than or equal to n max 1 less than or equal to ij less than or equal to n of sigma k equal to 1 to n mod a i k b k j and that in turn is less than or equal to. So what I can do is I can replace all these a i k's with the largest a i k value the largest magnitude entry in the entire matrix that will only increase the value of this summation I can replace this b k j's with the largest magnitude entry of the matrix b that will also further only increase the value of this summation and so that is less than or equal to n max 1 less than or equal to ij less than or equal to n of sigma k equal to 1 to n a infinity where this is the maximum magnitude entry which is the norm a infinity with two bars as I defined it here times b infinity and now there are n such terms here so I get an extra factor of n by removing this summation here and so this is equal to n times a infinity times n times b infinity which in turn is equal to the norm as I defined here times the norm as I defined here okay so with a small modification to the definition I can get I can get a definition which is indeed a matrix norm any other any questions so far