 So now we move on to discussing about norms. This is chapter five in the textbook. So I can say that probably whatever we discussed so far is loosely things you've already seen before. There's probably not too much new material that you have never heard of that I've covered so far. But from now on, hopefully we will, you'll get to see some new things when we discuss these norms and its properties. So what is the norm? A norm is simply a way to measure length. So we've already seen the Euclidean norm, which is given by, if I take a vector z, then z transpose z, which is equal to summation i equal to one to n, z i square. Okay, here it's implicitly assuming it's a real vector. So the mod is not required. So let me not confuse you and remove the mod here. Okay, this is one way to measure the size of a vector. And I think the last time I alluded to this, but there's a, for example, a simple generalization is I could define, I'll just write a subscript a here and I can define the measure of a length to be equal to summation i equal to one to n, a i times z i squared, where a i's are some numbers greater than zero for i equal to one to n. So we'll see that in just a few minutes, we'll see that this is also a valid definition of the norm of a vector. And a question that we'll answer is that are there other generalizations? And also, how does one extend this to somehow measure the size of a matrix? So in general, we humans are very fond of associating numbers with everything. And so that once you associate a number with things, you can even think about rank ordering things based on that number that you're associating with each of these quantities. And so perhaps that could be one way to think about why we are interested in norms is because it allows us to associate a number with different things. So let me formally define a norm. So let v be a vector space over a field f. So for the purposes of this course, think of it as either the real line or the complex plane. Then a function f or not f, we'll define it this way, which maps from the vector space v to the real line is a vector norm. If for all x, y belonging to this vector space v, it is true that the norm of x is greater than or equal to zero. Norm of x, actually I'll call this one a, if and only if x equals zero to c times x, norm is equal to more c times the norm of x for every c belonging to f. And the last one is the triangle inequality, norm of x plus y, this is the only place where y enters into the picture. So this property is called the non negativity property and this property is called the positivity property. This property is called the homogeneity property and this is the triangle inequality. Now, couple of variations that if the property one a is not satisfied, then it is called a semi-norm. And if the triangle inequality is not satisfied, then it is called a pre-norm. So we'll refer to some of these later when we want to state certain properties. So for some properties, it's enough if the norm that we're considering is a pre-norm. For some other properties, it's enough if it's a semi-norm but for others it needs to be a norm. So these are the four things that a function which maps a point in the vector space to the real line needs to satisfy for it to be considered a norm. So essentially it's mapping to r plus, positive half of the real line. So the first property is that if is a vector norm then norm x minus norm y in magnitude is less than or equal to the norm of x minus y for every x, y and v. You can see that this is very similar to a triangle inequality. So if I just want to illustrate this by a picture in two dimensions, then if I have a vector x and another vector y, then x minus y is x minus y actually this vector, so change color. This is x minus y. Then what it's saying is that the length of this vector is more than the difference between these two lengths. So how do we show this? It's very simple. So what we do is we take y and we can write y to be equal to x plus y minus x. So this means that if I take the norm of y, if I take the norm both sides and then I apply triangle inequality, this is less than or equal to the norm of x plus the norm of y minus x. So this means that if I take norm y minus norm x that is less than or equal to the norm of y minus x which is equal to the norm of x minus y because scaling by minus one will only scale the norm by the magnitude of minus one, which is equal to one. And similarly, oops, similarly you can exchange x and y and or you can write x to be equal to y plus x minus y this exchange is x and y here and that will give you that norm x minus norm y is less than or equal to norm of x minus y. So you see that this and this both are actually less than x minus y which can be compactly written as norm x minus norm y is less than or equal to norm of x minus y. Okay, so earlier we looked at this thing called the usual inner product or the dot product where x y was defined to be y transpose x or y Hermitian x for complex, the complex case. So this is a, I mentioned several times that this is a special case of a definition of an inner product. The more general definition is like this. So let V be a vector space over F, then this dot comma dot bracket, which maps from the Cartesian product of V with itself to the field F is an inner product if for every x, y and z belonging to this vector space V it is true that one, the inner product of x with itself is greater than or equal to zero, which is the same as our non negativity. And second, which again I'll call one a x, x equals zero if and only if x equals zero, which is the positivity property. Linear in the first argument x plus y z equals x z plus y z, this is also called the additivity property. And then there's a homogeneity property, c x, y equals c x, y and the fourth property is that x y is the complex conjugate of y x. So if you exchange the arguments, then what you get is the complex conjugate of the inner product. So this is the formal definition of an inner product. Any function that maps V cross V to F is an inner product if it satisfies these five properties. Okay, so the reason why I brought up the definition of the inner product is because there's a close connection between inner products and norms. And here's the lemma that makes this connection. If is a vector inner product on V on a vector space V, then the square root of the inner product of x with itself for any x is a vector norm on V. So this means that if I define this to be a vector norm where the inner product has these four properties, then I can show that this notion of a vector norm satisfies the four properties I need in order for, in order to define a vector norm. And therefore it is a valid vector norm. I won't prove this here. I'll leave it for you to look up in the text, but it's a straightforward proof. The proof essentially, the only idea that the proof uses is the Cauchy Schwarz inequality, mainly to show this triangle that the triangle inequality holds. The other properties are trivial from the fact that this is a inner product and it comes from here itself. But you need to show the triangle inequality for which you need the Cauchy Schwarz inequality. One other thing is that a norm divided from derived from, so when we define a norm in this way, we say that such a norm is derived from an inner product. And a norm divided from derived from an inner product, it satisfies what is called the parallelogram law, which norms that are not derived from an inner product, we'll see examples of that momentarily. They don't need to satisfy this. So the parallelogram law is that U plus V, if U and V are two vectors, then this norm squared plus norm of U minus V squared is equal to two times the norm of U squared plus the norm of V squared for every U V belonging to this vector space V. So if you think of this as the inner product square, you can see that when I take the inner product of U plus V with itself, I'll get the inner product of U with itself plus the inner product of V with itself plus two times the inner product of U with V. When I take the inner product of U minus V with itself, I'll get the norm of U squared plus the norm of V squared minus two times the inner product between U and V. That is for the real case, for the complex case, you will get something like U Hermitian V plus V Hermitian U, but you will get exactly the negative of that out here. So those two will cancel. And so all you're left with is two times norm U squared plus norm V squared. So if it's derived from inner products, it's easy to see that the norm satisfies this parallelogram graph. And it's called the parallelogram law because these two terms are, if I construct a parallelogram from, with U and V as the sides, then if I complete this parallelogram, then this vector is U plus V and this vector is U minus V. And so what it's saying is that the lengths of the diagonals of the parallelogram squared, if you add them up, that is equal to two times the lengths of the sides squared, the sum of the lengths of the sides. So this part is length squared of diagonals and this part is of the sides. Okay, so now the next thing I want to talk about is various examples of vector norms. Since we are like really at the last tail end of the class, I'll stop here and continue with this in the next class.