 So, suppose you write C as A times B, where A is a matrix of size m by k and B is a matrix of size k by n. Then one way to view this is in terms of an inner product, which we have seen in one of the previous classes already, but we will also formally define it in a few minutes, but inner product. So, suppose I have A and B as vectors of the same length, then the inner product is defined as A transpose B, which is just a real number. Okay, then if I consider A i transpose, what is the length of a vector? The length of a vector is the number of elements in it. Okay. Sir, wouldn't it just be equal to the dimension? You like dimension? I can write that. Okay. No, but you are right in that we will momentarily define length of a vector also and so same dimension or size is what I really meant here. Okay, so suppose A i is the ith row of A and B i also in R power k is the jth column of A, jth column of B. Hello, sir. Yeah, please go ahead. Sir, what is VECS in bracket I? Vectors. Vectors. Okay, sir. So, the ijth entry of the product AB is nothing but the inner product between the ith row of A and the jth column of B. So this is one way to view matrix multiplication. Every element of the product matrix is an inner product between a row of A and a column of B. The other representation is the column representation. Sir, it should be sigma A transpose P j. There is no summation here. Yeah, so there is no summation here. This is the inner product representation. So if I have two matrices and I am taking their product, recall that if you want the 1, 1th element of their product, you have to take the first row of this matrix, first column of this matrix, take their inner product and that gives you the 1, 1th element of this product. If I want the say 2, 3th entry of this product matrix, then I must take the second row and then the third column. So there is one more here and then I take the inner product of the two red vectors and that gives me the 2, 3th entry. So this is called the inner product representation. The other thing is the column representation. So C j, if I define to be the jth column of C, which is going to be in R to the M, then this is equal to summation i equal to 1 to k a i b i j. This is true for j equal to 1, 2 up to N. So each column of this matrix C is a linear combination of the columns of A and what are the coefficients they are given by b i j. The jth column of B gives you the coefficients of the linear combination that will form the jth column of C. So it is a linear combination, I repeat it is a linear combination of the columns of A with the coefficients given by the jth column of B and the third is the outer product representation. So we can also write the entire matrix C. So we are going from small to big. Here we looked at how to write an i jth entry of C, a single entry of C, this is for getting you an entire column of C and this is a representation which will give you the entire matrix C itself. This is equal to summation k, I will write it in terms of i, i equal to 1 through k a i b i transpose. So this is the i th column of A and this is the i th row of B. Now since A is of size m by k, the i th column of A is of size m by 1 and the i th row of B is of size 1 by N. So when I take this outer product I will get an m by N matrix. And so the matrix C is the sum of k rank 1 matrices of size m by N. Okay so now we move on to the inner product. Sir what is A i and B like column representation? Sorry I did not understand your question. Sir in column representation session what is A i in that summation, inside summation? These are the columns of A and A i is the i th column of A. Then that summation runs from 1 to m, sir. 1 to, okay so maybe just hold on one second, A is of size m by k. So A has k columns, yeah so the summation goes from 1 to k. So what is the question? There are k columns in the matrix A, A is of size m by k. A 1 is the first column, A 2 is the second column, A k is the k th column. Okay sir now I got it. Jth column of C belongs to R to the m. The jth column of C is in R to the m, yes. Every column of C is in R m. C is of size m by m. But it is written R to the power of m there. R to the power? m in column representation. C j is the jth column of C, it is in R to the m. Oh R to the n. Okay thank you. You are right. Yeah each column is in R to the m. Okay and there are n such columns. Is that clear? Yes sir. Yeah these things are a little tricky. It's good to think about it independently. Yeah but sometimes it can be a little bit confusing. So the usual inner product. So given two vectors x in C to the n and y in C to the n. So I'm using complex vectors here because it's just the more general definition. So the inner product between x and y is written like this and it is equal to y Hermitian x. Or it is defined as y Hermitian x and this is also called the dot product. So Hermitian is nothing but the conjugate transpose. So you take the transpose of y and then you take the complex conjugate of every entry. That is y Hermitian. Y Hermitian inner product with or dot product with x is defined to be the inner product between x and y. There are other definitions possible but we'll revisit that later in the course. But this definition here it satisfies two properties that I can immediately tell you. The first is that if I take alpha x1 plus beta x2 inner product with y. This is going to be equal to alpha inner product of x1 and y plus beta inner product of x2 with y. But if I take the inner product between x and alpha y1 plus beta y2. What will I get? A conjugate. X y1 plus beta star x y2. So it is linear in the first argument and it is conjugate linear in the second argument. Now we can immediately define orthogonality. A set of vectors said to be orthogonal if every pair of vectors in the set are orthogonal. So you take any two vectors, you take their inner product, you get zero. Then we say that if that happens for every pair of vectors in the set, we say that the set of vectors are orthogonal. Now one inequality, very famous one related to the inner product is what? There is one very famous inequality associated with the inner product. Cauchy shots. Exactly. So before I put down the Cauchy Schwarz inequality, let me just say one small thing. If I take square root of the inner product of x with itself, this is called the Euclidean length x and c to the n. And also it is also denoted by norm of x2. Now if the inner product of x with itself is one, then the vector x is said to be normalized or that x is a unit vector. So basically if I take any vector x in c to the n, then x over square root of the inner product of x with itself is a unit vector pointing in the direction of x. Sir. Yes, please. Sir, what is the subscript 2 in the above statement? What does it mean? Can you take a guess? In the product of two quantities? No. The square root, take a square root, right? Yeah, no, not that. So the point is that if you look at what this thing is doing, it's taking every entry of x, taking the mod square of it, adding that up across the entries. And then you're finally taking a square root. Okay, because you're taking the square of each of the entries and then eventually taking the square root, it is denoted by norm x2. So if, I mean, since you asked the question, I'll say that norm xp is going to be equal to, okay, so this is a definition of what is called the p norm of x. It's also known as the LP norm and it turns out it is a norm p greater than or equal to 1. Okay, but I haven't defined, I haven't defined what a norm is. So for now you take it on faith that if for any p greater than or equal to 1, I can define norm xp to be this quantity here and it actually is a norm. So we'll study that later. So there are other kinds of inner products and norms that you can define, which we'll study later. But now I'm ready to state the Cauchy-Schwarz inequality. Sir, I have a question. We cannot define 0 norm, right? So you can define norm xp for p less than 1 and you can take p equals 0. You can even take negative p. But for p less than 1, this norm xp will not be a norm. And so I don't want to get into the details right now because I haven't defined norm yet. We will define it in one or two classes and then we can discuss these properties. But for now, I just want to relate it to this. Somebody asked me why it is norm x2 here. And so I'm just answering that it is actually coming from a more general definition of the length of a vector which can be defined like this. And this is a valid definition of the length of a vector for p greater than or equal to 1. So let's not worry about what happens when p is less than 1 or equal to 0 or even negative just for the moment. We'll come back to that point later. So the Cauchy-Schwarz inequality simply says that the inner product between y and x in magnitude is less than or equal to the inner product between x and itself power half times the inner product between y and itself power half. And with equality if and only if x and y are linearly dependent. Basically they point in the same direction. So the proof is very quick. So I'm just going to in the dying minutes of this class run through the proof. All you do is you start with noting that x inner product with x is always greater than or equal to 0 with equality if and only if x equals 0. Okay, as I wrote it here, you can see that the inner product of x with itself is the sum of the squares of the entries of x power 1 by 2. And so if this equals 0, the sum of the squares of the entries are is 0, which is only possible if every one of these entries is 0. Because they're all non-negative quantities. So now we'll take this x minus lambda y inner product with itself. And this is because I'm taking the inner product of a vector with itself and lambda is some scalar here. And since it's the inner product of a vector with itself, this is always greater than or equal to 0 with equality only if and only if x minus lambda y equals 0. So from this, you can already see that the equality condition will only be satisfied if x equals lambda y so that this difference is 0, which means that x and y are linearly dependent. And this is true for every x, y belonging to c to the n and lambda belonging to c. So we just expand the left hand side. And if you expand it out, you will see you just have to take this Hermitian times this and expand it out. And this is equal to x Hermitian x minus lambda x Hermitian y minus lambda star y Hermitian x plus mod lambda square y Hermitian y. Now if y equals 0, then this inequality is trivially satisfied. So if y x inner product is less than or equal to this times this, if y equals 0, the left hand side is 0, the right hand side is also equal to 0. And so this inequality is trivially satisfied. So I'll just say y equal to 0, the Cauchy Schwartz is trivially satisfied. So assume y is not equal to 0. And in that case, I can choose lambda equal to y Hermitian x over y Hermitian y, which is equal to the inner product between x and y divided by the inner product between y and y. If I substitute this value of lambda here, you see that I get y Hermitian x times x Hermitian y, which is the same as mod of x Hermitian y squared divided by y Hermitian y. And this term is also y Hermitian, the conjugate of y Hermitian x is nothing but x Hermitian y times y Hermitian x divided by y Hermitian y. So I'll just, and in this last case, I get mod of y Hermitian x squared and one of the y Hermitian y's will cancel and I'll be left with y Hermitian y in the denominator. So just writing it out, the above becomes x Hermitian x, which stays as it is minus x Hermitian y squared over y Hermitian y minus x Hermitian y squared over y Hermitian y plus x Hermitian y squared divided by y Hermitian y. So things I've used here, so this is greater than or equal to 0 and these two terms obviously cancel. And so which implies that x Hermitian x minus x Hermitian y squared over y Hermitian y is greater than or equal to 0 from which the inequality follows. All I have to do is to take this to the other side, bring y Hermitian y to the top and then take the square root throughout. So I used a couple of things here, one is that x Hermitian y complex conjugate equals y Hermitian x. Okay, this is one, I guess this is the only side result that I use and you can see that you can, this is not difficult to see. So you can see that this is true, just write out what it is then, so just keep in mind that y Hermitian is equal to y transpose complex conjugate. Okay, so that's the proof of this inequality. Okay, we'll stop here.