 So, let's begin. The last time we looked at the inner product and the Cauchy Schwarz inequality, this was just a recap. And then we discussed about orthonormal basis and some of their properties. And we went through the Gram-Schmidt orthonormalization process, which can be used to produce an orthonormal basis for the vector space spanned by a given set of vectors. And then we looked at some properties of orthonormal matrices. So today we will talk about two other topics. The first is determinants and the other is norms. So are there any questions about the previous class before we begin? Okay. So we'll start talking about determinants. So throughout we'll consider square matrices, A is in R to the m by m. Okay. So I think all of you certainly know what the determinant is. Is there anybody in the class who hasn't heard of or seen what a determinant is? Okay. So I assume all of you know what it is, but we'll just formally define it so that we're all on the same page on this. So if I consider the matrix Aij to be the matrix obtained, it's an m minus 1 cross m minus 1 matrix obtained by deleting the ith row and jth column. Okay. Then we define determinant of Aij, which is the determinant of a slightly smaller matrix to be the minor associated with the element small Aij. Okay. And we define Cij to be the cofactor of Aij, which is equal to minus 1 power i plus j times determinant of Aij. Then this is the definition of the determinant of A, this is called the cofactor expansion. So the way we define the determinant here, it's a recursive definition. The determinant of a matrix of size m by m is defined in terms of the determinant of a smaller size matrix of size m minus 1 by m minus 1. So determinant of A is equal to the summation i, j equal to 1 to m, Aij times Cij. And notice that there is a floating index i, i belonging to 1, 2 up to, and I can also define it to be the summation over i going from 1 to m, Aij, Cij. And now there is a floating coefficient j here and I can compute this for any j. Okay. So they are basically m squared ways of compute, sorry, 2 m ways of computing the determinant and they all give the same answer. Okay. So there are 2 m ways of computing the determinant and all these give the same value. So this is another surprising fact about matrices is that you take a square matrix and there are 2 m different ways, seemingly different ways of computing the determinant. But whichever way you do it, you always get the same answer. How do we see this? In fact, it's not that difficult. All you have to do is to take the trouble to write out what the determinant expansion would be. And when you write it out, you see that actually whichever way you write this expression out, it's all the same terms that will finally show up in the determinant formula. And that's why they all actually should have the same value. Now in order to complete this definition, I need to know what, how to compute the determinant of a 1 cross 1 matrix because Cij itself is something that is minus 1 power i plus j times determinant of an m minus 1 by m minus 1 matrix. So determinant of a single element or a scalar is that value itself. So this is a recursive definition. And yeah, so I think, so this is the basic definition of a determinant. Now we will discuss many properties of the determinant, but maybe just to keep the order here, one, so I'll just make one or two small remarks. One is that if a is a triangular matrix, then the determinant is equal to the product of the diagonal entries. And then the other thing is that if mod of the determinant of a is the, is called the principal volume of a matrix. So this is the volume of a parallel piped, who's defined by the columns of, yeah. So this is the volume of a parallel piped, defined by the columns of a. So for example, if I, I can define a matrix A, which belongs to R power m cross n, which is a non-square matrix, right, m by m, maybe my handwriting is not clear, but this is m cross m. Okay. Thank you. Okay. So this is the volume of a parallel piped defined by the columns of a. If I, I can only show you things on a two dimensional case. So if I define vectors associated with the two columns of a. So let's say it is two, three, and five, four, something like this. Then I take two along the x-axis, and then three along the y-axis. This is a vector that looks something like this. And then I take five along the x-axis, and I take four along the y-axis. This is another vector like this. So I use these two and complete a parallel piped. And then I look at what is this area. So that's what I mean by the volume of a parallel piped defined by the columns of a. This is called the principal volume of a matrix. And this also happens to be equal to the magnitude of the determinant of a. The other small point I want to say is that this is a multi-linear function of a. That is, it's actually linear in each element by itself. But it's a combination of various terms. So the same element could potentially, so it's a combination of various terms involving different elements of a. So just to make this point clear, I'll actually take the trouble to write out the determinant of a 3 cross 3 matrix. And we also write it in this way by to put two bars around it that can also denote the determinant. But for the moment, I won't do that because you can see here I'm using the two bars to denote the magnitude. So this determinant of this is equal to a11, a23, sorry, a22, a33, minus a11, a32, a23, minus a21, a12, a33, plus a21, a32, a13, plus a31, a12, a23, minus a31, a22, a13. So I should take any one element, like for example, a11. It appears in two terms and it's a linear term that appears here. So for example, if I unilaterally increase a11, then the value of the determinant will also unilaterally, will also linearly change. It's a linear function of a11 by itself. And if I take a22, a22 appears here and here. It also appears in exactly two terms. If I take a33, it appears here and here. Also appears in exactly two terms and so on. And each term contains three terms from this, which is the size of the matrix. This is 3 cross 3. So each term that appears here contains exactly three terms. And then there are a total of six terms. And each term appears linearly in it. So it's a multilinear function of the entries of a. Sir? Yeah. Sir, could you explain what is the meaning of multilinear? It's linear, but there are multiple terms, each of them being linear. So when you have a single variable, you know that the variable could, the function could be linear or nonlinear. But when you have multiple variables, then it could be linear in each component, but then it could be the sum of many such terms. So for example, if I had a function like a11, a12, a13, a21, a22, a23, a31, a32, a33. This is also a multilinear function of a. It's linear in every one of the components. So here you have a sum of multiple such terms. Yeah, go ahead. But here it's not sufficient to scale just one element, right? We'll have to scale either one whole row or one whole column to scale the determinant. No, no, no, I'm just looking at what happens to the determinant if you were to scale one particular entry of the matrix. It'll become different. The determinant won't be scaled then. Because the element a11 appears only on the lower case. Think of it this way. Suppose I consider a matrix T, 5, 3, and 4. And I say, so suppose this is a matrix B. And then I call f of T equals determinant B. So this is actually a function of T. So if I define f of T to be determinant of B of T, then f of T is linear. Yes, yes. This is what I'm trying to say. Okay. This is for this. Now, there is an alternative definition, which is sometimes useful by itself. Okay, so it is through three properties. So the first is that the determinant of the identity matrix is always equal to one, no matter what the size of the identity matrix is. To the determinant changes sign when two rows are exchanged. The determinant depends linearly on the first row. So any function set which satisfies these three properties is a valid definition of a determinant. So basically what we are doing here is we are mapping a matrix to a number. And any mapping of a matrix to a number which satisfies these three properties is in fact a valid definition for the determinant. And it's the same definition as what we discussed earlier. Okay, so this is another definition, so just for you. And we'll come back to this later when it's useful. Okay, so now several properties. Well, but before I discuss these properties, maybe I'll just explain what I mean by this. So if I take a matrix, say determinant of A, B, C, D. This is equal to A, D minus B, C, which is equal to the negative determinant of the matrix C, D, A, B. And also, if I take a permutation matrix. So for the moment, just keep in mind or I'll just say that a permutation matrix is a matrix obtained by exchanging. So this is my short hand notation for exchanging rows of the identity matrix. Okay, this is some one way to think of a permutation matrix. A permutation matrix, if I denote it by P, then determinant of P is always equal to plus or minus one. So for any permutation matrix, it's obtained by exchanging rows of the identity matrix. And in the exchange rows, the sign of the determinant changes. The determinant of P is plus or minus the determinant of the identity matrix, which is equal to one. And by linear on in the first row, what I mean is that if I take the matrix. So determinant of A plus A dash, P plus B dash, C, D. This is equal to determinant of A, B, C, D. Plus the determinant of A dash, B dash, C, D. So I can take linear combinations of the first row or any row and the determinant can be split as the determinant of not linear combinations. I can consider the first row to be linear combination of vectors A, B and A dash, B dash. And the determinant splits as the sum of the determinants. The first being the determinant where the first vector is the first row. The second being the determinant of the matrix where the second vector is the first row. And similarly, if I take determinant of the matrix T, A, T, B, C, D. This is equal to T times the determinant of A, B, C, and T. So it's linear in the first row. But very important is that determinant of B plus C is not equal to determinant of B plus determinant of C in general. So here it is true. When B and C differ only in one row, then determinant of B plus C is equal to determinant of B plus determinant of C. But in general, it's not true. And further, determinant of, say, T times A is not equal to T times determinant of A. So only when you scale a single row that the determinant scales by the same factor T. Okay, so now we'll discuss several properties of the determinant. If two rows of A are equal, then what happens to the determinant? Zero. Zero. Exactly. The second property. So why is this true? It's because if I exchange a pair of rows, if two rows are equal, let me exchange those two rows. The matrix remains the same, but the determinant is supposed to have changed the sign. But the only number which, when you change its sign, remains the same number is zero. So that's why the determinant is zero. Subtracting a multiple, so if you do an elementary row operation, subtracting a multiple row from another row, it leaves the determinant unchanged. So subtracting or adding is the same thing. So multiple one row from another row leaves the determinant unchanged. Hello. Yes, please. Sir, in the previous section, you specifically mentioned about the fast row. It is linear with respect to fast row. What about other rows? They'll be same. So what do you think? Sir, I think same. And why do you think so? So all you do is, if you take a linear side, if some other row is a sum of two vectors, what you do is you exchange that with the first row. The determinant changes the sign. Now you apply this rule. Now you get two different matrices. Now you exchange the rows back. It changes sign again and goes back to the original determinant. Yeah, so minus minus will be plus again. So it will determine. Exactly. So that's because if I take like this, what I've done is I've subtracted L times the second row from the first row. Now I can use my rules, namely that if I look at a matrix where the first row is the sum of two terms, then I can split it as the sum of two determinants. So I've written the first determinant here, A, B, C, D. And the second determinant is like a determinant of minus L, C, minus L, D, C, D. But the first row is being scaled by a factor P, which is equal to minus L here. So I can pull that minus L out and write this as determinant of C, D, and C, D. And this determinant is zero because two rows are identical. And so then the determinant of the matrix remains unchanged if you subtract a multiple of the second row from the first row. See, I'm showing you this proof for the 2 cross 2 case. But please keep in mind that showing a proof for a 2 cross 2 case is not really a proof. I'm showing it for the 2 cross 2 case in this example because the same exact thing can be done for a 3 cross 3 or a 4 cross 4 or 8 cross 8, or any size, m cross m case. Okay, the same idea exactly holds. And so there is no harm in this particular case to consider a 2 cross 2 case and say what is happening. Hello, sir. Yeah. Sir, sign now determinant also get changed when we exchange the columns. Yes, and that will be obvious in a second because it turns out determinant of A is equal to the determinant of A transpose. And exchanging columns is the same as exchanging rows of A transpose. Okay, sir. And so this is obvious again because if A has a row of zeroes, all you have to do is to add one other row to this row of zeroes. And because adding a row or a multiple of a row to another row leaves the determinant unchanged, the determinant does not change. But now you have a matrix with two identical rows. Then by property one, the determinant of A is zero. The other way to do is to think of it is to think of the cofactor expansion formula where you're expanding along that row, which is the row of zeroes. And if you're expanding along the row of zeroes, then the coefficient of every term in the cofactor expansion is zero. So it can only add up to zero. This is a property I already said, which is if A is triangular, then determinant of A, A11, A22 up to AMM. Okay, so basically for triangular matrices, the determinant can be obtained by just taking the product of the diagonal entries. So in particular, remember that you get the row reduced echelon form by doing elementary row operations. And these elementary row operations involve either subtracting a multiple of a given row from another row or exchanging rows. So if you exchange rows, the sign of the determinant will change. If you subtract a multiple of a row from another row, then the determinant will remain unchanged. And so when you do these elementary row operations, you will get a matrix which is ultimately upper triangular. And for upper triangular matrices, the determinant is the product of the diagonal elements. And therefore, one way to compute the determinant of a matrix is to first find the row reduced echelon form. And then take the product of the diagonal elements. And then you also keep track of how many row exchanges you had to do, why in the process of finding the row reduced echelon form. If you had to do an even number of row exchanges, then the determinant is simply the product of the pivot elements. If you had to do an odd number of row exchanges, then the determinant is the negative of the product of the pivot elements. So basically, this implies determinant is of the pivot elements in the row reduced echelon form. And plus, if even number of row exchanges might change, minus odd. So basically, maybe to just write this determinant of, in the case of a 2 cross 2, you can see it immediately, A, B, 0, D. Can you please explain the point number 3 using properties 1 and 2? So if A has a row of zeros, then what you have to do is to add one of the other rows to the zero row. That doesn't change the determinant. Then now you have a matrix where two rows are identical. And by property 1, determinant of A is 0. Okay, now for this property to see that if A is triangular, then determinant of A is 0, then the one way to see that is suppose the diagonal entries are non-zero. Okay, then you have a matrix. So I'll just show this in a picture to give you the idea. But actually, I'll just write this in the case as a sort of example to give you the idea. But the arguments hold in the general case. So I have A11, A12, say upper triangular, A13, 0, A22, A23, and 00, A33. Then what I can do is I can subtract a multiple of the third row from the second row, such that this entry A23 becomes 0. So I'll just write that here. So I'll replace R2 with R2 minus A23 over A33 times R3. Then this gives me a matrix A11, A12, A13, 0. Now A22 will remain unchanged. And this becomes 0, 00, A33. Then I can do something similar like replace R1 with R1 minus A13 over A33 times R3. And then I can again replace R1 with R1 minus A12 over A22 times R2. Then what happens is both these terms get eliminated and you will be left with A11, 00, 0, A22, 00, A33. So basically, if it's an upper triangular matrix or even a lower triangular matrix with the diagonal entries being non-zero, then you can do one more round of elimination and reduce it down to a diagonal matrix. And in this process, I only subtracted multiples of a row from another row. So that doesn't change the determinant. I've not changed the determinant throughout this process. So the determinant of this matrix here is the same as this determinant of a diagonal matrix. But for a diagonal matrix, the determinant is very easy to find. What I do is I first recognize that I can take out A11 from this because it's linearly dependent on the first row. It's linearly dependent on A11. So if I take this matrix and let me call it D, then determinant of D is equal to A11 times the determinant of this matrix 100, 0, A22, 0, 00, A33. Then what I can do in this matrix is I can exchange the second row and the first row. And then I know that the determinant has changed sign, but I'll keep that in mind. Now in the first row, it's linear in A22. So I can pull out an A22. And then I can once again exchange rows 1 and 2. So the determinant will once again change sign. It'll be back to the same old sign as here. So I can write this as A11, A22, determinant of 100, 010, 00, A33. And then I can pull out A33 and write this as equal to A11, A22, A33 times the determinant of the identity matrix, which is equal to A11, A22, A33. So it's the product of these diagonal entries. So suppose the diagonal entries are non-zero, then I can use elimination. So that means that the determinant remains unchanged. Also, the diagonal entries remain unchanged. And thirdly, the off-diagonal entries are eliminated. So now we are left with a diagonal matrix. So by properties 2 and 3, determinant of, okay, let me write it a little more systematically. If D is D110, D22, DMM, then determinant of D is equal to D11, D22, DMM. Therefore, is the product I equal to 1 to M, AII, for triangular. Of course, if the diagonal entry is 0, what happens? If you do this elimination step, you will get an all-zero row, which means that determinant is 0. So let's continue. So the next property is that determinant actually tells you, if you can evaluate the determinant, it tells you whether the matrix is singular or non-singular. So if A is singular, so basically what is a singular matrix? A singular matrix is a matrix which is not invertible. So if A is singular, then determinant of A is 0. A is invertible, determinant of A is non-zero. Okay, so basically the idea is that if A is singular, then it has a rank which is less than M, which means that it has dependent rows. And so we will study this later, but basically you can compute what is known as an LUD composition of a matrix where L is a lower triangular matrix. And it is the row reduction matrix, which has once along its diagonal and U is an upper triangular matrix. So basically I just make some small note on this. If A equals LU where L is lower triangular with once on the diagonal and U is upper triangular, then L basically represents the row reduction matrices. L is the product of the row reduction matrices, which does not change the determinant. Sir, should U also have all the diagonal elements as one? No. So U has the pivot elements along the diagonal, but there is another decomposition which we will study. I'm not going into the detail here because we're going to study these decompositions of matrices separately later on. The other way to think about it is you can also do something like, and also I've not considered the case where you are using, you may have to do row exchanges in order to find this LU decomposition. The more general thing is there is actually PA equals LDU decomposition where L and U are lower triangular and upper triangular with once on the diagonal and D is a diagonal matrix and P is a permutation matrix. Okay, this is the more general form, but for now I'm just giving you a little feel for why it is true that determinant of A equal to 0 for a singular matrix. So if you consider the process of doing the row reduction, the sequence of steps involved in the row reduction can be consolidated. Each row reduction step is actually a linear transform. So what you're doing is a sequence of linear transforms and the effect of all those linear transforms is the product of all those linear transformation matrices. And that is a matrix L which has once on the diagonal and it's a lower triangular matrix. So basically since each step involved is a lower, is a row reduction step, it does not change the determinant. And as a consequence, determinant of A is actually equal to the determinant of U which will be 0 if A is singular since has dependent rows. Okay, so if A has dependent rows, then U has an all-zero row. So if this is true, let me put it this way, U has an all-zero row. Okay, so then determinant is 0, but if A is non-singular, then you will have the pivots say I can call them D1 to Dm on the diagonal of U. So determinant of U will be equal to D1, D2 up to Dm which is equal to determinant of U. So I want to be a little careful here and so I'm going to write it in this way plus or minus this is equal to the determinant of A. In the sense that when I started out here, I assumed that there is no row exchange operation that was performed. So determinant of A is either determinant of U or minus determinant of U depending on row exchanges. Sir. Yeah. Sir, when you call this pivot element, so just another name for diagonal elements. Yes. Okay. So in the context of matrix factorization, the factors that come out when you do the row reduction operation, the diagonal entries of the reduced matrix are called pivot elements. So maybe one way to think about it is that I can ask what are the diagonal elements of any matrix, but these diagonal elements may not be equal to the pivot elements. Okay, but the diagonal elements will be equal to the pivot elements for triangular matrices. For non-triangle matrices, the diagonal elements will not be equal to the pivot elements. The pivot elements are obtained by taking the diagonal entries of the row reduced matrix. Does that answer your question? Yes, sir. Yes, sir. It makes sense. Thank you, sir. Property 6 is very important and also a property that is again to me really not obvious, but the determinant of the product of two matrices is equal to what? Product of determinants. Yeah. How does one show that determinant of AB equals determinant of A times determinant of B? Before I discuss that, one immediate consequence is that if AB is the identity matrix, then B is the inverse of A, then the determinant of the identity matrix is 1. Therefore, we can say that determinant of the inverse of A when A is invertible is equal to 1 over the determinant of A. So determinant of AB is 1, which is because AB is the identity matrix. And on the right-hand side, I have determinant of A times determinant of A inverse. So the determinant of A inverse is 1 over the determinant of A. I will give you one proof here. There is another proof in the textbook, which you can potentially look up on your own. So one simple proof is like this. If I consider a function D of A, which I define to be equal to determinant of AB divided by determinant of B. Now, I am assuming here that B is non-singular, so that its determinant is non-zero. But that is okay because if B were singular, then we know that the rank of a matrix cannot increase when you multiply it by another matrix. So if B is singular, then AB or if B is rank deficient, then AB is also rank deficient. And therefore, determinant of AB will be 0. And the right side is all equal to 0 because it's determinant of A times determinant of B, which is equal to 0. And so if determinant of B is 0, then the result determinant of AB equals determinant of A times determinant of B is obvious. So we'll consider the case where determinant of B is not equal to 0 and consider this function D of A defined to be determinant of AB divided by determinant of B. Now, if I look at the alternative definition of the determinant, so this is now a function of A, it's mapping A to a number, it's mapping it to this particular number, determinant of AB divided by determinant of B. Now it has the following properties, 1, D of the identity matrix. If I replace A with the identity matrix, I have determinant of B divided by determinant of B equal to 1, D of A changes sign to rows of A are exchanged. That's because if I exchange two rows of A, then I also end up exchanging the same two rows of AB because exchanging rows can be thought of as left multiplication by a permutation matrix. And so exchanging rows of A is the same as in the effect of exchanging a pair of rows of A on the product AB is the same as exchanging those two rows of AB. And so D of A changes sign when two rows of A are exchanged and 3D of A depends linearly on the first row. So it satisfies the alternative definition. So this you just think through a little bit, you'll see it's true. It's because this determinant AB here depends linearly on the first row of A. So basically this means that it satisfies the three properties, say the alternative definition of determinant and so which implies D of A is nothing but determinant of A. And so determinant of A equals determinant of AB divided by determinant of B meaning that determinant of AB equals determinant of A times determinant of B.