 Welcome back to our lecture series linear algebra done openly. As usual, I'll be your professor today, Dr. Andrew Missildine. This is our first video for section 3.5 entitled Matrix Factorizations. This is going to be continuation of the elementary factorizations we had presented in the previous section. Now, why do we care so much about how to factor matrices? In algebra, it's often important to undo the process of multiplication. That is factorization. We see this in arithmetic all the time. So like if you give me the number 6, you might want to factor it as 2 times 3. There's information we gain by thinking of the factorization as integer. We do the same thing with polynomials. If you take, for example, the polynomial x squared minus 1, this factors as x minus 1 and x plus 1. We might use this factorization to help us solve the equation or it tells us something about the polynomial when we can factor it. Factorization can be extremely useful in solving problems that arise with such objects like with, for example, with the equation x squared minus 1 equals 0. By the 0 product property, we can infer that either x equals 1 equals 0, which would imply that x equals 1. Or we see that x plus 1 equals 0, which would imply that x is equal to negative 1. We can gain information from this factorization and find the solution set to this equation x squared minus 1 equals 0. We can find out the solution sets negative 1 and 1, something like that. Such factorizations for matrices can be equally useful. Although no equivalent idea of prime factorization or maybe irreducible factorization is going to exist for matrices, the idea of unique factorization doesn't quite work the same way. There are still many, many, many useful factorizations for matrices, much like the elementary factorization we saw in the previous section. In this section, we want to talk about generalizations of elementary matrices. And that's actually what the topic of this very video is about. The generalization of the elementary matrices or the elementary matrices 2.0 is going to actually lead very naturally to the LU factorization, which we'll see later on in this lecture. So when it came to the three elementary matrices, which came from the three elementary row operations, there was three kinds. One of them was what we call a scaling elementary matrix, the scaling type. This had to do with when you multiply a row by a specific non-zero scalar. That's equivalent to multiplying by the matrix for which you have ones along the diagonal just like the identity matrix. Maybe you have some constant C somewhere in the middle. You have ones everywhere else and you have these zeros everywhere else. This idea of a scaling matrix, it looks like the identity matrix, but one of the ones along the diagonal was replaced with something else. Maybe a two, maybe a three. Now we're also going to relax the condition a little bit in this section and allow that to scale by zero. Now that's not properly an elementary matrix because multiplying by zero is not an acceptable elementary row operation because it can't be undone. It's not invertible. We're going to allow it for this situation. We could scale a row by zero. Well, if we want to generalize this, why do we have to require only one non-unital value along the diagonals? What if we allow any of the numbers along the diagonals to potentially be any number we want? It could be one, it could be two, it could be the square root of pi, it could be zero, whatever. And this then leads to the idea of a diagonal matrix. A diagonal matrix will be a square matrix, say it's in by in, for which every number that's not on the main diagonal will be zero. In particular, we have these numbers along the main diagonal, we'll call it D1, D2, D3, all the way to Dn. So we're going to have n many numbers right here. But everyone else off of the diagonal, these are all going to be zeros. All of these numbers right here, these are all going to be zeros. And sometimes you see me draw a giant zero to represent that all of these numbers are going to be zeros. It's a very common notation when you see in linear algebra here. Now, that's not to say that the numbers on the diagonal are non-zero. Like I said, for a diagonal matrix, we do allow for zeros along the diagonal. So an example of such a thing, we'll see some more just in a second. We would allow something like one, zero, two, and then zero is everywhere else. This is an example of a diagonal matrix. We allow any number along the diagonal we want, just as an example of such a thing. Now, the identity matrix is an example of a diagonal matrix. All the entries along the diagonal are just a one. The zero matrix is also an example of a diagonal matrix. Like if we take the three by three zero matrix, this would be considered a diagonal matrix because the definition of a diagonal matrix means that everything off of the diagonal is zero. It makes no stipulations on what is on the main diagonal. So the zero matrix would be an acceptable example. In a diagonal matrix, the diagonal entries need not be the same. They can be anything they want. Like the zero and identities, those are ones where that diagonal entries are all the same, but we could get anything. Like we saw the one, zero, two matrix a moment ago, right? Sometimes people use the notation, something like diag, and then they'll have a list of numbers like one, zero, two. And this is sometimes shorthand for, oh, we're going to have the diagonal matrix. We have one, zero, two along the diagonals and everywhere else because to describe a diagonal matrix, we just need to know who are the diagonal entries. That's all there is to it. In general, an in by in diagonal matrix D is of the form displayed right here. You can see, you can see here on the screen and every diagonal matrix can then be factored as a product of elementary matrices of scaling type. This is again, if we allow the idea of a zero scaling. So like if you take the matrix one, one, zero along the diagonal zeros everywhere else, this will be a matrix for which you would scale the third row by zero. Again, that's not an elementary matrix. But if we allow that slight singular example of a scaling matrix to come out, then every diagonal matrix can be factored uniquely, and I should say uniquely because again, the unique factorization doesn't exactly exist, but we can factor every diagonal matrix as a product of these scaling elementary matrices. And in fact, all as scaling elementary matrices can have at most one non-unital diagonal entry, we can view diagonal matrices as their generalization. So these are scaling elementary matrices 2.0, right? We just kind of squish them all together. Now similar to how scaling matrices multiply, we can see that if A is a matrix. So you have some matrix A. And if you calculated D times A, what this is going to do is this is going to be the matrix where the ith row of A is in fact multiplied by this factor Di. So you're going to multiply the ith row of A by Di. So diagonal matrices have the nice clean fact that if you multiply on the left by diagonal matrix, it'll just scale every row by that value. And if you go on the other side, if you take A times D, what's going to happen here is that the ith column. Oh, I don't like using I to describe columns. Let's switch it up and call it J. That's our usual convention. A, D here, the ith column of A here is going to be multiplied by this factor D, J. So multiplying a diagonal matrix on the left scales all the rows, multiplying by a diagonal matrix on the right scales all the columns. That's what I'm trying to say right here. So in particular, a product of two diagonal matrices is also going to be a diagonal matrix whose diagonal entries are simply going to be the products of the corresponding diagonal entries. So this is some important thing about diagonal matrices that if you have two diagonal matrices of the same size, when you add them together, you're going to get a diagonal matrix again. If you scale a diagonal matrix, you're going to get a diagonal matrix. So the span, the linear combinations of diagonal matrices will be diagonal. And so we could talk about the space, the space of diagonal matrices, space of diagonal matrices. This would be a subspace of f to the n by n. And in fact, this is going to be an n dimensional subspace. Because basically you can choose freely any of the numbers along the diagonal. And now give you a basis for this space here. You're going to take the first one, E11, then you take as a second basis elements, E22, and you continue all the way down to E, n, n. And that's what we meant by the basis of such a vector space. Now, this is sort of a curious object, though, that in addition to being closed under scalar multiplication and addition of matrices, this is also closed under multiplication. That's what we meant by the product of diagonal matrices is, in fact, going to be a diagonal matrix. Now, furthermore, if no diagonal entry is zero, a diagonal matrix, it will be a product of elementary matrices, like I mentioned before, it's of the scaling type. And hence, by the non-singular matrix theorem, the sum, it will be non-singular. That's what I was trying to say, sorry. So that if there's no zeros along the diagonals, this thing will be non-singular. And therefore, its inverse will be given by the following formula. You just take the reciprocals of each of the diagonal entries, and that will then form the inverse matrix if you're non-singular. So a diagonal matrix will be non-singular if and only if it has non-zero entries along the diagonal. Let's look at some specific examples here. Some of these we've already alluded to here. Take the matrix A, which is 3 by 3, which is a diagonal matrix. It has a negative one-half, one and three along its diagonal. Well, we can see very quickly that this matrix will be non-singular because the diagonal entries are not zero. You have negative one-half, one and three. And so by taking reciprocals, the inverse of this matrix will be very easy to compute. You're just going to get negative two, the reciprocal of negative one-half. You'll get one, the reciprocal of one, and then you'll get a one-third instead of a three. Powers of this matrix are also pretty easy because when you multiply diagonal matrices together, you just multiply together the diagonal entries. So A to the fourth just means you're going to take negative one-half to the fourth, which is one-sixteenth. You're going to take one to the fourth, which is just one. And you're going to take three to the fourth, which is just 81. A to the negative fourth is just as easy. You just take the reciprocals of all these numbers here. So you get sixteen one and one over 81. Let's consider a factorization of this thing here. That our matrix A, because you have these two non-unital values, that is these values that are not one, anything that's on the diagonal to one, I don't have to worry about the factorization here. But because you have these non-unital values along the diagonal, you can factor A into a product of elementary matrices using that. We're going to get an elementary matrix, which is associated to scaling the first row by negative one-half. And we're going to get an elementary matrix associated to scaling the third row by three, in which case you do something like this. And if you scale by zero, you're going to throw in a matrix, which is zero along the diagonal somewhere along the way. And then also, just as a quick example here, if we take this matrix A and we multiply it by, say, the matrix one, two, three, four, five, six, seven, eight, nine, eight, nine, we can very quickly do this product here because multiplying on the left by A, you're going to scale the first row by negative one-half. So you get negative one-half, negative one, and negative three-halves. You're going to scale the second row by one, which does nothing. And you're going to scale the third row by three, which gives us 21, 24, and 27. That would be the matrix product there. If we did it the other way around, if we take one, two, three, four, five, six, seven, eight, nine, and multiply by our diagonal matrix A in this situation, we're going to scale each of the rows and such by these values. So the first row gets scaled by negative one-half. So you get negative one-half, negative two, and negative seven-halves. The second column, excuse me, you're going to times by one. So two, five, and eight. And then you're going to scale the third row by three, which is going to give us nine, 18, and 27 again. And so we can see that these things are not true in general. And they're not, I should say they're not equal. Matrices don't typically commute. But the diagonal matrix is pretty easy to do multiplication by. You just scale, you scale rows if you multiply on the left and you scale columns if you multiply on the right. Now I mentioned that matrix multiplication is in general non-commutative. There is a special family of diagonal matrices for which matrix multiplication is going to be commutative. And these are called scalar matrices, not to be confused with the scalene elementary matrices we were talking about. A scalar matrix is a matrix which just has the same diagonal entry, right? So it's a diagonal matrix, we have the exact same number along the diagonal with no exceptions to that. Be aware that if you have such a scalar matrix, this will just look like C times the identity matrix because you could factor the C away. And so a scalar matrix is really just a scalar multiple of the identity matrix. Let's call this C for a moment, capital C. And if you have any matrix times that by A, what happens here is that, well, capital C is just the same thing as lower C times the identity. And by associativity here, we'll get the identity times A, which is just A. And so multiplying by a scalar matrix is the same thing as multiplying by a scalar itself. And that's why we actually call these things scalar matrices that multiply by a scalar matrix is just the same thing as scalar multiplication. We can think of matrix multiplication as a generalization of scalar multiplication, at least in the context of matrices. And I want to mention here that since C A is the same thing, this is going to be the same thing as A times C I, right? This is a property of matrix multiplication. This is going to equals AC. Scalar matrices are exactly those matrices that commute with all matrices whatsoever in terms of matrix multiplication. That's not true for diagonal matrices in general, but scalar matrices do, in fact, commute with everything. And so this is our first elementary row, elementary matrix 2.0. The scaling elementary matrices can upgrade to this general family of diagonal matrices.