 This video corresponds to lecture one in Trivenin-Bau on matrix vector multiplication. Of course, to do numerical linear algebra, we have to do linear algebra, and the basis of linear algebra is matrix vector multiplication. Usually written as AX equals B. Now, to actually compute AX equals B, let's assume that A is an M by N matrix, meaning, of course, that it has M rows. And N columns. So, of course, this is an operation we can easily do. For example, this is a 2 by 3 matrix. If we have a 3 vector, X, we expect to be able to compute B, a 2 vector, as 4, 0, 14 is 18, and 3, 0, 8 is 11. Of course, AX equals B is something we can compute easily by hand for small examples. But if we want to program a computer, we have to really understand what's happening piece by piece. So, let's suppose that, since A is an M by N matrix, that it has entries like this. A11 up to A1, let's see. So, it has M rows and N columns. So, this must be A1N. And then A21 up to A2N, and so on to AM1 up to AMN. So, the first index tells you which row the element is in, and the second index tells you which column the element is in. So, a generic element here is Aij. Therefore, B would take the form B1 up to BM, and X would take the form X1 up to XN. And we can compute the elements of B in a straightforward way by writing down the dot product. So, Bi is, well, I want to take, say, the ith row of the matrix A and dot it with the vector X. So, I have to count from 1 up to N and take, in the ith row, take that jth entry and multiply it by the jth entry of X and add them up. So, of course, you've seen this formula many times and applied it countless times with hand arithmetic. One thing I want to point out that you might not have thought about is how difficult this process actually is in terms of number of operations. So, I have to multiply, for each i, I have to multiply Aij by Xj, and j runs from 1 to N. So, this operation for a single Bi requires N multiplications. And then we have to add those up. Now, it looks like it's another N, but actually it's N minus 1 additions. Because, of course, to add together two numbers, I need to add once. To add together three numbers, I need two plus signs and so on. So, to add together N numbers here, I need N minus 1 additions. So, overall, I have to do two N minus 1 steps of arithmetic to compute each Bi. And, of course, then I have to do that for each i from 1 to M. As you know from linear algebra, when we write AX equals B, and we think about X and B as being vectors, we often think about A as being a transformation or a function or a map whose input is X and its output is B. So, we think about A as being the operation, X as being the input, and B as being the output. But there's another way of thinking about this, which is often very beneficial geometrically and computationally. So, let's think about A not so much as being a function comprised of coefficients, but rather as a list of vectors. So, the first column of A is A1, the second column of A is A2, and so on, the nth column of A is An. Each of these vectors, each Ai vector is in either RM or CM, depending if we're using real or complex numbers. And that means it's the same length as the vector B. It lives in the same space as B. So, one way we can write this is when we write, say, the vector B, well, let's see here. So, I can write this as A1, A2 dot dot dot An, and then times the vector of coefficients, X1, X2, up to Xn. And if you think about how matrix multiplication works, I'm taking each of these objects and scaling it by these numbers. So, I can rewrite this in the following way. This is the number X1 times the vector A1. And this is the number X2 times the vector A2, and so on, Xn times the vector An. So, in this sense, I'm thinking of not of A as being the function and X as being the input, but rather as X as being the function, or at least some sort of operation, acting on a list of inputs A1 through An. So, in other words, we are constructing the vector B out of the columns of the matrix A by choosing the coefficients X. The reason for this level of abstraction is that we can really think about the idea of the image or the range, or, as we will call it, the column space of the matrix A. So, let's define the range of A to be all vectors B that can be obtained as B equals AX for some X. By the previous discussion about working on the columns of A, this is the same thing as all vectors B that occur as linear combinations of the columns of the matrix A. So, this is why the idea of an image or range and column space are the same thing, because the operation of matrix multiplication, thinking of an input X being mapped to an output B, is actually the same thing as asking, if I choose different coefficients X to build a linear combination of the columns of the matrix, what can I possibly get? For example, in the previous 2 by 3 matrix, we might ask, what is the range of the matrix A? Well, it's clearly a subspace of R2, or, if we're working over the complex numbers over C2, because the output is a 2 vector, so it's some space in R2. We know it's a subspace because it's the span of something, it's the span of the columns of A. So, by definition, the range or the column space is the span of the columns of A. So, in this case, the range of A is the span, and I'll use an angle bracket like this to denote a span of some vectors, of 4, 3, 2, 1, and 7, 4. Now, in fact, in this case, you can see just by inspection and your own experience with row operations, that, say, the first 2 vectors, 4, 3, and 2, 1, are already independent. If you row reduce this, you can get on to Andy, 1, 0, 0, 1. So, that tells me that these 2 vectors are independent, and they span all of R2 already. So, in fact, this span of these 3 vectors, the third one, say, is not necessary. The first 2 already span the entire space. The span is all of R2. The other key idea for a matrix A is the idea of a null space or kernel. So, the kernel of A are the inputs X, such that AX equals 0. So, often we think about this as what are the inputs that give the output 0, if I think about A as being a function or being a map. But, based on the discussion of columns in the previous slides, we can also say this as these are the, so the kernel of A are the coefficients, X1 through Xn, that build a linear combination of the columns of A to make 0. In other words, how many different ways, and what are the ways, I can combine the columns of A to make the 0 vector. Of course, something you know is that if there is only one way, namely X1 through Xn equals all 0s, we say that A is, what's the word, non-singular. If it's a square matrix, or more generally we just say injective for a non-square matrix. So, that comment about the column space or the range, and the kernel or the null space of a matrix, that leads us to the fundamental theorem of linear algebra about the rank and the nullity of a matrix, and then about invertibility and all the different properties of matrices. Before we do that though, I want to demonstrate how to do these computations on a computer, simply matrix multiplication, and really how we represent matrices in a computer, especially using Python. So, first of all, mathematically, we have a matrix A, which has entries A11, A12, up to A1n, and that's a row, right? And we read the left as a human, we usually read it left to right, and then go down to the second row, A21, A22, up to A2n as a row, and then we continue up to Amn. So, we need a way to represent this on a computer, and one thing about computer memory is it's essentially linear, and so we need a way to break this up into reasonable pieces. So, for example, if I say that A is the matrix 427314, well, of course, the way I read that off verbally was 427314. So, in a computer, we want to somehow have a way of representing these numbers, and we want to then tell a computer that the first three numbers make a row, and the second three numbers make a row, and you want to put those rows in that order to make a matrix. So, in the next quick video, I'll do this in the sage, or in Python rather, and using the sage math cell to demonstrate how we use NumPy, the Python package for fast array manipulation, to work with matrices and vectors.