 under this broad title, I am going to look at first of all we define formally what is the vector space. Then we move to norm linear spaces and also what are called as 1x spaces. Then we have inner product spaces and we will also touch upon Hilbert spaces. So, these are made up of some of the famous mathematicians and then we look at Gram-Schmidt process in the end position. So, these are the four sub areas which I need to cover. Now, Gram-Schmidt process has applied to vector space. Now, what is the vector space? What did we know about vector spaces? When we start thinking about vector spaces from what background we have from undergraduate, we normally imagine a vector in a three-dimensional space in a vector say x, let us call this vector v, it has three components x, y and z and this is how we normally imagine a vector space. Now, what is that functional analysis is to distill essential properties of this vector space and then come up with a new definition called vector space which is more generally which can be applied to any set of objects that are relevant to us when we do computational or analytical mathematics. Now, actually the other course that we are doing in that also in the beginning there will be some introduction to these vector spaces. So, now what is it need to generalize and why we need to generalize? First of all, we can look at this vector space or set of all vectors in this vector space. So, if I let set as a set on which certain operations can be done. What are these operations? Addition, you can add two vectors and get a vector. The nice thing is that you get a vector in the same space and you can multiply a vector by a scalar and you get another vector in the same space. So, these are the two generic properties of any two vectors in this space and I could use this to generalize if I generalize the notion of a vector space. It is not enough to just generalize this notion of a vector space, we also need something more to work with vector spaces. We need to know about the length of a vector because that is a critical thing that we use when we actually work with vectors in three dimensions. So, we need to have a generalization of that which is called as norm of a vector. We will talk about norms of a vector. It is not enough to just talk about norm. We have notions of a sequence in one dimension. So, a sequence which is converging to a number to something called limit. So, what is a limit in three dimensions? So, we need to actually generalize the concept of convergence and limit. There are some funny things that happen when you start working with higher dimensions spaces and that is where we had mentioned is one-axis spaces and inverse spaces. So, these are some special category of spaces which we are going to look at. Now, it is not enough just to work with norm and convergence meaning something more. What else you use in three dimensions? What is the important geometric concept that you need? Coordinate. I will not coordinate. Coordinate is of course, when you define the space, many lines coordinate system will come together. It is not a coordinate. It is angle between two vectors. It is a very, very important concept. Well, we in three dimensions, everything come together package. We don't really think of these things separately. But when you generalize this concept to any other space, we need to make efforts to define what is angle between two vectors. And also, one of the most important concept that you use in three dimensions is orthogonal vectors, 90 degrees, two vectors and then well of course, the Pythagoras theorem which is used in many, many ways. So, what we are going to look at initially is it possible to generalize this concept and develop some notions of vector spaces on generic sets which are useful in mathematical analysis. Why am I doing all this in the beginning of a course which is supposed to be computational methods and you would be starting with development of recipes. Well, if you understand these grand generalizations which were probably done in the beginning of 19th century, beginning of 18th century, then it becomes very easy to understand the foundations of different numerical methods that we are going to study. So, this six or seven lectures which might look disconnected in the beginning are actually deeply connected with what we are going to do later, ok. So, this forms the foundation and you will understand basis of many, many methods if you understand this concept of vector spaces, orthogonality and so on. Many of these things are unknowingly used when you do your undergraduate courses without you know being given a thorough explanation. Here, we will lay a systematic foundation vector spaces. Now, let us well soon I am going to talk about you know four dimensional and five dimensional, end dimensional spaces and then I will also move to something called infinite dimensional spaces and well it is not possible to visualize. In fact, it is not possible to visualize anything beyond three dimensions. You cannot visualize a four dimensional space on a five dimensional space and obviously not an infinite dimensional space. So, there is a word of caution before we move into this is that it is enough to know your geometry, school geometry well. It is enough to know your undergraduate three dimensional world well. If you understand the concepts in undergraduate three dimensional world or your school geometry it is you will understand everything I am doing, ok. It is only a it is only a matter of generalizing these concepts. Same concept which I have been using right since our eight standard we are going to generalize them into something very very elegant. So, now let us begin with let us begin with the concept of closure or closure of a particular set for an operation is defined as if you take any two elements from the set that x be a set that x is a set and there are any two elements say x and y belong to belong to set x then x operation y also belongs to the set x. Now, this operation here I have written as plus it could be any operation it could be multiplication it could be division, ok. So, even any set if you take any two elements of this set and if you perform an operation for example, multiplication, ok. And if the element that results after performing the operation also belongs to the set then it is called a closed set, ok. Set of integers is closed on addition. Let us not close not division. The next concept that is important is of field what is the field and division correct. So, field is a set of elements closed under. So, well the well known examples of field set of real numbers set of complex numbers and these are the two fields which are they are going to use. So, these as R, R will denote the set of real numbers and C will denote the set of complex numbers that is closed under two operations. A vector space is a set of objects addition. So, if I take any element x and y belonging to x then x plus y also belongs to x. But it is not enough to have just a set of objects we need something more to define a vector space we also need a field. For example, a field could be R. So, let us call the field an f. So, we need two things here a set of objects x and a field f. If I take any scalar alpha from f and any x belonging to x then alpha times x is called a scalar multiplication this also belongs to x. So, a vector space a concept which is generalization of three-dimensional vector space is nothing but a set of objects which actually satisfy these two operations or a set which is closed under these two operations given the field f. It depends upon the combination. So, these x and f they are a combination you cannot separate them you have to consider them together. But let me start generalizing and giving you examples of spaces which are my first example is going to be x corresponds to Rn and f corresponds to R. What is Rn? Rn is n double a vector which has n components. So, a general vector here x that belongs to x will be represented as it has n components x1, x2, xn it may have n components. Can you give me an example where you need such a thing? Let us say I am dealing with some chemical reactor and I decide to associate a vector space that defines different variables. So, here x1, x2 to xn could consist of for example, x1 could be temperature in the reactor, x2 could be pressure and x3 to xn could be different chemical species that are present inside the reactor. So, this is a vector that represents the state inside the reactor or let me take a distillation column. Distillation column will have different trays and on each tray you have temperature pressure composition. So, if there are 20 trays and it is a binary distillation column how many variables you expect to have? 20 temperatures, 20 pressures actually pressure to be varying across the bottom then compositions about 60 well. You can have correlations between y and x. So, there would be 80 elements in a vector or 60 elements in the vector that defines all the variables at an associated distillation column which has 20 trays binary distillation column and so on. So, I can think of examples from chemical engineering which would actually have a way to deal with vectors at a higher dimensional vectors. But an example that would tell you what is which combination will not be a vector space. See for example, if I take x to be rn and f to be c instead of complex numbers, will this form a vector space? Why? The scalar multiplication will break down. If I take a scalar which is complex multiply it to a vector which is real valued I will not get an element from x. So, this is not a vector space. But my next example is, my next example is a little unconventional. So, now I am going to move to a set of real valued matrices. So, this is my x, this corresponds to my x. And my f is still set of real numbers, set of real valued matrices. Does this form a vector space? Yes. Why? The additional two matrices. If you take any two n cross n matrices, if you add them you still get an n cross n matrix. If you take a scalar and multiply it to an n cross n matrix, you still get an n cross n matrix. So, if I take any two matrices say a and b which belong to f, then I can say that any ufa times a plus beta times b belongs to also x, sorry this is x here and alpha, beta belong to f. So, for any matrix x, for any matrix a and any matrix b which are both n cross n, so these are elements, these are vectors in this space, vector space. If I take a scalar multiplication of a and a scalar multiplication of b, then this sum should also belong to this space which is true for any n cross n, any n cross n vector. So, and my fourth example is something that we are going to use quite often in this course. So, my example number four is this is denoted as CAB. So, CAB is set of continuous functions, set of real valued continuous functions, set of all real valued continuous functions on interval AB. So, we have a function say f t which and a function g t both of which belong to x, then and if I take any if I take any two scalars say alpha and beta that belong to f, then alpha times f t plus beta times g of t also belongs to x. Now, real valued continuous, have you come across this kind of functions? Well, where did you study this kind of functions? Fourier series? Not Taylor series, Taylor series is not true. Fourier series. What happens in Fourier series? You talk about functions on minus 5 to pi or 0 to 2 pi. Remember something? rings a bell? A and B and I have no questions. Are we going to look at Fourier series much more in detail in the next few lectures? So, do you agree with me? If I take a continuous function, if I take a continuous function and multiply by a scalar, will it simply a continuous function? It will simply a continuous function, right? If I add two continuous functions, will the addition be continuous? f of t is continuous and g of t is continuous. Adding two continuous functions, I still get a continuous function. So, scalar multiplication, vector addition, both properties hold in this abstract set. It is difficult to visualize how this set looks like. You know, we are used to visualizing in three dimensions. Nevertheless, the properties that hold, the fundamental properties that hold in three dimensions also hold in this set. That is very, very important. Okay? So, if you have a vector, if you have a vector which belongs to this set, multiply it by a scalar, you get a vector in the same set. Very important. And if you have a vector which is, well, this is not enough to just define vector spaces. We also have to talk about subspace. So, very, very important concept in three dimensions. What is the subspace in three dimensions? What are the subspaces that we go into three dimensions? Line passing through the origin, yeah, correct? Then is the only subset? A plane passing through the origin. So, it is a plane not passing through the origin. We can be a subspace. Where does it come from? So, let's define what is a subspace. I want to define a subspace. So, this is, I have set of vector x and field f. And then let m be subset of x. m is some subset of x, a non-empty subset of x. And if I take any two elements in m, so x and y, they belong to m. Alpha, beta belong to f. Then alpha x plus beta y should belong to. The way we define subspace is a non-empty subset, a non-empty subset of original space x. Okay, like example that he gave just now. A line passing through origin or a plane passing through origin. Okay, but why was origin important? That origin is required in this space is hidden in this definition. Can you dig out? The word is that the main thing is any alpha, beta belong to f. So, what about 0, 0? If I choose alpha to be equal to 0 and beta to be equal to 0, then 0 times any vector plus 0 times any vector is giving me 0 vector. 0 vector should be contained in the space. Okay, so if I have a subspace, then it follows from this definition that the 0 vector, the origin should be contained in the space. So, only those sets, only those sets which contain the origin qualify to be subspaces. Okay, now let's understand this little more. If I take, okay, let me try to draw a subspace. Imagine that this square, the plane which passes through this, this is a plane that passes through the origin. Okay, now there are two situations, two scenarios. One is this is a finite, this is a finite set. It's only like a piece of paper. Okay, let's look at this piece of paper which is passing through the origin. It is finite size, it is passing through the origin. Will it form a subspace? Just because it passes through origin, will it form a subspace? Yes. What will it say? It should also have x plus y belonging to x. The closure is new. I can take two elements, this is passing through the origin. I can take two elements such that x plus y belongs to... But not for all x plus y. Correct. The word all is important. It should happen for every x plus y. Okay, it may happen that if I take a finite set like this and not the infinite set, it may happen that for two vectors, say this vector and this vector, the addition may not belong to this small set. Sir, why we are talking about zero into subspace? Isn't this included into the vector space? Of course. So why is it... But vector space's origin is included. It is obvious. The subspace is a small set. Subspace is a smaller set. So does every smaller set qualify to be a subspace? Is the question an honest? Okay. So what is, for example, what is zero element in this... What is zero element here? Constant function zero. Ft equal to zero over interval a to b. Okay. Here it is important to remember that t belongs to... A component. A component. Well, we get... Do we get these kind of functions? Is it capable of engineering? Think of temperature profile in a heater center. Okay. My A would be... My A would be zero to one. Well, it will not be time. t... Don't associate t with time. It could be space. So z is my spatial variable. Okay. It varies from zero to t. Okay. And I can look at the temperature profile inside... Inside a heater center. Is it a continuous function? Yes. It's a continuous function. It's a continuous function. These kind of vectors... These kind of vector spaces are very, very commonly encumbered in chemical elements. Of course, the zero... The zero element would be... Well, not zero temperature. We often talk about perturbations. And in steady state. If you have a steady state and a perturbation... The perturbation vector is zero, which means you are at steady state. So you will have... You will not have... In the case of a heater center example, the zero element technically would be... Everywhere you have zero temperature. Such a heater center doesn't exist. But the space... In the space, you can of course define zero vector, which is... So... So coming back to this subspaces, every subset doesn't qualify to be a subspace. Okay. This thing is important. If I take alpha, any scalar alpha, multiply it by... Multiply it to my vector. Then the resulting vector also should be included inside the space. So if I take this vector, and if I multiply it by a large scalar, a new vector might be here, which is not included in this small finite set. This is not a subspace. So just going through origin is not sufficient. Okay. You need to have closure. You need to have closure of this operation, alpha times x for any alpha, plus beta times y. This sum should be... So... Can I give you an example? I'll just give you an example, completely different, but generalize this concept. Let's look at our... Let's look at this space. Okay. Set of polynomials. Set of polynomials. Are they continuous functions? Right? Set of polynomials is a continuous function. Okay. Set of polynomials, let's say, defined over a to b, or 0 to 1, if you want to... If it's convenient for you to imagine, 0 to 1. So set of polynomials, defined over 0 to 1. What will be the set? What will be that set? So let me take this set. Let me take this set, S, or we call it m here. Let me take this set, m, which is 1, t, t squared up to t to the power s. Okay. Will this combination be also a polynomial? This is an untholdered polynomial. Okay. This is an untholdered polynomial. Now, when I'm visualizing each one of them as untholdered polynomial with some coefficient 0. Okay. So this particular, this particular set, set of all possible polynomials with any alpha 1 to alpha and from the field, this particular set will form a subspace of this vector space. Because these are continuous functions. These are continuous functions. And then if you take a polynomial, if you take a polynomial, a finite order polynomial, add to it another polynomial, you're going to get a finite order polynomial. Okay. So all those properties, all these properties will hold a set of polynomials and then you can show that this is a subspace, 0 element will be there, 0 function, 0 polynomial. Okay. So all the things that you need in this set. Okay. I'll give you another example of subspace. Can you think of a subspace for n cross n matrices? For example, let's take my set here. So you understand this if you understand these examples, because just writing this definition is too abstract unless you associate with some real examples. It's not possible to understand these concepts. So let my x be set of, my field is my field is r. Okay. I'm going to define a subset m, which is a subset of x, which is a non-empty subset of x. Now, m here, set of all symmetric matrices, is it a subspace? What is a subspace? If I take a scalar alpha, see when you call the matrix to be symmetric, A transpose is equal to, okay, just test this. You know, will alpha times A transpose, will it be equal to alpha times A? For any alpha. By the way, what is the zero element in this place? What is the zero vector? Null matrix. So if I set alpha to be zero, it's a symmetric matrix, right? It belongs to the subspace. What about alpha? If I take any two matrices, A and B belong into this subset m, okay? Will their linear combination, will their linear combination also belong to m? If this is symmetric and if this B is symmetric, will this addition also be symmetric? It's a symmetric matrix. So this subspace or the subset defined by this, defined by this is a set, belongs to subspace. Not every set will have a subspace. But this particular set of set of cemented matrices will form. Likewise, if you go to a set of complex valued matrices with field to be complex numbers, you can define with Hermitian matrices. Set of all Hermitian matrices will be a subspace of the set of complex valued matrices. Okay, so these are the general examples. What is the next thing that you need when you start looking at vector spaces? What is the thing that you use most? Well, one of the most important concepts that you use is basis and dimension. What is dimension of vector space? What is dimension? How do you define dimension? This is the common misconception. Okay. Let us look at this subspace. This is my this is a line passing through the origin and all of us agree that this is a subspace. Okay, this is a line passing through the origin. Any vector on this any vector on this line will be represented by three components. Does it mean it is it is a three dimensional subspace? What is the dimension? Why? Number of independent vectors are only one. So, just because a vector has n components does not mean that does not mean that you know the dimension of the vector of the vector space or a subspace is one. The dimension of this subspace is only one. Okay. And there is only one independent direction. Let us say you call this some vector x. So, any vector on this vector v. So, any vector on this line will be alpha times v. Okay. If alpha is minus it will go in this direction and alpha is plus it will go in this direction. So, but it is basically alpha times v. So, there is only one independent direction. Okay. So, likewise a plane cutting a plane passing through the origin. What is the dimension of the space? Because there are only two independent vectors. So, we need to now generalize this concept of dimension. Well, to generalize this concept we need to analyze many other concepts. We have to we have to have notion of linear combination defined after that. We will have to define what is called as a basis set and then move on to moving. Go through the nodes I have given many more examples of vector spaces. So, I am not as I said I am not going to write everything on to the board should look at the nodes. Now, if I am given a vector now I have to introduce one important notation here. Because we are going to work with set of vectors and each vector might be n double. Okay. Each vector might be n double. So, I have to introduce a new notation. Now, I am going to consider a set here a set x i where x I have some space x here and this x i these are vectors that belong to this set x where i goes from 1 to n. It is quite possible that my set is nothing but m. So, n doubles okay. So, an element here there are n vectors and each one of them is a m double. So, where I am going to define this is x i it corresponds to x i 1, x i 2, x i. So, this notation is centered to our course. I am going to use it very often that you have a set that. So, superscript in brackets is used to indicate i th vector okay and a subscript is used to indicate the component of the i th vector okay. So, x i 2 is second component of i th vector and so on. This notation is very, very often. Now, if I choose any set of scalars if I choose any scalars say alpha 1 alpha 2 alpha n okay. Then a vector which is defined by alpha 1 x 1 plus alpha 2 x 2 this vector is called a linear combination of this vector is called linear combination of this set of vectors. This is the set of vectors belonging to space x okay. Alpha 1 to alpha n are some scalars arbitrary scalars belonging to no field f. And then the vector that you get alpha 1 times x 1 plus alpha 2 times x 2 up to alpha n times x n this particular vector obviously we are dealing with vector space. So, or we are dealing with a subset which this linear combination if you should talk about a subset alone then it is a finite set and if you take all possible linear combinations of this these vectors they give a special subset that is called a span. You thought you could understand the equation sir. So, I will give you the utenance matrix. No. This is a vector this is i th vector okay. This is i th vector see for example I may have two vectors. Let me take a five dimensional space. So, I have vector 1 okay. 1, 2, 3, 4, 5 and vector 2 which is 5, 4, 3, 2, 1 okay. Now, how do I refer to third element of vector 2? So, I will say 3 that is equal to vector 2 third element okay. Similarly V2 5 will be. No, no, this is just a rotation just a rotation that which we are going to use very very often whenever you have subscript in brackets or superscript in brackets it implies that it is a i th vector okay and if I want to refer to the third component of i th vector then I will use x i z. It is not a matrix. It is a rotation. Now, this kind of things do appear in numerical methods in computations because we do iterative procedures okay. You start from one vector and then you get another vector and then you get another vector okay. So, you have a sequence of vectors and that is where you need to have this little complex rotation. So, sometimes we develop algorithms in which we need to worry about this superscript and then this subscript together okay. That is why we need to have this rotation okay. So, this span is set of all possible combinations understand? Set of all possible combinations. If I give you two vectors in three dimensions if I give you two vectors in two okay first start with two dimensions if I give you any two vectors let us call them V1 and V2. Are these two vectors in two dimensions? What is this set? So, span of V1 V2 this corresponds to alpha V1 plus beta V2 for any alpha beta that belong to F that belong to the field F what is this set? Field F here is real numbers. What is this set? It is a plane passing through original okay because two independent vectors two linearly independent vectors as we have defined what is linear independent vectors. So, two linearly independent vectors if I take all possible linear combination then what I get is the span and the span is nothing but well let us say that this is a subspace this is the subspace if I give you third vector in the same subspace okay which is V3 what will be alpha V1 plus beta V2 plus gamma V3 same okay that is because this third vector V3 is linearly dependent upon V1 and V2 and then you can what you get here so, if I have some more vectors belonging to the same set here okay same subspace and if I take all possible linear combinations I am not going to get a different set I am not going to get a different set okay I will get the same set which is this plane I cannot leave this plane if I take a linear combination of any two vectors in this plane I cannot leave this plane what is the minimum number of vectors that are required to generate this plane okay the minimum number of vectors that are required to generate a subspace or a space is called as the dimension of the vector space okay what is the dimension of this particular subspace this is a subspace right the span is a subspace what is the dimension dimension is to because you need only two linearly independent vectors you need only two linearly independent vectors to generate all vectors inside this plane okay this is a two-dimensional subspace of three dimensions this is a two-dimensional subspace of everyone with me on this okay now let us consider these two vectors what will be the span of these two vectors what dimensional plane is R5 two-dimensional plane is R5 okay if I take span of this alpha times 1,2,3,4 and 5 plus beta times 5,4,3,2,1 for any alpha beta it will be equals to R this all possible linear combinations of these two vectors is called as span of is called as span of these two vectors and this span will be nothing but a two-dimensional subspace a two-dimensional subspace of R5 what is R5 five double space considering of vectors each vector has five double has five components so number of components in a vector doesn't define the dimension okay this is a fifth five-dimensional just because it has five components doesn't mean this linear combination will specify if I take only one vector say V1 what will be alpha times V1 it's a line and it's one-dimensional subspace it's a one-dimensional subspace of R5 five-dimensional space okay if the one-dimensional subspace of well so far so good we'll probably define what is basis and move on to some more some more insights into why this is all required when do I need this to do our next lecture