 in last lecture we discussed about span of a set of vectors and then we graduated to basis. So a linearly independent set of vectors is called basis, if it generates the entire space a linearly independent set S that belongs to vector space X which generates the entire space that means the span of that set is equal to the entire space then it is called as a basis vector set it is called as a basis for that vector space. If the number of elements in the basis is finite then we have a finite dimensional vector space and if this linearly independent set is an infinite set then we have an infinite dimensional vector space so number of linearly independent vectors that generate the entire space that defines basis and if that number is finite we have a finite dimensional vector space if that number is infinite infinite dimensional vector space example would be set of continuous functions over 0 to 1 or set of continuous functions over 0 to 2 pi set of continuous functions over minus pi to pi all these all these vector spaces and we will encounter many such examples but these two three examples I gave you just now because we have seen something like this in some examples we have looked at a boundary value problem in which we had to look at set of continuous functions over 0 to 1 or set of continuous functions over 0 to 2 pi this is something which you have looked at when you studied Fourier series in your undergraduate or minus pi to pi right studied Fourier series and will be anyway revisiting Fourier series soon next week so we have these two concepts infinite dimensional space and then we will be using finite as well as infinite dimensional spaces throughout our study of numerical methods and this particular so this is not something which I am just introducing out of completion or something this is something which we are going to use and you will see when we start talking about transformations how these spaces come into play we need to introduce one more concept which is called as a product space okay so product space a product space is composed of two vector spaces okay I can create a new vector space I can create a new vector space by combining two vector spaces or more vector spaces okay that is called as a product space well the simplest example of product space is Rn where R is a vector space line one dimensional vector space okay line and I can create R2 by R cross R okay I can create R3 by R cross R cross R and so on so likewise I can create a vector space by merging two or more vector spaces those are called as product spaces so if I take two vector spaces X and Y both defined on field F okay then X cross Y what I mean here is this is X cross Y X cross Y this is also a vector space so if I take an element X belonging to vector space X if I take an element Y belonging to vector space Y then this combination XY it belongs to the product space okay as I said simplest example of this is R2, R3, Rn or C2, C3, Cn n dimensional vector spaces are the simplest example of this product spaces see the way you can go about thinking about it is that you start with R R cross R is a vector space then R2 cross R is a vector space okay I get R cube so R cube cross R is a vector space so I get R to the power 4 and so on R to the power not power 4 R4 and so on so my vector spaces could be compositions of multiple vector spaces but this I can extend to some other to create more complex spaces see for example my X my X can be set of real numbers and my Y my Y can be set of continuous functions over 0 to 1 my Y can be set of continuous functions over 0 to 1 I can create a vector space which is X cross Y okay which is I can create a vector space which is R cross X cross Y it is possible to have a product space which is defined like this okay we will hit into these kind of spaces when we start talking about boundary value problems and so on so examples of this will come in ample later when we when we have when we study about transformations of mathematical problems into computable forms so that is where we are going to need this so I am going to give you examples right now because many examples will come a little later but this is an important concept product spaces now having defined linear vector spaces vector spaces sometimes they are called as linear spaces sometimes in people refer to as linear vector spaces words are used interchangeably now here we need to define more structures and as I said the first thing that comes to your mind is length of a vector okay so I need to define what is length of a vector when I go to a set of continuous functions over some interval what is length of a vector now okay so we have to systematically define okay something equivalent to magnitude or length okay what we define here is a concept called norm okay now you can have a vector space defined in mathematics in function analysis you can have a vector space defined without definition of a norm or length on it just set of objects which satisfy certain criteria okay so what I want to say here is that norm is an additional structure that we are putting on the space so we have norm vector spaces or norm linear spaces as they are often referred to norm vector spaces is a vector space together with a definition of a norm okay a vector space well f here is the field associated with the vector space okay okay so norm vector space is nothing but a vector space together with a norm defined on it and the definition of norm and the definition of vector space it is not that given a vector space there is a unique way of defining norm we will see what is a norm and then you will understand why I am saying it is a pair okay so for example in three dimension itself we will define different norms and three dimensional space with one norm will give you one normed space three dimensional space with another norm will give you another normed space okay so it is not that the three dimension space if you are working three dimension means you have to use one particular norm nothing like that so a vector space together with a norm definition will give you a norm linear space and then it will have certain properties so what is this norm so norm is a generalization of concept of length of a vector so when I say length of a vector all of you think probably from your under graduate experience in one direction saying that well if I give you a vector in three dimensions with say three coordinates x, y, z then x square plus y square plus z square whole it is to half this is what we know as norm of the vector or length of a vector we do not know any other way of looking at it so a vector space together with a definition of norm defined on it forms a norm vector space so what is a norm norm is a real valued function norm is a real valued function in fact it is a function that is defined from zero to infinity it is not defined on the negative side of the real line it is a function defined from zero to infinity so given vector space or let us work with set of real numbers so given a vector space norm of a vector x that belongs to x norm of x is a function that is defined from x to r plus set of positive real numbers well length if you look at why I am saying this because length of a vector in three dimensions always positive it cannot be negative right so I need a generic function which exactly has similar properties as that of the concept of length in three dimensions okay do not forget that at any point that we are generalizing concept from three dimensions to look at the notions in higher dimensions so the first property that the norm function should satisfy is norm of x should be greater than or equal to zero for any x that belongs to x norm x equal to zero so this norm function should be always greater than zero it can be equal to zero only for one vector only for one vector that vector is the origin if it is zero for any other vector which is a non zero vector then that function cannot define a norm so the first criteria for a function to qualify as a norm is that it should be a positive function and then it should be zero only for the zero vector should be non zero for any non zero vector the second quality is if I multiply if I multiply the vector x by a scalar alpha then then what should happen norm x should be equal to mod alpha norm of alpha times x norm of alpha times x should be always equal to mod alpha times norm x for any scalar for any scalar alpha it is very very important second property and the third property of the length function in three dimensions is triangle inequality okay so this is a way of saying that if I am given a point the shortest distance to the point is the straight line okay so the third property is now triangle inequality is a result which probably you study in your undergraduate undergraduate you study in your high school okay the sum of length of two sides of a triangle is always greater than the third side this is this is just the same that result which is being generalized so if I take any two points in the space the shortest distance is shortest distance in the straight line connecting the two points any other way I try to go to that point in the vector space by addition of two vectors you know that will be the larger path that will not be so this is this is a very very fundamental property of the concept of length in the three dimensions which we are generalizing to any other vector space okay so now anything any any function that satisfies these properties would qualify to be a norm function will qualify to be a norm function see for example in three dimensions three dimensions well one way to define norm is very very common so so my first example is r3 x is x corresponds to r3 okay and norm x if there are three components x1 x2 x3 then x1 square plus x2 square plus x3 square whole is to half this is the definition of the norm in fact this will be called as two norm okay so x this space together with this definition of norm will give you one norm linear space but this is not the only way to define norm all these three properties are very obviously satisfied with this I do not have to even check well my second example my second example is x corresponds to r3 and then my norm definition is absolute of x1 plus absolute of x2 plus apps of x3 you can check you can check whether this qualifies to be a norm will it be 0 only when x1 is 0 x2 is 0 x3 is 0 and if any one of them is not 0 then the value will be greater than 0 so first property satisfied what about a second property if I multiply a vector by a scalar alpha the norm will get multiplied by alpha okay the third property follows also in a straightforward manner from the inequality that if I take any two scalars is always this then mod alpha plus mod beta this is a simple equality inequality for scalars if I if I look at each component as a scalar I can apply this inequality to each one of them it will follow that the third triangle inequality is also satisfied by this norm so this norm this this function this function which is from the space x to r plus it will always give me a positive value okay equally qualifies to be a norm of the of a vector in three dimensions I do not have to I do not have to always think in this term the third so what I wanted to realize is that this vector space and this vector space are two separate vector spaces two separate norm vector spaces because a normed space comes with a definition of norm on it so don't say that if I am working with r3 which means I have to have two norm okay I have I am working in three dimensional vector space and I have one norm this is called one norm okay so in general I can show that any function if x is r3 and I can define what is called as a p norm x1 to power p plus x2 to power p plus mod absolute of x3 to power p whole raised to 1 by p where p is a positive integer where p is a positive integer this also forms a vector space with p norm defined on it so this I am calling it xp so r3 I can extend these definitions of r3 to rn n dimensional vector space a two norm on n dimensional vector space will be x1 square plus x2 square plus x3 square up to xn square whole raised to half in general actually I can define this for any p which is an integer so one norm two norm are you know some special examples of a p norm vector space okay and I can very easily extend this definition to I have written here for r3 I can extend this to rn I can extend this to cn and so on okay but it is important to note that these three are three different norm vector spaces the definition of norm is different it is not same okay what about set of continuous functions how do we define norm on it well before I move to that let me give you one more example which is in line with the p norm the my fourth example is going to be x now instead of writing r3 I will move to rn and then I am going to define what is called as infinite norm okay so for any element x that belongs to x infinite norm is defined as max over i is equal to 1 2 to n mod xi infinite norm this is a n dimensional vector space if I give you a vector there are n components I find out absolute of each component and take max of that okay I find out absolute of each component and then take a max of that so there are n values absolute values and then find out the maximum you can show that this also forms a norm it satisfies all the three properties of a function to be a norm it will always be greater than 0 if x is not equal to 0 it will be equal to 0 only when x only when x is equal to 0 or the origin and then if you multiply a vector by a scalar the norm will get multiplied by mod of the scalar triangle inequality will be satisfied so some of these will be exercise problems that's why I'm not doing it on the board my fifth example is x corresponds to set of continuous functions over 0 to 1 and then I'm going to define of I'm going to define a norm for an element say function f t which belongs to x is norm of f t well let me let me extend norm of f t using the earlier notions so I could define I could define an infinite norm which is max over t belongs to 0 to 1 okay infinite norm of this function infinite norm of this function is maximum value of the function over the interval absolute of the maximum value absolute of the maximum value very very important why absolute is required because the norm has to be a positive number it can when will this be zero when will the maximum be zero absolute of the maximum when when you have zero function okay if I multiply a function f by a scalar alpha what will happen to the norm since it's mod absolute value of the scalar will be multiplying the mod right so this this all these see for example if I take a scalar alpha so what is what is absolute of alpha times f of t this will be mod alpha into f of t so if I put max operator that t belongs to 0 to 1 so which is same as all right so I start with I start with this and I show that it is mod alpha times mod alpha times f of t I can prove the third triangle inequality yep the norm is maximum value maximum absolute of the maximum value over interval 0 to 1 over interval 0 to 1 no for a function given a function if I give you a vector one vector okay you find you find you find norm of that vector no so if I give you for example sine t okay where t goes from 0 to 1 you are expected to find what is the norm of that find the maximum value of sine t over interval 0 to 1 find absolute of that that will give you norm of that okay so not over all functions it is over t the max is not over the functions max is over t okay so likewise if I take any two what about the third thing what about the triangle inequality what about triangle inequality if I give you two functions say f t and gt which belong to which belong to that x set of continuous functions over 0 to 1 then what about triangle inequality what we know that at a particular point if I fix t for a fix t if I fix t then then I get function values right then I get function values so these are real numbers we are talking about real valued functions by the way so for real numbers what I know is f t less for a fix value of t I can write this okay now I can use this inequality and prove the triangle inequality so now what I am going to do so since this holds for every t okay I can argue that max it holds for all t between 0 to 1 so I can write I can take a max operator here okay this is this is nothing but this is equal to norm of f t plus gt right this will be the infinite norm for f t plus gt you agree with me but using this inequality we can say that this is always less than or equal to max of it follows from this inequality it follows from this inequality that initially I am taking I am looking at these as two scalar numbers if you give me two real numbers this this inequality holds see if I take a function over a over an interval if I fix myself to one t I will get one scalar value right let us say this is this is sin t this is cos t okay if I say that my t is 0.5 I will get some specific value of sin and cos what we know from a very fundamental inequality is that if I add two real numbers okay then that sum will always less than or equal to mod of the sum of mod of two real numbers okay now now actually I want to look at max over all t so I am graduating from this inequality to this inequality okay but what is this this is norm f t infinity what is this okay so what I have proved is triangle inequality that if I take infinite norm of sum of two functions okay it is always less than or equal to this plus this this is norm of function f t this is an infinite norm of function gt is that okay everyone with me on this well I can define now norms which are similar to one norm or two norm or p norm on these spaces infinite norm is not the only way to go about defining norm on this space so my next example is it sixth or seventh what seventh example my sixth example is x corresponds to set of continuous functions over 0 to 1 and my norm definition so for any f t that belongs to x my norm of f t is p norm is integral 0 integral 0 to 1 I can define a norm which is integral 0 to 1 integral 0 to I need a I need a norm to be a scalar value right I need a norm to be a scalar value if you look at n dimensions if you look at n dimensions how was the norm defined some of absolute values right some of absolute values for one norm some of square absolute values and then takes square root for the two norm for a p norm it was defined you know absolute value of each element raise to p sum it and raise to 1 by p and so on okay so this definition should be logical extension because instead of sum I am getting integral because now t is varying continuously in n dimensional vector space the index I was finite 1 to n okay so we had a summation here we have an integral here we have an integral well p norms though they are defined we normally use only one norm two norm and infinite norm the three norms which are very very commonly used are one norm two norm and so so f t one norm would be integral 0 to 1 mod f t dt and two norm is integral 0 to 1 mod f t square dt whole raise to whole raise to half okay so this is this is this is extension of the ideas from three dimensions or from n dimensions to a set of continuous functions and infinite dimensional space now it is not necessary that this should be limit here should be between 0 to 1 I could actually define a set of continuous function over any interval say a to b any interval minus 5 to plus 10 whatever doesn't matter my integral here will change from a to b okay my integral here will change from a to b and so on okay but what you should realize is that this vector space together with a definition of a one specific norm will be one norm linear space one normed space okay so set of continuous functions with one norm is one norm linear space set of continuous function with two norm is another norm linear space they are not same definition of length is different okay how we measure length in one system is not the maybe measure length in the other system well there might be advantages of this norm over this norm or advantages of this norm over this infinite norm and indeed that's why we keep using so called two norm very very often that will become clear as we graduate to inner product spaces little later so why is this why is this guy so special why why we don't seem to use one norm or infinite norm so often as against this two norm will become clear nevertheless don't think that two norm is the only way to define norm there are many other ways of defining norm okay now to understand this concept of norm i think we should have one or two more examples and then things will become clear i have actually given here some arbitrary functions which could be used to define norm i'll just talk about one of them right now here and then we will move on to some other concepts which are important like convergence and i mean what is the use of defining norm the one of the use of defining norm is to talk about limit and convergence okay now what happens in iterative processes is that you have some idea about iterative processes like Newton-Raphson everyone i think knows about Newton-Raphson okay in Newton-Raphson what you do you start with a guess value and then you get another guess from the new case you construct third guess and fourth guess so you get what is called as a sequence of vectors okay the question that you need to answer when you solve a numerical problem is this is that is this sequence converging to something is it converging to my solution in fact that's what is in your mind but at least before whether to know whether it is going to the solution or not you would like to know whether it is converging so convergence is a very critical thing in numerical analysis and when you work with n dimensional spaces you need to generalize the concept of convergence we know about a sequence of real numbers converging to a point and we call it limit all those things we have done in 1200 right but now what is the meaning of a function sequence converging to a limit we will have to talk about these ideas okay so that's why we need this now just to give you just to give you a feeling that the norm can be defined in different ways I'll just take the same x which is set of once differentiable continuous function I'll take set of once differentiable continuous function okay and let me let me define a function you have to tell me whether it will define a norm or not so what about candidate function so I have this function f t that is equal to that belongs to this x so this is a once differentiable continuous function okay now I am defining a norm of f t as max over t belongs to 0 to 1 mod of df by dt I am taking derivative of the function okay df by dt question is does this function define a norm why 0 great so what does it mean a non-zero a non-zero vector is giving me 0 value a non-zero vector is giving me a 0 value so this is a non-zero function f of t is equal to constant okay and that will yield that will yield 0 norm if I use this definition so this function cannot be cannot qualify as a norm okay there are most of the examples given listed here we should go through them this is on page 12 okay I have listed many other functions and I have solved systematically so you have to if I give you a function and ask you to check whether this function satisfies to be a norm or not you have to you have to systematically look at all the three axioms first you have to look for whether non-zero element gives you a 0 vector okay whether the second thing is alpha times a original vector does it give you more alpha times and a third property is triangle inequality okay even in three dimensions there are multiple ways of defining norms see for example you can define a norm in three dimensions if I have x is r3 if I give you a matrix w which is symmetric and positive definite then I can define a norm of a vector x in three dimensions as x transpose wx okay you can show that x transpose wx will also sorry raise to half x transpose wx so we call this as two norm with matrix w where w is some symmetric positive definite matrix if this matrix is if this matrix is singular which means if it is semi-definite and not semi-definite what is the semi-definite matrix not all eigenvalues are positive so it means it has some eigenvalues equal to 0 so some eigenvalues are equal to 0 we will visit this animal definite positive definite semi-definite many times so let us wait for that right now let us wait for that but just think about this this w cannot be singular if this w matrix is singular then it will not define a norm because a singular matrix singular matrix can take a non-zero vector to 0 yesterday we looked at that right we had a singular matrix with three columns which are really dependent a singular matrix can take a non-zero vector to 0 vector so which means a non-zero vector can give you a 0 value for the norm which is not acceptable so it will not be a norm okay so w has to be a okay so the next thing that I want to want to talk about is convergence of a sequence of vectors so as I said we are going to deal with iterative processes so let us say you will have iterative processes which will give you xk plus 1 is equal to some function f of xk this is a we have we have a iterative process in which we start with a guess solution and then that guess is used to construct the next guess and then that guess is used to construct the next guess Newton-Laphson method all of you know about this we are going to generalize the Newton-Laphson method from one dimension to n dimensions or to a function space and so on so that time you will have sequence of vectors in n dimensions and I need to know whether they are close to each other or they are diverging how do I know about that I need to use definitions of norm if I have a sequence of functions generated in the method how do I know whether this sequence of functions is converging to a solution or it is not converging I need to use definition of norm that is where we need this norm definition generalize because we need to talk about you know convergence of a sequence we need to talk about limit of a sequence so we will look at this concept in my next lecture it is important to understand keep in mind that why it is being done it is being done for dealing with iterative processes okay so in the next lecture we will look at conversions and sequences we will have something called Cauchy sequence which almost converges and a conversion sequence and we will also look at some funny properties funny things that happen in infinite dimensional spaces you have a sequence which converges but the limit is not in the space the limit is outside the space so and so I will show you some examples where these funny things happen so