 Okay, so let us begin with just a quick review of axioms for norm. So given the vector space, given the vector space x and of course the field f, we wrote three axioms for defining a norm on this. So this function called norm was a function that was from x to r plus, r plus is real positive integers, real positive real numbers including 0, so r plus is set of positive real numbers including 0. So this is a function from an element in x, this is a compact way of writing from element in x to r plus and we said the three axioms, one is that norm x is greater than 0 for all x that belong to x and x not equal to 0 vector and norm x is equal to 0 if and only if x is equal to 0 vector. So this was the first axiom, the second axiom was norm of alpha times x is norm x mod alpha or absolute value of alpha times norm x, where alpha is a scalar belonging to field f and the third axiom is triangle inequality. So this states that distance of x plus y, take any two elements from vector space x, distance of the vector x plus y is always less than or equal to or the length of vector, norm of vector of x plus y is always less than or equal to norm x plus norm y, this is generalization of triangle inequality that you know for one dimension or in three dimensions for triangles generalized to any other space. So we said anything, any function that satisfies this criteria it should be a real positive function, it gives you should give you a real number, it should be a non-zero real number or when x is not 0 it should be zero real number or should be zero when x is not equal to 0 vector and so on. So these are and then we saw couple of examples that that can that of functions that can be classified as a norm or that cannot be classified as a norm. So both are important because you understand something better when you see where these one of these axiom fails. So there are multiple ways of defining norms not a unique way, a pair of a vector space together with or a linear space together with a definition of norm gives you a norm vector space okay, so that is another take home message, well why did we do this I yesterday said that we are doing all this because you know we want to talk about at some point about limits and sequences. So why do I need to talk about limits, when we work in numerical methods we are forced to look at sequences of vectors okay, I will just give you a very brief example will actually do this much more in detail later, let us say I want to solve this equation, these two are coupled equations and I want to solve them simultaneously okay, I want to solve them simultaneously, these kind of problems I am writing it in abstract form very often we encounter these kind of problems, well I am going to write this in abstract way as f function 1 x y is equal to 0 and function 2 x y is equal to 0, there are two functions the two functions f 1 x y is equal to 0, f 2 x y is equal to 0 okay, well these kind of equations arise steady state of a CSTR, concentration and temperature are linked, so first equation could be energy balance, second could be material balance and then you get two equations in two unknowns concentration temperature you have to solve them for okay, so I am going to define a vector, this is my function vector, I am going to call this as f of x, well let me call some new variable is equal to 0, so my neta is a vector, my neta is a vector which comprises of x and y and then I want to solve for f neta is equal to 0 vector, this is my 0 vector, this is my 0 vector okay, I want to, I am just writing the same thing in a different format okay, now what method you know for solving this, how do you solve this, pardon me, bijection method, bijection method is for difficult to scale to two variables, one variable well defined bijection method is there, you can have a bijection method for two variables but well let us take a very simple iterative scheme, let us construct a very simple iterative scheme, I will write neta plus f neta, I will add this vector neta on both sides okay and then I will construct an iteration whether it will converge or not is a different story but I will construct an iterative process, so I will start with some guess vector neta 0 that is let us say well I do not know what the solution is, so I am going to guess some solution, so let us say I start with say minus 1 and 1, this is x this is y okay and then what I want to do is to say that neta k plus 1 is equal to neta k plus f of neta k is it okay, I have just formulated an iteration scheme in which I start with vector 0, I take this 0 vector substitute here okay I will get vector 1 okay I take vector 1 substitute here I will get vector 2 okay, how do I know whether this sequence of vectors is converging to something, pardon me what is difference, see it is a two dimensional vector now I just for the convenience written two dimensional vector I could have done this in n dimensions, I could have written this in n dimensions, n equations in n unknowns very very common chemical engineering starting trying to solve steady state energy material balance for a plant you can get thousand equations in thousand unknowns okay pardon me but what of difference vector norm of the difference vector okay, so we have to talk about a vector converging to another vector, a vector converging to a solution what should happen at the solution let us say if x star is a solution neta star is a solution f neta is equal to 0 well what she says is correct that one thing is that you know should be equal to 0 of course at the solution so at neta star so that is that is f neta star is equal to 0 fine okay but I am starting a iterative process okay so what I am going to get is I am going to get this vector sequence x neta 0 neta 1 2 and so on I am going to get this vector the question is is this sequence is this sequence converging to neta star does this go to neta star okay that is the question I need to answer see this is the solution if I pluck neta k here it is not going to be equal to 0 it is not going to be equal to 0 so it is going to be some other small number probably is it small okay so how do you answer this question how do you answer this question in general n dimensional spaces or function spaces that is where we need to now talk about I may have a scenario where I have a sequence of functions okay I have sequence of functions and I will give an example I am going to show you a small demo also sequence of functions so the question is is this sequence convergent is this sequence convergent so these kind of problems are always encountered in numerical analysis okay because every method almost every method that you have for solving you know most of the problems through computing is iterative you start with the guess and you come up with a new guess and so on okay so there is there is there is this need to look at convergence of sequences so we are going to define two notions one is Cauchy sequence so I am taking a set of an infinite set of sequences or vectors infinite set of vectors which are generated by some process you know it could be some iterative scheme by which you are working or whatever it is now I want to know how do I formally define convergence a sequence of vectors is said to be Cauchy if difference between x n minus that is nth element in the sequence and mth element in the sequence if this tends to 0 norm of this tends to 0 as m and n become infinity so more and more elements you generate in this okay more and more elements you generate in this the vectors come closer and closer okay well in one dimensional vector space so in one in one dimensional vector space that is set of real numbers well when a sequence is Cauchy it converges to a limit inside inside the set but in the sequence that depends upon the space funny things can happen if the space is not complete what is this business of completeness will come to that soon before that let me define convergent sequence so there are two different notions one is Cauchy sequence other is convergent sequence these are not niceties just for the sake of nice mathematics these are very very relevant to computing now what is the convergent sequence so I am considering this sequence again in fact this is a short hand notation for sequence I am not going to write every time k going from 0 to infinity or k going from 0 to n whatever it is curly braces x superscript k is a sequence okay in a normed linear normed linear space or a norm vector space now this is said to be convergent to a vector x star if this is said to be convergent to an element x star if difference between x star and x k goes to 0 difference between x star and x k goes to 0 as k goes to infinity okay so what I want to show you is that it is not obvious that a Cauchy sequence will always be convergent it depends upon the space that you are considering a convergent sequence is always a Cauchy sequence but vice versa is not necessarily true okay a Cauchy sequence may not be convergent a convergent sequence is always a Cauchy sequence okay now examples will make it clear why I am talking of this funny things and you will also realize that this is something that you deal with every day when you use computers okay so I am going to take a example of a vector space in which a Cauchy sequence is not convergent I am going to take an example of a vector space in which a Cauchy sequence is not convergent okay so basically I am going to and I want to give an example of this idea that convergence to a particular element is something different and it depends upon the space my first example here is my space x is my first example here is set of rational numbers q and I am taking field f also to be q okay I am taking a field also to be q so this combination will form a vector space okay and then I can find very easy sequence in this vector space which is Cauchy but not convergent okay a simple example is now consider sequence whether I start index with 0 or 1 does not matter I am starting with 1 x2 is 1 by 1 plus 1 by 2 factorial and so on so my nth element in this sequence is 1 by 1 plus 1 by 2 factorial plus 1 by 3 factorial I think this is a well known series where does it converge to e but e is it a Cauchy sequence it is known to be a Cauchy sequence it is a convergent sequence in real line on the real line on the real line where does it converge to okay so this sequence xn this converges to element e as n tends to infinity we know that this particular element tends to this particular element tends to e but e is not a rational number okay so this element where it converges to is outside this space okay so you have a funny situation you have a Cauchy sequence you have a Cauchy sequence if you apply the definition of Cauchy sequence if you take any two elements as n and m goes to infinity you take difference it goes to 0 that is very easy to show look at any book on real analysis you will see this proof it is just one or two pages of proof that this is a Cauchy sequence but in this particular space it does not converge okay it does not converge and in this space I can find many such sequences I can find a sequence that is almost converging to pi but pi is irrational number pi is not there inside this space okay pi is not there inside this space so likewise you know I have this sequence so this sequence that is 3 by 1 11 by 3 41 by 11 and so on and so on it converges to okay not a rational number I can find infinite such examples where you have a convergent sequence you have a Cauchy sequence but not converging to an element inside this particular space so those are rational numbers so this sequence is converging somewhere but it is not converging inside this space it will never converge inside this space yeah so e does not belong to set of rational numbers that is what you are saying we know that from in we know that in the real line this will converge to e why these are all rational numbers yeah so you can always define one common denominator it is a rational number it is a rational number no all these are rational numbers I think you have to we can talk about it little later this particular thing is these are all rational numbers they are all rational numbers they are not irrational numbers so also you mean to say that 1 by 3rd may not be expressible but rational number is whether you can write it as integer upon integer I can always write it as integer upon integer okay whether you can express it as a continued fraction we are not looking at that problem right now the true representation is integer upon integer I can have a common denominator for this it becomes a rational number okay you are confusing between its representation in the computer I am coming to that okay so do not confuse between the two okay so do not confuse do not confuse one third with 0.33 okay do not confuse that with 0.33 okay if this is true about q it is also true about qn I can define a space product space which is qn dimensional space my x can be qn I can take a space which is where do you get qn when I am doing computing in a computer okay I can deal only with finite dimensional vectors I can only deal with finite dimensional vectors and in computer you cannot represent many of these you know irrational numbers I cannot because computer has a finite precision if I take 64 bit precision the resulting number which you approximate as e actually will be a rational number something divided by some I have to truncate right I cannot I cannot have a representation do you understand what I am saying in a computer whatever is the precision 64 bit you know 128 bit you go to very high high precision computer okay any number is actually represented as a you know in using binary 101010 sequence and there is finite number of bits used to represent a number okay so that number will always be representable as a rational number something divided by something I truncate it okay so the point which I want to make is that incomplete spaces are not so a lean you know when you work with computer you are working with incomplete spaces okay and we have to bother we have a Cauchy sequence which does not converge okay Cauchy sequence which does not converge in a computer I will have a Cauchy sequence which does not converge to a number so it is true value see for all practical purposes we say that well this is almost close to e but it is not e right it is not e we take a approximation of pi maybe you know correct up to 1000 decimals but it is not pi okay so we are working with this incomplete spaces and then let me give you one more example and I want to show a demonstration here of an incomplete space so my second example is set of continuous functions over minus infinity set of continuous functions over minus infinity to infinity this is my second example okay and I am going to take an element I am going to construct a sequence in this particular vector space and what I want to demonstrate is that this sequence will converge to a discontinuous function okay I have a sequence of continuous functions converging to a discontinuous function okay so you are trying to solve some partial differential equation or some problem you construct the solution as a sequence of continuous functions okay or continuously differentiable functions the sequence might converge to a non-differentiable non-continuous function okay so you can have funny situations so my sequence here is this 1 by 2 plus my sequence here is a sequence of functions these are continuous functions defined over interval minus infinity to plus infinity okay this is a function sequence defined so t goes from minus infinity to plus infinity my k changes k would be 1 2 3 4 5 okay I will get different functions for each value of k okay so I will get k goes from say 1 2 and so on okay k goes from so I am just going to show you okay so this function sequence I just want to animate and show you what is happening okay so this is for k equal to 1 this is for k equal to 6 I mean going to increment by 5 okay and see what is happening this is k equal to 11 16 and so on I just go on right so I am going closer and closer towards this step kind of a function right I am going closer and closer to the step function so if you do this I have gone only up to 100 or so if I do this if I do this by incrementing k much longer much to a larger value this will converge to a step function okay so model of the story is that I am starting with a set of continuous functions okay I am generating a sequence in this set but this sequence does not converge to element in the set okay this sequence does not converge to an element in the set so there is a problem okay so if what is nice about real line what is nice about real line that every real line every sequence which is Cauchy will converge to an element inside the right every Cauchy sequence on the real line will converge to a number on the real line okay so in some sense real line is a complete set there is nothing outside it okay whereas set of all rational numbers is incomplete okay there is something outside okay and the sequences here seem to converge to something which is outside the space okay seem to converge to something which is outside the space so what is nice about real line it is complete space okay what about what is nice about because real line is a complete space okay same thing is true about R2 two dimensional vector space any sequence in two dimensional vector space will converge to a point in two dimensions okay any sequence in you know n dimensional real Rn will converge to element in Rn but in Qn there are holes you know so where the sequence will be Cauchy but it will not converge okay so this spaces you know in which all sequences converge within the space are called as complete vector spaces and these are spatial vector spaces okay so there is something different about the spaces in which so we move back to the blackboard so we want this nice property to hold even in a vector spaces okay so we call this we will call this vector spaces these vector spaces which have this spatial property as complete vector spaces or they are named after a famous mathematician Banach who actually founded this one of the founders of functional analysis so what is the Banach space so every Cauchy sequence to converge to an element in space this word here every is important every Cauchy sequence okay if I can find one sequence which does not converge the space is not a Banach space every Cauchy sequence should converge so the real line or Rn or equivalently if you take complex numbers Cn okay they have some very nice property they are all complete spaces function spaces need not be complete spaces instead of continuous functions we saw is not a complete space well in functional analysis you talk about completion of an incomplete space you add all the elements and then create a new space okay which is which is complete and so on but we do not want to go into that those details right now I just wanted to sensitize you about the fact that even in the computer we are working with incomplete vector spaces okay and then you can get into funny situations in computing in advanced computing because of this incomplete behavior well so far so good we talked about so we started generalizing notions from three dimensions do not forget that we talked about a vector and then we said that are essential properties of a set which the two essential properties vector addition and scalar multiplications okay so these two things hold in a set then or if a set is closed under vector addition and scalar multiplication we call it a vector space any set so we freed ourselves from the notion of a vector space which is just three dimensional okay we can now talk about set of continuous functions set of continuously differentiable functions set of twice differentiable twice differentiable and you can so now how many such spaces are there are infinite such spaces okay then we said well we now that is not enough to have just generalizations of vector space we also need notion of length so we talked about norm right we talked about norm was in some same generalization of notion of magnitude of a vector okay and we said there are so many ways of defining norms okay and a pair of a vector space and a norm defined on it will give you a normed vector space or normed linear space okay so this up to here fine now we need something more I need angle okay one of the primary thing that you use in three dimensions one of the most fundamental result in our school geometry or in three dimensional geometry is Pythagoras theorem and I need Pythagoras theorem in all these spaces what am I going to do okay I need Pythagoras theorem so I need orthogonality I need perpendicularity one of the most important concepts that you use in applied mathematics in modeling in physics in chemistry in everywhere orthogonality is a very very quantum chemistry chemistry in the sense might wonder where in chemistry so orthogonality is very very important and we need to generalize the notion of orthogonality and that is where we will start looking at inner product spaces okay we start looking at inner product spaces now here the attempt is to generalize the concept of dot product how do you define angle in three dimensions well if I am given any two vectors say x and y which belongs to R3 how do I find the angle between them so what I do is I find out x cap which is a unit vector in this direction normally I take a two norm here well y two norm will come to that why not one norm okay so there is something special about this two norm and y cap is equal to and then we say that dot product dot product that is x cap cos theta cos theta angle between these two vectors is just x cap transpose y cap right is a fundamental way by which we define angle between any two vectors in three dimensions now can I can I come up with something that will generalize notion of angle in three dimensions when do you say two vectors are perpendicular in three dimensions dot product dot product when dot product is zero cos theta is zero two vectors are perpendicular okay so I am going to peg onto these ideas well that dot product between unit vectors is defined is used to define angle when dot product is zero you call two vectors to be orthogonal okay and come up with a generalization in a product spaces okay of concepts of angle orthogonality and then once you have orthogonality you have Pythagoras theorem okay I can talk about Pythagoras theorem in any n dimensional infinite dimensional space of course it has to qualify certain properties what are those properties those are the properties of a inner product space so now we have to start questioning what is what is characteristic of an inner product okay see we had three properties of magnitude what were the three properties of magnitude magnitude is always non negative okay for a non zero vector and zero for a zero vector okay alpha times you get you know you multiply mod alpha gets multiplied to the norm and triangle inequality likewise what are the essential properties of inner product in this which can be used to generalize in any other vector space those vector spaces those vector spaces are going to be called as inner product spaces because we are going to define a normed vector space in which additional structure is put called inner product all these spaces which we are discussing till now we did not talk about inner product okay so now I am going to introduce something new which is a inner product space which will have definition of inner product what we will realize is there empty number of ways to define inner product and so the way of defining generalizing orthogonality is not unique okay and so we will see this from our next lecture.