 from the range space and the operator t, I want to find out x. This is called as inverse problem and these are the problems which we normally have to solve. So, the core of this particular course is dealing with these kind of problems and the next class which is identification problems. So, here what falls under this? So, solving A x equal to b. So, solving A x equal to b given A and b classical problem which given right hand side given operator you want to find out x right. This is probably the problem which you will solve most often in this course. Then so, we are given some differential equation. So, the other classical problem is only initial value problem. We are given a differential equation. So, this is the operator we are given f t, f t is equivalent to b or f t is equivalent to the vector in the range space. I want to find out solution x t which satisfies the condition that initial value is equal to alpha and initial rate is equal to beta. I am given two initial conditions, I am given the operator, I am given the vector in the range space I want to find out solution x t inverse problem. Operator is known, the range space vector is known initial conditions are known I want to find out x t inverse problem. So, likewise ODE boundary value problem is an inverse problem, boundary value problem is an inverse problem or solving a partial differential equation that we encounter in engineering are mostly inverse problems. We are given the vector in the range space, we are given the operator we have to find out x that satisfies the differential equation boundary conditions and solution gives you the vector in the range space. So, these two problems are conceptually similar a x equal to b or this operator operating on x t giving you this vector f t and then the solution should satisfy these two. So, these kind of problems are inverse problems. So the third class of problems that you encounter in engineering mathematics are identification problems. So, you are given x and y and you are asked to find out operator t. The classic problem here is model parameter estimation. Suppose I want for some particular material you want to find out c p as a function of temperature. So, you have this a plus b t plus c t square I do not know a b c I have been given values of c p I have been given values of temperature. So, I have been given x I have been given y y here is c p x here is temperature what I want to find out is the correlation is the operator. I want to find out the operator finding operator it in this case reduces to finding out a b and c. So, given data parameter estimation problem what are the other parameter estimation problems you have seen in chemical engineering? Rate expressions reaction rate expressions you have measured the rate of change of a concentration of a particular species and then you have proposed expression you do not know the parameters you have rate values you have concentration values you want to fit find out the parameters of the rate expression or you know you are trying to fit some thermodynamic correlation p v t correlation you have data for p v and temperature and you have a proposed model you do not know the parameters you can fit estimate the parameters from data. So, you are trying to find out the operator knowing I mean if you if you look at y as a effect and x as a cause. So, the operator t operates on x gives you y y is the effect. So, you know cause and effect you want to find out the operator another example is estimation of transfer functions in process control you give a input perturbation you measure the output you try to fit a transfer function into that data. All these are examples of identification problems ok. So, bulk of our bulk of our work in this course is going to be inverse problems and then we will also look at identification problems to a large extent. Direct problems are not going to be focus I am not saying that direct problems are not important, but they are relatively easy to deal with these two problems are more difficult and we should get a better understanding of these problems. Now, the main problem in most of the cases not in every case in most of the cases is that once you have formulated a problem it may not be possible to construct analytical solutions to the problem ok. Particularly if the operator is non-linear ok. So, I can say in general when operator t is non-linear you cannot construct analytical solutions well there might there are of course, many exceptions, but the cases where you cannot solve are far more than the cases where you can solve. So, in general you can say that when the operator t is non-linear. So, lot of numerical analysis is all about transforming a problem which is not analytically computable to a form which is numerically computable ok. So, actually my original problem is y is equal to t of x ok and where you know y belongs to space y and x belongs to space x ok. I am not able to solve this original problem. So, what I do is I approximate t using approximation theory and then I get let us call let us call it let us call it t cap let us call this t cap ok. I get a t cap which actually works on x tilde and gives me y tilde ok. Here y tilde belongs to y n and x tilde belongs to x n. So, these are finite dimensional spaces typically typically these are finite dimensional spaces and then we end up solving this problem not this problem. We hope that the solution that you get suppose I solve a inverse problem which is I wanted to solve the original inverse problem I end up solving an approximate inverse problem ok. And I hope that the solution x tilde is close to x ok x tilde is close to x. So, this is generally the situation now how do you get from here to here next about 10 to 12 lectures are going to be how do I go from here to here. So, this looks abstract right now, but keep this in mind in background that this is what we are going to do ok. So, we might start with a partial differential equation and end up with non-linear algebraic equations ok. See this might be a partial differential equation what you end up here might be linear algebraic equations or non-linear algebraic equations ok. So, what you start with and what you end up with can be completely different ok. So, it is not that because I start with a differential equation I will end up with a differential equation ok. Now, when you go from here to here there is no unique way of constructing tcap. There are multiple ways of constructing tcap. Same problem can be approximated discretized in multiple possible ways that is what we are going to see here. And each one of them has advantage and disadvantage. So there is nothing like the method to discretize. And as you go along doing numerical problems you will develop your own preferences as to. So I am going to talk about not just one method I am going to talk about multiple methods. So you might wonder why am I talking about multiple methods because there is no one way to solve the problems. Sometimes some methods are simple but if they do not work you need to go to more complex methods and so on. So when you attack a problem you should have a repository of tools or a repository of you know approaches to deal with a problem. And then you can go on you know simple method first it does not work go to a more complex method does not work go to a more complex method. What do you mean does not work the proof of the pudding is x tilde close to x does it make sense. Now you in real situation you never know what is x true x but since you are an engineer if you look at the solution you can make out whether this makes physical sense or not. Whether the solution make sense as a engineer as a scientist as a physicist you can make a judgment and then decide whether your method is giving reasonable results or not. So there is lot of subjective element here which requires development of expertise. So even though we are dealing with applied maths which everything cannot be automated and that is why we are in business. There is still scope for improvement you know for interpretations for doing it differently getting better solutions and so on. So now let us begin. So what is the trick? What is the basic trick that is so if you ask me to distill out one basic idea which is used to do this transformations from t to t hat. Cutting across all the methods for boundary value problems or partial differential equations for all kinds of things. What is one trick that is used? Well if you ask me to sum it up I will say that approximate a function by polynomial. That is a trick that is a underlying trick you know if I cut across many of the methods and what I am going to show in next few lectures is that how this one trick is used to you know deal with variety of problems. Starting from non-linear algebraic equations to partial differential equations to boundary value problems to all kinds of problems we just use one trick in multiple ways. So basic idea is approximate now the question is well is it just an observation what is the basis? Why should I approximate a function by a polynomial function? Why not cosine functions? You know why not exponential functions? Why polynomials? What is so great about them? Well of course they are convenient when you do calculations but not just that there is something deeper into why polynomials are used for approximating approximating functions and then developing different methods for solving the problems. So fundamental concept here is a concept of a dense set. See approximations is something that we very very often used in mathematics for example pi. You know when you start using pi you start using it as 22 by 7 but pi is not equal to 22 by 7. It is an approximation of it is a approximation of pi for doing you know rough calculations not exact calculations. When you start using for example E we never use the true value of E right we use an approximation a finite truncated approximation of E and do calculations right. So what allows you to do that? What allows you to do that is that you can approximate a real number using a rational number. You can approximate a real number using a rational number. This property of rational numbers is something special about rational numbers you can approximate any real number as close as you want by a rational number and this particular property is exploited by us when we do computations and the best example I said is pi being replaced by 22 by 7 or whatever there are different rational approximations of pi you remember something else. So 141 by or there is some other approximations also which are not so popular in the school books but what we use in school book is 22 by 7 right we are always. So why we can do this why or I told you that in when you are doing computing all the numbers are finite precision right no number in a computer. So there will be some missing numbers if there is a finite precision okay not all numbers can be represented particularly it is not possible to represent you know all real numbers pi may not be truly representable you can represent using some truncation because when you do finite precision okay if I write some expansion for pi I can write that number divide I can write a integer divided by some 10 to the power something and that will be a rational number right. So it is not it is not the correct value for pi it is a rational approximation for pi okay. So I am able to do this because of this denseness property okay. So what is a dense set okay. So let us go over the definition a set D a set D is set to be dense in a normed space. So first of all we have to work in a normed space we need norm okay without that we cannot work with this numbers or vectors. So for any element if I give you any element x in x I should be able to find and if I give you an epsilon okay I should be able to find an element D belonging to dense set D which is close to x okay how close there is an epsilon. So you know it is like saying if my x is pi okay and if I specify epsilon you should be able to come up with a rational number which is close to pi such that the difference is less than epsilon. Now you know she might say that my epsilon is 10 to the power minus 3 okay. I will come up with one D which is one rational approximation of pi pi minus D is less than 10 to the power minus 3 and he comes up and says no no no I do not accept 10 to the power minus 3 I want 10 to the power minus 9 is it possible? It is possible to find given pi it is possible to find a rational approximation such that pi minus that number is less than 10 to the power minus 9 and somebody does not accept 10 to the power minus 9 and you know he says 10 to the power minus 17 fine. I can find a rational number which is pi minus that rational number will be less than 10 to the power minus 17. So every epsilon any epsilon that is very important okay. So what does it mean that on real line these rational numbers are everywhere you know I can use them as approximation of something else which I am not able to represent which is very nice okay which is very nice. So I can use rational number as an approximation of a real number and that is why I can work in a computer okay. So when I am working in n dimensional space okay in Rn even though I may not be able to represent you know all elements of the vector Rn because a real number may not be exactly representable I can replace it by its rational approximation okay. See suppose I have a vector which is like this suppose I have vector which is E minus pi pi square root 7 in a computer can I really work with E minus pi I cannot right I actually replace this by some approximations. So some rational approximation of E some rational approximation of this some rational approximation of this and so on and why we can do this because set of no no understand the philosophy why we can do this is because set of rational numbers is dense okay rational numbers are everywhere you can just if you want to represent a real number pick a very close rational number you know you will be having a good approximation okay now I want a similar result to this in set of continuous functions okay what I am going to work with now we have seen that you know when you when you deal with partial differential equations when you deal with boundary value problems when you deal with ordinary differential equations you will be dealing with set of continuous functions okay so this is nice here that you know I can use I can use rational approximations some you know some some q1 q2 q3 q4 so this is the rational approximation of this and in my computer I can do calculations with this the same way I want analogy in set of continuous functions this is a similar idea conceptually similar idea is that set of polynomials is dense in set of continuous functions okay set of polynomials is dense in the set of continuous functions in the same sense the set of rational numbers is dense in set of real numbers okay polynomials you know you can approximate anything by polynomial any continuous function by polynomial so this was a landmark theorem given by a German mathematician Weierstrass I think somewhere in 1850s or 1860s and this is a celebrated theorem by called as so this is a well-known result well this particular result is what is called as an existence result okay I will tell you I will tell you what I mean by existence result it doesn't tell you how to construct a polynomial approximation it assures that given a continuous function there exists a polynomial which is arbitrarily close to the continuous function now what is arbitrarily closeness you need norm okay what is arbitrary closeness you need concept of norm so now what do we consider here we consider the set of continuous functions over an interval a b together with infinite norm okay together with infinite norm so this is the space this is the norm defined on it this is a norm linear space okay now Weierstrass theorem Weierstrass theorem tells us that well I will move on to here to complete this theorem statement so if I if I give you any epsilon greater than 0 any degree of accuracy epsilon will specify how accurate you want the approximation okay and if I pick up any continuous function f t from C a b set of continuous functions over a b then then there exists a polynomial very very important result so what does it say given any epsilon you give me the accuracy that you want how close an approximation you want okay you can specify that epsilon and give me any function f t which is a continuous continuous function okay then there exists a polynomial approximation I am going to call this as p n t n will be let us say order of the polynomial okay such that f t minus p n t is less than epsilon okay is less than epsilon is this is this clear so what is this norm this norm is absolute norm we are finding out difference between maximum of the absolute value see if I give you a function let us say sin t okay this theorem tells me there is an nth order polynomial such that sin t minus the polynomial absolute of this okay maximum order interval will not exceed epsilon you specify epsilon I will construct a p n okay you give me an epsilon I can now how do you construct the p n is not what is told by this theorem it just says that there exists okay how do you construct that approximation well that is a different story it only it only assures that there exists a polynomial which is arbitrarily close how do you find out that particular polynomial is not is not given by this theorem but it tells you that there exists a polynomial so which means which means when I am approximating when I am approximating a transformation I could use this basic idea I could use this basic idea to transform a differential equation or transform you know boundary value problem or a partial differential equation into some simplified form we will do it much more in detail but I will just give you a very very simple example okay so do you see parallels between here and here we are talking about finding out a rational number which is arbitrarily close to a real rational number and using that rational number for calculations instead of the real of the real number okay same thing same idea we are going to do here okay the true solution the true solution would be a continuous function okay I am going to approximate that continuous function by a polynomial function why that will help me to solve the problem it will help me to solve the problem and you know by by transforming the operator as solve the problem in a different way which is easier than the original problem well one one simple this kind of things will hit on later let us let us look at a very very simple demonstration see if I have if I have dx by dt is equal to some f of x okay and I have given you initial condition corresponds to say x0 okay now what is x t let us say x star t is the true solution x star t is a true solution okay is this a continuous function it has to be a continuous function it has to be in fact differentiable function not just continuous it has to be a differentiable function so it is a continuous function any continuous function can be approximated by a polynomial function I propose a polynomial solution which is say p and t or I will call it x t which is a0 plus a1t plus a2t square let me propose a polynomial solution now this is a polynomial approximation this is a polynomial approximation this is not a true solution but I can substitute it here I can substitute it here and I can say that well I want approximate solution such that so what is dx by dt a1 plus 2 a2t right and then I can substitute this here so for any t I want this equation to hold that is a1 plus 2 a2t is equal to f of a0 plus a1t plus a2t square right I am doing something which is very naive will do it much more sophisticated manner afterwards I just want to carry some point here look this is a differential equation I started with I approximated using a polynomial form with unknown coefficients I do not know a0 a1 a2 true solution is x star what with what boldness I can do this I know at a continuous function can be approximated by a polynomial function okay I substituted this what happened what looked like originally a differential equation now looks like an algebraic equation with unknowns a1 a2 a0 the problem is transformed from a differential equation to an algebraic equation okay so this idea of using a polynomial approximation of a continuous function will be used to transform problems which are originally so original operator at t was a differential operator t prime or t cap what you are getting here looks like an algebraic so I was talking about you know starting with an original problem transforming the problem and solving the transform problem so we might actually computationally this is easier to track than this we might solve this as compared to this what we get by this approach is the approximate solution not the true solution remember that okay this is approximate solution but then if you can accept 22 by 7 in place of pi you can accept these approximate solution and as long as it is close and you know your physics is you know preserved in some sense qualitatively you do not bother too much about the difference between the two okay so this is how it is going to help us in transforming the problems so what is what is used so we are star approximation theorem as such we never revisit again but it is the foundation everywhere you know we are star theorem comes at in a hidden form it is everywhere because we approximate we approximate continuous solutions using polynomials so somebody ask what is the basis why polynomials because polynomials are dense why should why should I be so much worried about a dense set you know dense set is something which can pick an element from a dense set and it can be as close as possible to the original you know element in the original set okay so dense set is a special set so just like set of rational numbers is a special set in real numbers polynomials set of polynomials is a special dense set instead of continuous functions okay so this is the foundation this is the you know cornerstone result but it does not tell you how to construct a polynomial approximation now we are going to use three different tricks for constructing polynomial approximations one is so there are three different ways by which we are going to construct the polynomial approximations first is the Taylor series approximation you are familiar with Taylor series expansions we will just revisit them briefly in the next lecture then we move on to polynomial interpolation interpolation polynomials okay and the third is the least squares approximation okay so if you understand these three basic concepts most of the problem transformations will be clear to you how the problem is transformed to a computable form okay then comes how to solve the transform problem okay so that is the next part so till mid-sem we will be actually working on now this will systematically look at different problems particularly boundary value problems partial differential equations non-linear algebraic equations and all kinds of things where we use these ideas these three ideas and transform the problem to a computable form so next 12 lectures are about problem formulation okay you have formulated the problem from physics and you got some problem which is which is coming from your courses in transport reaction engineering whatever heat transfer strength of materials whatever your specialization so those those original problem is coming from there okay I want to compute a solution for this a numerical solution for this problem so I use all these tricks to transform the problem to a computable form and then I solve that computable form okay construct a solution which is approximate numerical solution to the problem okay so in the next class we will start with Taylor series approximations okay I will very quickly review Taylor series approximation what it what is the basis behind Taylor series approximation you are aware of only one variable Taylor series we will move on to multi variable Taylor series okay polynomial functions in n variables okay and then we will look at for example one of the application of Taylor series would be Newton Raphson method okay then we will move on to show that this Taylor series approximation actually actually gives rise to the finite difference method of solving boundary value problems finite difference method of solving partial differential equations okay and so on or the polynomial interpolations so we will develop in the class method of orthogonal collocations and see how orthogonal collocations arises from polynomial interpolations and so on so all these three different approaches give rise to different ways of problem discretization and that will be the center theme for next few lectures.