 In our last lecture we were looking at Taylor series approximation and using Taylor series approximation for problem discretization. So we looked at solving non-linear algebraic equations of this form. So what is the discretization process involved here I want to point out you know what is the original operator what is the discretized operator and how we are solving a problem which is not the original problem but a different problem so that I want to point out here. So one thing which I stressed in our last lecture was polynomial approximation actually is the cornerstone of approximating different problems or discretizing different problems in numerical analysis and Weierstrass theorem gives us the foundation why we can approximate a continuous function using a polynomial function. Now how do you do it Weierstrass theorem is only an existence theorem it does not tell you how to construct polynomial approximation so we said we are going to look at three different ways of constructing polynomial approximation one of them is Taylor series approximation and then we looked at example of multivariable Taylor series that was developing Newton's method for solving non-linear algebraic equations. So what I had been talking about earlier is that you have this y is equal to t of x where x belongs to a subset m of x x is our vector space and y is x is the domain y is the range this is the original problem and then I said that we actually end up solving y tilde is equal to t cap x tilde we end up solving a different problem than what we started with so I just want to just oppose the two things what we started what we wanted to do here actually was to solve f of x that is equal to f1 x f2 x I wanted to solve this problem okay using Taylor series approximation well this original problem n non-linear equations and unknowns is not analytically solvable in general maybe there are some cases where you can solve it but in general this is not analytically solvable we come up with a simplified problem we actually solve we start with a guess solution x0 and then we solve this problem dou f by dou x so this is my initial guess so I wanted to solve this problem I wanted to solve this problem okay this is the original simultaneous non-linear equations actually the way I end up solving this is by approximating okay where I end up solving this is by approximating so this is my t cap what is t cap here t cap here is a sequence of linear algebraic equations which are constructed from the original problem and what we hope is that this sequence this sequence will finally go to so x0 x1 xk will tend to x star will tend to x star what is x star x star is my solution of the original problem so I hope that f of x star is equal to 0 vector I hope to converge to this solution x star okay so this one is my newton's method here this is my newton's method this is my newton's method I hope that the sequence of vectors which is generated by this method will eventually converge to x star okay what is x star x star is the solution so how do we how do we check whether convergence is occurred typically we keep checking for f of so we keep checking for convergence whether f of x has become really small where epsilon 1 and epsilon 2 are typically very very small numbers so we keep checking whether the convergence is occurred and we want to we want to know whether the vector where the sequence is converging whether that solves this problem equal to 0 we actually cannot check in a computer exactly equal to 0 so we check for a norm this could be any norm this could be infinite norm this could be one non two norm whatever is whatever you like to use any one of the norms can be used so original problem has been approximated using Taylor series and then the approximate problem is solved and then we hope that the sequence generated in the approximate problem solution will tend to the solution of the true problem now I have just introduced this newton's method here we'll be revisiting newton's method again much more in detail there are many modifications to make it make it converge and or how can you how can you accelerate so we'll talk about it later right now I'm introducing newton's method just as an application of Taylor series approximation okay about one and one and a half months later we'll revisit this newton's method much more in detail solving nonlinear equations much more in detail but the point to convey here is this was achieved through Taylor series approximation so the basis for this was using Taylor series so what we have done is f of x f of x k plus one okay has been actually approximated as f of x k plus dou f by dou x evaluated at x k so this into if you like a close look at newton's method what we have done is we have approximated x k plus one in the neighborhood of x k so when I start with zero x zero I start with the initial get x zero okay and then instead of solving for f of x equal to zero I solve a linearized approximation of this set of equations okay and that gives me x k plus one so x zero will give me x one then I substitute into the sequence I linearize at x one I get x two I linearize at x two I get x three so this is a sequence of vectors which I generate by local linearization okay so original problem which was solving nonlinear algebraic equations was solved by constructing a sequence of linear algebraic equations and approximations from the foundation of applied mathematics or the applied engineering mathematics because we cannot most of the time solve the original problem we have to approximate by some means and convert into a computable form that computable form is then used to construct a approximate solution. Now let us look at next application of Taylor series now I am going to look at solving boundary value problems OD or ordinary differential equations boundary value problems or partial differential equations using Taylor series approximation okay so I will be converting my boundary value problem either into a set of algebraic equations or a partial differential equation I will be converting it into either algebraic equations or I might be converting them into a set of ordinary differential equations and so on so I will convert it into a standard form which then can be uh attacked using a standard tool what is the standard tool that is used here when you are solving ax equal to b right see here here each one of this problem each one of this problem is just solving linear algebraic equations each one of them is like solving a I would if I want to put a notation here ak delta xk is equal to bk in abstract form I am solving these kind of problems right this is a matrix n cross n matrix delta xk is a vector bk is a vector these vectors are changing this matrix is changing but I am solving them repeatedly to cover with the solution of the ordinary algebraic equations okay now let us look at you know problem discretization using uh or boundary value problems discretization using Taylor series approximation what I want to convey here is that now I am going to develop this method of uh you know approximating local derivatives using Taylor series which is something which I am sure you have done it in your undergraduate you are all of you are aware of forward difference approximation backward difference approximation central difference approximation all these approximations local approximations you are aware and then you may have used it to actually simplify some boundary value problem or some if you have done some work on numerically solving this now what is it that that I want to convey here what I want to convey here is that a tool that you are using to discretize boundary value problem or to discretize partial differential equation is same as the tool that is being used to construct Newton Raphson method or Newton's method underlying ideas are same okay problem is different application is different but basically we are using Taylor series approximation okay so that is that is the so now let us look at okay so now this problem is like this that I have let us for the time being take a set of uh differentiable functions on interval 0 to 1 so this is 0 to 1 and I have some continuously differentiable function okay and this is some point let us say this is uh this is some some point z is equal to or z is equal to z bar this is a point at this point I want to construct a local approximation of derivative of this derivative of this function okay something which you know very well how how this is uh then I am sure power difference backward difference has been taught to you at some point so now I just want to put it in the context of Taylor series so that the connections become clear okay so this is this is my interval and it locally I want to so what we do is around this point around the point we take some small perturbation okay so I can take a perturbation so let delta z greater than 0 be a perturbation okay that delta z greater than 0 be a perturbation and I want to look at Taylor series expansion of f z bar plus delta z I want to look at Taylor series expansion of z bar plus delta z I also want to look at Taylor series expansion of z bar minus delta z okay and then using so I want to expand this function now this function which I am looking at should be continuously differentiable once twice twice depending upon the order that order of approximation that you want to develop so this is so I want to develop Taylor series approximation in the neighborhood of z is equal to z bar and then use that further to discretize boundary value problems okay so let me well I am going to change a notation a little bit instead of using f here this is because further when I develop boundary value problems I want to use a particular notation so I am going to use here u is a continuous function continuously differentiable function which we want to so if I take u z bar plus delta z I can write this as u z bar plus so the notation here means the notation here means that we are actually computing all these derivatives at z is equal to z bar okay and so on so I am going to write terms up to third order and then say r4 z bar delta z okay so similarly I can write u z bar minus delta z that is u z bar minus so here delta z square so I am expanding each one of them each one of these scalar valued functions differentiable functions as a Taylor series expansion in the neighborhood of u in the neighborhood of z is equal to z bar okay what we know what we know about a Taylor series is that the derivatives of original function and derivatives of the approximation are identical at z is equal to z bar okay this is something which we know about a Taylor series expansion so I am going to use this property to construct approximations see the first equation there are multiple ways I can arrive at approximation starting from these two equations so equation number say 1 and equation number 2 okay so one way is one way is I rewrite this equation as though actually we can write exact differentiable because we do not have to these need not be partial derivatives these can be exact the not exact this these are not partial derivatives there is only one variable so I can write d here no need to take partial derivatives okay so du by dz at z is equal to z bar I can rewrite this as u z bar plus delta z minus u z bar minus half so one way is I can write this as one way is that I can write du by dz bar as u z plus delta z minus u z bar by delta z plus some terms this involves second order derivatives and all the higher order terms okay so I could choose to neglect the terms that are if delta z is very very small I can choose to neglect terms of order delta z and higher and then I get an approximation which is forward difference approximation so this is my forward difference approximation if I choose to neglect terms of delta z and higher order okay if delta z is very very small so if I am in a very small neighborhood of z bar I could actually approximate this I could use this as an approximation for the local derivative the same way is I could rearrange the second equation so the first equation I get like this the second equation would give me du z bar by dz right now right now just remember that I am writing equal to okay that is because this is this plus something okay if you do not neglect it if you do not neglect this is exact equality if you do not neglect it it is exact equality moment with moment we choose to neglect this delta z and higher order terms then it is approximation of this will be local approximation of this will be this is the forward difference so likewise I can develop a backward difference approximation using this one so this will become u z bar minus u z bar minus delta z upon delta z so plus the terms which you take on this side so these are all these terms are of order delta z these terms are of order delta z delta z square delta z z cube and so on so when the first term when the first the the the order of this is the smallest term that we are neglecting so in this case delta z we are neglecting in this case also delta z higher we are neglecting so this becomes my backward difference approximation this becomes my backward difference approximation but these two approximations okay are somewhat inferior because we are neglecting terms of the order delta z and higher if there was approximation where we could neglect terms of delta z square and higher okay then that would be a better approximation of the local derivative than these approximations so that turns out to be the central difference approximation so the way we derived central difference approximation is we subtract we subtract equation 2 from equation 1 subtract equation 2 from equation 1 and then we arrive at the central difference approximation in central difference approximation what will happen is that this second derivative here will vanish if you notice if I subtract from this term this term this is positive this is positive this will vanish okay but the third order derivative will remain okay the third order derivative will remain and what we get here is after after doing this subtraction and rearranging so here what you get is so when you subtract equation 2 from equation 1 and do a rearrangement okay you will see that this the terms of delta z disappear you get in the residual you get the first term which is delta z square okay so the so what we the way we write is that this is order of order of delta z square so the terms delta z square and higher are neglected and what you get here is the central difference approximation okay so this is my central difference approximation this is my central difference approximation so this central difference approximation is preferred over is preferred over forward difference or backward difference that is because in forward difference or backward difference you neglect terms of the order delta z and higher in this case you neglect terms of delta z square and higher okay so this is This is a better approximation than forward difference or backward difference approximation. What about second derivatives? I could use these ideas to come up with, I could use these ideas to come up with this the stellar series approximation to approximate the second derivative here. So if I rearrange these equations, I just have to be careful what I eliminate. If I rearrange these equations, you can develop subtract and rearrange and you can develop the second order difference equation. So you can develop this d2u by dz square is equal to uz plus delta z, z bar plus delta z minus 2u, so this is my approximation of second order derivative or d2u by dz square at z equal to z bar and this one is the residual term. You can see here this is again of the order of delta z square. So the order of approximation in central difference and order of approximation in the second order derivatives is identical. So we prefer to use this and this together, we normally do not use the first order approximation except at some boundary points, we will come to that when we use the forward difference or backward difference. But typically we use these two together because the order of error in both of them is identical. So far so good. So how do I use this to solve a problem which is a boundary value problem, let us go to that. Now what is a general boundary value problem? I am going to write it in a generic form and then we will come to specific examples. So now I am concerned about discretizing a boundary value problem using Taylor series approximation and the concept that I developed just now I am going to use them to solve this problem. Now let us write this generic problem here, so I am writing a generic boundary value problem which we encountered normally in engineering, chemical engineering or most of the engineering problems. So this is d2u by dz square, so typically we have this second order differential equation which is I am writing it as some psi, this could be a linear differential equation, this could be a non-linear differential equation, some differential equation. In different context you will get different differential equations. Now when you are studying transport, when you are studying your analytical methods, you will encounter many such equations and we will also when we actually solve problems we will also come across many such equations. Right now I am writing it in an abstract form, a second order ODE can be written in abstract form, u is the dependent variable, z is the independent variable, z spans from 0 to 1, okay, z spans from 0 to 1 and so this is my ODE, ordinary differential equation which holds over 0 to 1, okay and at the two boundary points I have two boundary conditions, okay. So my BC1, I am going to write this as F1 dz0 du0 by dz u0 0 is equal to 0, this is my first boundary condition, I am writing it in the abstract form, we will look at specific examples. So this is my first boundary condition and this is my second boundary condition. So there are two boundary conditions, at z is equal to 0 and z is equal to 1. The differential equation, the differential equation should be satisfied everywhere in the domain 0 to 1 except at the boundary points. What do you want to happen at boundary points, there are two boundary conditions, okay. So this would arise from the specific nature of the problem, see for example if you have a double pipe heat exchanger, you will have some conditions of temperature at the entry, okay. So you will have some conditions of say rate of change of temperature does not change after fluid leaves the boundary, so those conditions will be given here, they could be rate, for example rate of change of temperature does not change after fluid leaves the heat exchanger boundary, that could be a condition in a double pipe heat exchanger here. The initial temperature of the fluid entering here, that would be a initial condition at z is equal to 0 specified, so depending upon whether it is a reactor, whether it is a heat exchanger, whether it is whatever, a distributed parameter system, you will have different boundary conditions and you want to solve this problem, okay. Please remember that this original problem is my y is equal to Tx equivalent, y is equal to Tx equivalent, okay x here is uz, okay and y here is you know 0 function and say alpha 1, alpha 2, well another way of writing this is this is equal to alpha 1 and this is equal to alpha 2, so if I put this, so this is my original problem, this is my original problem which I want to actually solve, okay. I am not able to solve this problem exactly except when the operator is linear and the boundary conditions are nice, you can actually construct analytical solution but and that you will be looking at when you will be studying that under strongly weighted theory, you will be studying all those kind of things but majority of the problems where the differential equation is non-linear or boundary conditions are not nice and simple, you cannot solve the problem analytically and then you have to construct a numerical solution to this problem, okay. So now I am going to use Taylor series idea to approximate this problem, okay. So I want to use these derivative approximations, okay now the problem is where do I use the derivative approximation, the derivatives are required at the boundary points, the derivatives are also required everywhere inside the domain, okay at how many points this differential equation should hold at every point, there are infinite points between 0 to 1, everywhere this differential equation should hold, okay. Now when I am solving it numerically, I cannot afford to formulate this differential equation at every point between 0 and 1, what I am going to do is instead of that, I am going to convert this, I am going to convert this, so this is my domain, let us put this z is equal to 0 here and this is z is equal to 1 here, okay I am going to mark what are called as grid points, okay I am going to mark grid points, okay so I am going to mark grid points, in general when I mark this grid points they need not be equidistant, they need not be equidistant but to begin with for a problem you could start creating equidistant grid points, so I am going to create grid points here, so I am going to label them this is my z1, this is my z2 and so on, so in general this is grid point zk, this is zk plus 1, this is zk minus 1, these are my grid points, okay so I am going to create grid points, so here z1 is equal to 0 which is less than z2 which is less than z3, how many grid points I am going to create n plus 1, okay so I am going to label them as z1 to zn plus 1, okay I am going to label them as z1 to zn plus 1 and then I am going to develop local approximations of the solution at each of the grid points, okay so what is the solution of this equation? The solution of this equation okay is let us call it u star z, this u star z is a function which is twice differentiable it should be, right otherwise it will not be a solution of this differential equation, the true solution is a twice differentiable function continuous continuously differentiable function over the interval 0 to 1, okay so remember this, this is a true solution, so the solution is a twice differentiable function over domain 0 to 1, okay I am going to construct a local solution, I am going to construct an approximate solution uz, uz is not going to be u star z, okay you will realize why it is not going to be star z very soon, okay what I am going to do is I am going to approximate this unknown solution right now it is unknown to me, I want to solve the problem I do not know what it is okay but I am going to approximate at any point, okay I am going to approximate the first and the second order derivatives of uz, so uz first of all remember is an approximate solution which is going to be constructed numerically, okay now at point k, at point z is equal to zk, okay I need if I want to enforce the differential equation what do I need, I need the first derivative, I need the second derivative, right see I need to enforce look at this differential equation here, I need to enforce this differential equation at every point in the domain, okay instead of that what I am going to do is instead of enforcing it at every point in the domain I am going to enforce this equation at some finite number of points, okay what are these finite number of points, these finite number of points are listed here z1, z2 you know see this domain I have marked, okay so in general you know you can mark this in such a way that zk-zk-1 is equal to delta z let us make a simplifying assumption that the gap between any two is delta z it need not be constant but I am making a simplifying assumption that these are equidistant points, okay so at z is equal to zk, okay I can say that du by so the derivative first derivative I am going to approximate as uk plus 1, okay before that let me develop a notation and then we proceed to this so let us develop a simplifying notation that uk it corresponds to u at zk the dependent variable u at point zk is going to be called uk, okay this is just a simplifying notation that helps us to write this equation in a very simple manner discretization in a simple manner now let us look at okay so my du, right what is how do you locally approximate the derivative, right so this is my z bar earlier I talked about a point z bar this is my z bar what is this, this is u calculated at z plus delta z this is u calculated at u minus delta z, okay so this is my local derivative what is my second derivative so this is going to be approximated as uk plus 1 minus 2 uk plus uk minus 1 by delta z square, okay so with these two approximations with these two approximation this equation here this equation here will be transformed to see where do I want to enforce this differential equation at the internal points in the domain, okay so what are the internal points here if you go back here what are the boundary points z1 and zn plus 1 so I should enforce the differential equation at all the internal grid points, okay so I am enforcing this differential equation at all the internal grid points, okay how many equations I will get here n minus 1 equations I will get, okay in how many variables what are the number of variables what are the unknowns just look here my unknown is u1, u2, u3 in general uk, uk minus 1, uk plus 1 how many unknowns are there n plus 1 how many unknowns are there yeah so unknowns are u1, u2, u3, u4, u5 up to un plus 1, okay so this equation which actually was supposed to be enforced over the entire domain okay now I am enforcing only at a finite number of points I have discretized my original problem, okay so this is original problem was a differential equation what do I get here when I substitute this non-linear or linear depending upon what the differential equation is I will get either linear algebraic equations or I will get non-linear algebraic equations so original problem which was a differential equation got transformed into set of non-linear or linear algebraic equations depending upon how the original differential equation is so we end up solving this problem instead of the original problem, okay now there are 2 more equations required how do you get the 2 more equations boundary conditions, okay so at boundary I have a choice what kind of approximation I use I could use an approximation which is forward difference backward difference or I could use approximation which is because at the boundary point see here this derivative at a point requires a point before and point afterwards so if I want to use central difference here I will need a point on this side and a point on this side so there are 2 approaches right now I will just talk about 1 the second one we will talk later so just to end this lecture I will say that there are 2 more equations required to solve this problem so we generate this u1-u0 by delta z so I am using the forward difference here u0 0 is equal to 0 and f2 I am going to use backward difference here un plus 1 minus un by delta z un plus 1 1 is equal to 0 so these 2 equations together with these n minus 1 equations okay when they are solved simultaneously I will get an approximate solution of my boundary value problem okay so original problem which is a boundary value problem ordinary differential equation boundary value problem get transformed to set of algebraic equations linear or non-linear depending upon what kind of depending upon what kind of differential equation that you have at hand okay here so this is discretization you realize this we started the original problem in some space and then you actually solve a problem in the finite dimensional space is are we working with finite dimensional spaces now see u is actually discretized uz is a function actually a continuous function for my infinite dimensional space the approximate solution has been constructed by discretizing at finite at finite points this is a function at finite points so this problem has been converted from a infinite dimensional space problem to a finite dimensional space problem why it is computable it is computable okay how do you get a better and better approximation you take more and more points but you know whatever you do you will still have finite number of points you can take you know somebody might say I will take 100 points somebody might say no I will take 1000 points but remember now when you take 1000 points you have to solve 1000 equations in 1000 unknowns simultaneously okay and we will be doing these kind of things in this course do not worry so the next assignment is going to be solving at least 100 equations in 100 unknowns that is what you should get this confidence that you can solve as many equations but these are a finite number of equations non-linear equations how do I use this how do I solve this problem if it is non-linear how do I solve this problem Newton's method I talked about Newton's method yesterday's lecture right or in the programming assignment we are using Newton-Raphson method Newton's method so once I get this okay I still cannot solve it so I have to further approximate okay so it is a cascade of approximations not just because finally we know how to solve ax equal to b okay so we are using ax equal to b to solve this non-linear set of equations but this non-linear set of equations is arising from discretization of a boundary value problem so you can see the levels of approximations you have approximation then again you approximate because the approximate problem cannot be solved exactly okay so let us continue with we will now like tomorrow's class we will see some concrete examples okay of boundary value problems where we will take some differential equation discretize it and see what happens remember these equations are coupled equations you cannot solve them separately okay because for any point you know k plus 1 and k minus 1 appear in the equation so these are all tightly coupled and you have to solve them together you cannot solve them separately.