 Good morning. Today, we will study one of the most important lessons of applied mathematics and that is numerical solution of ordinary differential equations. In many courses and many textbooks of differential equations, you might find that the analytical solutions of differential equations are discussed first and then it is said that in all cases you may not be able to solve the differential equation analytically. In fact, in majority of cases you will not be able to solve and in such situations we must resort to numerical solutions. However, we prefer to precede the discussion of analytical methods by the solution of differential equations by numerical means. The reason is that this is most general and this is the way of historical development of the field also. That is first the initial attempt where in terms of finding the values of the solution function and it was later found that in certain situations the solutions can be expressed in terms of those form expressions in special cases. So, in those special cases where such an analytical solution is possible, we can do much more than getting just values, but it is the values of the solutions which have the prime place in terms of the determination of the solution of ODEs. So, we follow that methodology that sequence of operations. First we try to find the solutions in the most primitive manner in terms of numbers and then we will examine how sensible are the resulting numbers and then we will consider those special cases in which analytical solutions are possible. So, first we start with numerical solution of ordinary differential equations. This will be our last topic in the module of differential equations and this in the module of numerical analysis and it will be a bridge between the module on numerical analysis and the next module which is going to be on differential equations. So, this topic we will cover through three lectures starting from the current one. First we consider the straight forward problem of one differential equation in one independent one dependent variable one function one independent variable x and one dependent variable or function y unknown function which we want to determine. Now, this is a first order differential equation in y with an initial condition given that is at one value of x the value of y is given. So, this makes it an initial value problem the initial value of y is given at x 0 and then for all subsequent values of x that is greater than x 0 we want to find out the values of y. Further matter we could have asked for the values of y for x values which are less than x 0 as well it is equivalent. So, we want to determine y of x for x in an interval a to b with a being the initial value of x that is x 0. Now, this is in a way somewhat analogous to an integration problem why here we have got d y by d x and we want to find out y. So, it is in a way integration and therefore, you will find that the numerical solution of O d is have a lot of similarities with the numerical integration problem. Only major difference is that this side the d y by d x is not dependent on x only, but it is dependent on x as well as y. So, a first order differential equation first we put in this form in which we get the expression for d y by d x in terms of x and y. Now, if we are lucky if it just happens that this function turns out to be free from y then it is a straight forward integration. On the other hand if it is dependent on x and y then we cannot solve the problem just by an integration. So, in that case what we can do still is that we may say that we will take a very small step to begin with. So, starting from point x 0 y 0 we will ask first that just a little ahead of x 0 at x 0 plus 8 what is the value of y that is at x 1 what is y 1. So, we have x 0 y 0 a point starting point and we want the next point x 1 y 1 where x 1 is only slightly ahead of x 0 that is x 0 plus 8. Now, for that very small interval if we can consider this function not to change significantly then what we can say we can say that the value of that function we evaluate at x 0 y 0 and that derivative that slope we multiply with the small change in x that is 8 and that gives us the corresponding small change in y and that delta y we add to the current y 0 and get next value of y that is y 1. So, that gives us a point x 1 y 1 now we know that y of x 1 is y 1 and d y by d x at that point we will be able to find out from f of x 1 y 1 f is a known function. So, we go on repeating this kind of small steps up to x equal to b this is indeed a method and shortly we will see that this is a method with a given name and on that we can do certain analysis and so on. So, before proceeding for that analysis we have a quick look at the two kinds of methods which we can use one is a single step method in which only the current value only the current x 0 y 0 and the corresponding f is used at every iteration and there is another family of methods which is called multi step method in which history of several recent steps are used. So, that kind of a method is called a multi step method most of our discussion here will be limited to single step method. Now, resuming the discussion of this method this method is actually called Euler's method and at every step this is the methodology at x and y n evaluate the slope by this and for a small step h use that value of d y by d x multiply that with small change in x which is h and that you when you add to the current value of y you get the next value of y and this way repetition of such steps will construct the function y of x that is for x equal to x 0 x 1 x 2 x 3 x 4 etcetera you will find the corresponding values of y y 0 y 1 etcetera and those points in the x y plane will give you the trajectory and that will be the solution of the differential equation. Now, if we construct the truncated Taylor series first order truncated Taylor series on of this function then we will find that the expected error of this Euler's method turns out to be of second order that is order of h square because up to h we are actually retaining the change. So, up to from h square onwards there will be errors. So, leading error term will be h square over every such small step that means that from a to b as we go on taking large number of steps that is if we have divided the interval a to b into n parts each part of size h and so on. In that case these kinds of errors we will get accumulated and then accumulated error as we saw in the problem of numerical integration will be of one order higher one order lower that is up to linear error. So, these are at every step we will have an order h square error. So, accumulated over steps the error will be larger larger means lower order of h h is a small step. So, the total order of the error total error order will be order h which is bad because it is first order error error will be in the first order itself. So, that means that if you use Euler's method you are likely to have a lot of error first order error means that unless h is extremely small you will find that the error is significant and the result is not very reliable. Now, one question that we should ask at this point is that is the total error over say 10 steps like this will be the sum of errors over the 10 steps the answer is not. So, in general the total error may be larger than the sum of the individual errors or for that matter it could be smaller also it depends upon the differential equation under question. So, how that happens that total total error could be larger than the sum of errors or smaller and what can be done to improve this error that we will explore next. First of all consider this situation d y by d x equal to f of x y with an initial condition given at x 0 the value of y is y 0 this is the initial point initial condition and as we if we could solve the correct solution of the differential equation and if we could have got y as a function of x then suppose y as a function of x for that the graph is this shown by the curve c. Now if we take a small step h and move to x 1 then here now I will enlarge this and try to show in this enlarged figure you can see better that is this curve the top curve is the true solution. Now as we start from the given initial condition x 0 y 0 then with the Euler's method we will be proceeding along the current slope that is whatever is the slope of the curve that means along this direction and however this tangent cuts the vertical line at value x equal to x 1 that is x 0 plus h we will get that point. Now from that point in the next step of Euler's method the current tangent will be used and that means that along the current tangent that is a tangent here we will be proceeding towards this. Now note that first of all we have because we moved along the tangent we have left this true curve currently we are sitting at curve which is a this curve second curve from the top. Now since we will be moving along the tangent from this point in this step that is in the second step we will be leaving this curve also and we will be actually landing up at the third curve from top that is we will be leaving c in the first step reaching c 1 then at the second step we will be leaving even c 1 and reach c 2 the next curve and so on. Now note that the error in the first step is this y intercept from the true value at x 1 to this value where we reach and the error at the second step will be actually must be counted along this that is this is the true value for the second step because it was trying to track this curve and it has reached here that means the error is this. Now this error in the second step plus this error in the first step will actually be somewhere here that is second step error plus first step error and the actual error is from here to here which is larger than the sum of these two. Now the sum of these two can be found if you draw a line from here parallel to this tangent that is somewhere here. So in this particular example the error of the first two steps is actually smaller than the total step that is individual error error in the first step plus error in the second step compared to even this sum the overall error accumulated error in the two steps is even larger. So this way errors not only accumulate by simple sum but might even magnify in another example that could have been lesser than the sum. Now what is actually happening we find that this way at every step we will be leaving the initial curve and then we will be leaving the next curve and then we will be leaving the next curve and finally after three steps we would have actually got a got to a point which is the solution of the same differential equation with the initial condition y at x 0 equal to so far below the bottom curve wherever the bottom curve crosses that value of y. So this way this will be the total error in three steps so that shows that with first order error getting accumulated the total error over a large number of steps may be a bad thing in the case of Euler's method. Now what is the reason while exploring the reason for this we also ask a question that from x 0 to x 1 we moved during the entire h along the slope which was given at the initial point. Was it a good idea the slope changes from this point to that point if the slope changes from this point to that point then how good is the idea of moving all the time with the same slope as given in the initial point. If we could figure out some average slope then that would have given us a good move. Now with that in that intention the Euler's method is modified and then we have what is called modified Euler's method or Heun's method. Now in improved Euler's method or modified Euler's method what is done is that a first step is taken now the problem is that it would be nice to be able to take an average slope but then how do we know the average slope to begin with we know only the y value here and therefore the slope here at the end point or in the middle point of the interval we do not know the y and we do not know the slope. Now in improved Euler's method what we do from here we take one tentative step and reach q q 1 reach q 1 along the slope here after that we say that it was perhaps not a good idea to move from here with this slope only. Now let us calculate the slope here after we have got this tentative point at x 1 we have got some value of y 1 call it y 1 bar and then we evaluate the slope here from the same f of x y at this point we evaluate the slope. Then we say that along that tangent with that slope the new slope here we draw a line suppose that goes here at p 1 then we say that what is the difference between p and p 1 this difference. Now we say that if rather than with the slope here if we had started with the slope here and moved then where would we have reached that is rather than along this slope but with this new slope which we have now evaluated if we had moved then we would have reached q 2. But then q 1 is wrong because it gives complete weightage to the initial slope. Similarly q 2 would be wrong on the other side because that would be giving complete weightage to the final slope the true picture should lie somewhere in between. So, what about taking the average of q 1 and q 2 that is q that is given by improved Euler's method or Heun's method and this is the analytical formulation that is from the value x n y n take one tentative step and call that value of y at x 1 as y 1 bar y n plus 1 bar and then evaluate the step if we use that slope that is f of x n plus 1 y n plus 1 bar with that you will have the step as h into this. Now this is one proposal and h into this is another proposal. So, what about the average of them average will be h by 2 into this plus this. So, this is the point taken for this step. Now this also will have some error but if you make analysis based on Taylor series you will find that the corresponding error is second order over the complete domain over the complete interval a to b that is accumulated error that will be of second order. So, that makes sense because we have made two function evaluations at this point and at this point and we have got a second order method. Now this can be continued with four function evaluations we can land up at a fourth order method and actually there is a family of methods called Runge-Kutta method. So, second order Runge-Kutta method generally would go like this first estimate k 1 which is same as K 1 is estimate for delta y. Second estimate now here we say that we do not make the evaluation at this end point but somewhere in between that is in between x and x 0 and x 1 somewhere where that we decide by alpha and beta that is alpha step from here and beta step in the y direction. So, if we do that then we get this point where we can evaluate the slope function and based on that we work out the second proposal for the change in y that is delta y and we can say that we have got the first proposal from here and the second proposal from here what about taking a weighted sum of the two. Now in this particular case the weight w 1 and w 2 you will expect that w 1 plus w 2 should be 1 otherwise this will not make very good sense as a weighted sum and take that k. Now we ask what weight values we should use and for that we try to force agreement up to second order that is h term in the error and h square term in the error we try to force to 0 to get the appropriate values of w 1 and w 2. How do we do that? We say that based on this if we try to work out y n plus 1 that will be y n plus this k w 1 into this that is w 1 into h f plus w 2 into k 2 that is this w 2 into k 2. Now if we try to work out at this point Taylor series of this truncated Taylor series of this then we will get f at x n y n plus alpha h into first derivative with respect to x plus beta k 1 into first derivative of this function with respect to y plus higher order terms. Now if we simplify this and collect h f terms together and then h square terms together then we get this. This is the Taylor series of y n plus 1 which we are getting. Now at x n plus 1 the Taylor series value of the original function y at x n plus 1 without this formula that would be y n plus h into d y by d x which is this plus h square by 2 into d 2 y by d x square which will be this by chain rule of this function that is this and so on. Now when we compare these two when you compare these two we get the error and in that we say that we want the error to have 0 value up to the second order. That means first order error should be 0 first up to first order this should match and second order also this two should match. So, the first order terms here and here if you match then you get w 1 plus w 2 equal to 1 which you would expect and then as you equate the second order terms you get this also. There is alpha into w 2 should be from here 1 half and when you equate these two you get beta into w 2 is half. So, there are three equations here in w 1 w 2 alpha beta if you try to solve them four unknowns from three equations you can get three of the unknowns in terms of one remaining unknown w 2. Now depending upon which value of w 2 you select you may get slightly varying second order Runge-Kutta method it is a family of method. Now if you take w 2 equal to half and that will mean that alpha beta will turn out to be 1 then you get what we just now discussed that is improved Euler's method. On the other hand popular choice of second order Runge-Kutta method is w 2 equal to 1 which means that you take the first proposal as Euler's method and for the second proposal for delta y that is k you evaluate the function at the midpoint of the chord of the step that has been taken. At that midpoint you evaluate the function and thereby work out the second step second proposal for the step and then take the second step only not the first step at all. So, that will mean that w 2 is 1 and w 1 is 0 this has been used only as a stepping stone for evaluating this function and after that this is not used at all the actual step has been taken equal to k 2. So, this is second order Runge-Kutta method similarly with four function evaluations you can get a fourth order Runge-Kutta method. Now, this point onwards I am not trying to derive a proof, but without proof let us state this that if you work out the first proposal as based on the initial value just like Euler's step and second proposal of delta y that is k 2 as we have just now done and then the third proposal as k 3 which is this in which in place of this k you use k 1 you use k 2 the fresh one and finally, the last evaluation is made with full step h and k 3 and for delta y here you use the most recent estimate and then you have got four different proposals for delta y and take this weighted sum 1 by 6 2 by 6 2 by 6 1 by 6 sum is 1. Now, this particular arrangement of the four possible delta y's ensures that the error over and up to fourth order is 0. That means over a particular sub interval over a particular step h the error that the leading error term that you will get will be h to the power 5 that is only up to fifth order only from fifth order there will be error and that means over accumulated over several steps for the entire domain the error will be fourth order and that is why this method is called fourth order Runge-Kutta method with four evaluations of the function you get a fourth order method. However, you can work out higher order methods also in the Runge-Kutta family, but then for getting a fifth order Runge-Kutta method it is not enough to make five function evaluations. However, with six function evaluations you can work out a fifth order method that particular arrangement has some further made which we will be discussing a little ahead. Now, through whichever method whether it is a second order method or fourth order method you develop the solution of the ordinary differential equation. How do you figure out whether the resulting value is right that is what is the magnitude of the error which is really important in a practical situation how do you determine the magnitude of the error. So, this is an in particular whether the error is within tolerance for that purpose you can do two things one is you can make an additional estimate which will give you a handle to monitor the error and based on that you can develop further efficient algorithm that is whatever best you get with a particular order of method. So, if you have additional estimate then you can develop an estimate of the error and on the same basis you can develop further efficient algorithm that is you can reduce the error as well as you can know how much is the error. So, it is very much like what we did with Richardson extrapolation in the case of the numerical integration process. Suppose, in a particular interval from x n to x n plus h you have developed one particular estimate of the value at x n plus h that is y n plus 1 1 estimate. So, we are showing it with this superscript 1 and that is the correct value plus some error. Now, if you have used the fourth order Unger-Kutta method then the leading error will be fifth order plus higher order terms. Now, what you can do is that you can now evaluate the same y n plus 1 not with one step of size h, but two successive steps of size h by 2 then the corresponding second estimate will be y n plus 1 plus two such steps. So, twice c into this coefficient is same constant is same c into the h by 2 to the power 5, because every step is now of h by 2 size. So, in the first step you accumulate c into h by 2 to the power 5 in the second step also that is the estimate the two half steps will give you this error. Now, note that here you have got two equations in the true value and c h you know and these two estimates you have got through the R K 4 formulas Unger-Kutta fourth order formulas. So, these left sides you know and h you know so you have got two equations in y n plus 1 and c these things you do not know. Now, two equations into unknowns linear equations into unknowns you can solve. So, as you solve that you find that difference of the two estimates will throw out one of the unknowns that is y n plus 1 and give you this is c h to the power 5 and this is c h to the power 5 by 16. So, difference is 15 by 16 c h to the power 5. So, this gives you that difference of two estimates and what is the error here? Error here is c h to the power 5 by 16 that is 1 by 15 times this. So, you have got an error estimate that is error estimate of the second value. Obviously, this value is better than this because this is with one step of this size this estimate is based on two half steps. So, this is a better estimate and in this the error is c h to the power 5 which turns out to be from here related to the difference of the two estimates. So, difference of the two estimates will be giving you 1 by 15 of the difference of two estimates is c h to the power 5 by 16 which is the error estimate here. So, if you use that or if you multiply this with 16 multiply this with 16 and take the difference then also you get the same result. In any case between these two equations you can solve for c h to the power 5 by 16 which is the error estimate and the true value or the best estimate till now that is y n plus 1. So, this is the best available value out of these two estimates. So, this way you get two things one is an estimate of y n plus 1 which is better than both of these which you get like this and two you get an error estimate of this that is c h to the power 5 by 16 which is the difference of these two divided by 15. So, now you can say that if that is within your required accuracy within your tolerance. So, if you find that the delta that you are ready to tolerate is quite large that means if delta is quite large compared to the epsilon value which you are ready to tolerate then that will mean that the current step size is too large for accuracy. So, in that case you subdivide the step further and take smaller steps. On the other hand if the delta that you get is much much less than epsilon that means what that means the error is within tolerance, but perhaps you are taking two small steps even larger steps would be alright to solve the problem within the required accuracy, but then you have already determined. So, many function evaluations you have already made that means computational step you have computational resources you have already spent. So, now you cannot do anything further to save that computational resources spent. So, that is why the at the starting you should step you should keep the step size not too small reasonable step size not too small and if you find that at some location the delta that is coming with this estimate is too large then that particular step you subdivide further. So, start with a large step size not extremely large, but reasonably large and then keep subdividing intervals whenever you find delta is greater than epsilon. So, that will mean that suppose you want to solve this from this initial point to this final point. So, this is x 0 where the value of y is given this point. So, what you can do you can say that initially this entire interval from x 0 to say final x this entire domain over which you want the solution of the ODE. So, you can say that I will try to solve it with four steps. So, this step you find that you develop one estimate and another estimate based on that you get a third much better estimate and an error estimate. Now, find out whether that error estimate is acceptable if it is acceptable then take the best value that you have got till now. Similarly, you try to do it for this step and suppose here again from this point you find that you have got one estimate second estimate and based on these two you get a third much better estimate and at the same time an estimate for the error. Now, suppose in this case the error turns out to be too large then you say in this location this step is going to turn out to be too large. So, you subdivide and do the same thing for this and this. So, now over this small step you operate and get this next whatever. So, whenever you find that the error estimate shows that too much error is coming then subdivide the interval further otherwise continue. So, this ensures that you keep on marching fast over those segments of the function where the function is smooth and those segments where you get quick changing first changing situation there you make small steps and where to make the small step that this delta will tell you. So, this is what is called adaptive step size algorithm. Now, another interesting thing another interesting feature of Runge-Gutta method is that as we discussed just now that with six function values we can get a fifth order formula fifth order Runge-Gutta formula. Not only that using a different combination of those six function values you get another formula which is a fourth order formula. Now, with this situation with six function values a fourth order formula is found embedded in a fifth order formula and this helps you in two ways that is you get two ways independent estimates of the value of y n plus 1 and an error estimate based on the same six function evaluation and this is called Runge-Gutta fail work method and most of the professional implementations use that R K F 4 5. However, if you are a beginner in this kind of implementations then and if you want to implement your own solution of O D routine then I would suggest that R K 4 itself fourth order Runge-Gutta method itself with these straightforward formulas is also going to be quite effective for your purposes with adaptive step size which we just now discussed. However, if you call a library sub routine then most likely you are going to get an R K F 4 5 routine available in most of the important library libraries Now, till now we have been discussing a single first order equation now, but then whatever we have been doing is directly applicable to a first order vector O D. If x remains scalar but y becomes vector and therefore, f of x y becomes vector function even then all the operations that we have been conducting can be still conducted that is why whatever we have been discussing in the context of a single first order O D can be immediately extended to a vector O D that means a system of first order O D is like this. Of course, in that case the initial condition has to be available for all the y's. So, the entire y vector has got an initial value initial vector y 0. So, this can be solved with the help of same formulas only with the change that y's become vector x remains scalar still. However, if you have a higher order differential equation then an n th order ordinary differential equation can be always converted into n first order ordinary differential equation for that the way is to develop the equations in the state phase. That means that you first define a state vector including the unknown function its first derivative second derivative up to n minus 1 s derivative. If the O D is of n th order that is if the ordinary differential equation involves derivatives up to n th order then the state vector will involve derivatives up to n minus 1 th order 1 order less. So, this vector completely defines the state of the system that the differential equation is trying to describe. And then you try to work out d z by d z d z by d x partly from this definition and partly from the differential equation how you see here z 1 is y. So, d z by d x is going to be y prime which is z 2 then y prime now z 2 is y prime that means d z 2 by d x will be y double prime which will be next entry that is z 3 that means from here to here except the last one z 1 prime will be z 2 z 2 prime will be z 3 and so on till z n minus 2 z n minus 1 prime will be z n this one this is z n z n prime will be the n th derivative of y and that you get from the differential equation. And initial conditions also will get this kind of values whatever is the values given for y y prime y double prime etcetera up to the n minus 1 th derivative. So, those will form the initial condition for z and then you solve the system with the same r k 4 formulas etcetera. So, in general if you have got a number of ordinary differential equations each of order higher than the first then also you can follow the same procedure and break down the entire system into a large number of first order differential equations. In general if you have got a system of higher order ODEs with higher order derivatives of the unknown functions y 1 y 2 y 3 y 4 etcetera being n 1 n 2 n 3 n 4 etcetera then you can cast the entire system into the state space form with the state vector of dimension n 1 plus n 2 plus n 3 plus n 4 etcetera. Let us take an example. So, this state space formulation is directly applicable when the highest order derivatives can be solved explicitly. For example, let us take this example this is a system of two ordinary differential equations in two unknown functions x of t and y of t which we want to solve. So, you want to solve for x of t and y of t. So, two unknown functions so we have got two differential equations and what is the highest order of derivative of x that is involved here up to second order. Similarly, what is the highest order of derivative of y that is involved up to third order. So, in the state space we will have x and d x by d t this will not have in the state space. That is if the if up to the n th order derivative is involved then up to the n minus 1 th order derivative will appear in the state vector. So, d 2 x by d t square is there as the highest order derivative of x. So, in the state vector we will have up to d x by d t. In y we will have we have the third order derivative of y as the highest order derivative. So, the state vector will have y d y by d t d 2 y by d t square that is up to second order derivative. So, the state vector will be x d x by d t this will not be there the highest order derivative will not appear. Similarly, for from y we will have y d y by d t and the second derivative of y not the third one because the highest order derivative will be actually found will be solved from the differential equation itself. Now, see what you have got three derivatives of three of the components here will be trivial because z 1 is x. So, z 1 prime will be d x by d t which happens to be z 2. So, z 1 prime is z 2. Similarly, z 3 is d y. So, z 3 prime will be y prime that is d y by d t which happens to be z 4. Similarly, z 4 is d y by d t its derivative z 4 prime will be second derivative of y which happens to be z 5. So, that is it. So, these three derivatives are trivial we need to find out what is z 2 prime and what is z 5 prime. So, that you find from here from this differential equation from the above first differential equation you can write y z 2 prime minus 3 z 4 z 2 square plus twice z 1 z 2 square root of z 5 plus 4 equal to 0. Just match term by term. We have written this differential equation y is in place of y actually we can write z 3 in place of y we can write z 3. So, just go on this equation. Even reading this first differential equation in terms of these as z 1 z 2 z 3 z 4 z 5. So, y is z 3 look up there then this second derivative is z 2 prime that is prime of this minus 3 then d y by d t that is z 4. So, z 4 here and then we have got d x by d t whole square that is z 2 square plus twice z 1 twice z 1 and then d x by d t which is z 2 z 2 and then square root of this which is z 5. So, square root of z 5 plus 4. So, from this differential equation can we not find z 2 prime we can find z 2 prime we take this whole thing on the other side of the equation divide by z 3. So, as we take this whole thing on the other side of the equation we get this and then we divide this whole thing with z 3 of course, this will be valid in a domain in which z 3 does not become 0. So, z 2 prime turns out to be a function of z 1 z 2 z 3 z 4 z 5 and we have already found that z 1 prime is z 2 and we have already found that z 3 prime is z 4 z 4 prime is z 5 and z 5 prime we need. So, for finding z 5 prime that is the derivative of this which is here actually. So, we do a little further work in this box here. The second differential equation from here we write in the language of z from here. So, e to the power x y e to the power x y. So, x is z 1 and y is z 3 into d 3 y by d t cube. So, that is z 5 prime which we want to determine is equal to minus or we can take is equal to and write the other things with the change sign z 3 z 5 to the power 3 by 2 minus 2 z 1 minus 1 minus e to the power minus 3. Do we get that plus. So, just read of this e to the power x y there e to the power z 1 z 3 into this third derivative which is the prime of z 5. So, z 5 prime is equal to. So, these things we will take on the other side y that is z 3 into d 2 y by d t square to the power 3 by 2. That means z 5 to the power 3 by 2 this also goes to the other side minus 2 x minus 1 minus 2 z 1 minus 1 plus e to the power minus t. So, that is here. So, that means that we can solve for z 5 prime as now you see the prime the derivative of the vector function z that is z 1 prime z 2 prime z 3 prime z 4 prime and z 5 prime. All these 5 are available in terms of the 5 z and t. That means together we can write the vector z prime as this function with these 5 components in the same standard form. So, that is this form and this is the state space equation. So, this is how we can formulate a system of higher order ODEs into the state space form and then use the same numerical methodologies that we discussed for the scalar single function. Now, apart from these single step methods that we have discussed till now there is a family of methods for which use several earlier steps results as well. For example, in single step method every step of h is taken as a brand new initial value problem in which the current value is there and from there we try to develop the slope function and take one step. Now, can we not try to capture the trend by learning from the history of several recent steps that is the kind of thing that is done in multi step method. So, typical multi step formula for solution of ODEs is this that is in which you take y n plus 1 that is the next value of y as the current value plus h into a sum of several function values a weighted sum of several function values c 0 into f at the next point plus c 1 into f at the current point plus c 2 into f in f of the at the previous point plus c 3 into f at the yet earlier point and so on. So, this is a typical multi step formula. Now, based on what do we determine the coefficients c 0 c 1 c 2 c 3 etcetera we say that we want the leading error terms h h square h cube h to the power 4 as 0 0 0 0. So, as a leading error terms are forced to be 0 that gives us a system of linear equations in the coefficient c 0 c 1 c 2 c 3 and those coefficients are used in a multi step formula. Now, you will say that we cannot use a multi step formula at the first step because at the first step all that we have is this starting from there x 0 y 0 before that there was no step no function evaluation. So, that is why a typical multi step formula can be used as a multi step formula only after the single step results have been accumulated. So, they have to be started with single step formula method single step method and after some steps have been developed then only this kind of a formula can be used. Now, there are two kinds of multi step formula one is explicit first c 0 is 0 that is the next value y n plus 1 is explicitly available in terms of the current and old values. On the other hand if c 0 is not 0 then the corresponding method is called not an explicit method, but an implicit method that means y n plus 1 has been expressed in the form of a function which involves y n plus 1 itself and from that to determine y n plus 1 we really have to solve an equation in y n plus 1. So, that kind of a method is called an implicit method. So, it is of course not explicit. So, it is difficult to evaluate requires a solution of an equation, but such methods have better stability and so far as the issue of stability is concerned in the next lecture we will discuss the issue of stability in more detail and talk of implicit method in comparison to explicit methods even in single step formulas the kind of formulas that we have been discussing so far. And the most famous most popular multi step formula is for the adams bashforth molten method in which there is a solution predicted using the adams bashforth adams bashforth formula and the solution is corrected using another formula which is the molten adams molten formula. So, this combination of a predictor formula and a collector formula by which you first predict and then make a correction. So, that kind of a predicted collector formula is used in multi step method and if you need multi step method then you can look up further literature on this kind of method. So, for the time being we stop here with this little summary over what we have discussed in this particular lecture. One is the basics of Euler's and Runge-Gutta methods and the second is the one of the very important professional practice of step size adaptation for really non-linear problems in which you use the step size in an efficient and effective manner without wasting too much of computational time. And the third important thing in this lesson is the state space formulation of dynamic systems with the way to reduce higher order differential equations or their systems into larger number of lower order larger number of first order differential equations and then develop the state space formulation in this manner which can be then numerically solved. So, this state space formulation is a necessary precursor to any numerical solution of ordinary differential equations. In the next lecture we will consider advanced issues of ODE solutions. We will discuss stability analysis and the utility of implicit methods and their specific usefulness in the case of step differential equations and then we will continue into discussing boundary value problems in ordinary differential equations. Thank you.