 So, in the last lecture we were looking at how to analyze convergence of non-linear procedures for solving non-linear algebraic equations, iterative procedures. And we said that in general we could write any iterative method for solving non-linear algebraic equations as a one equation. See I want to solve for f of x equal to 0, x belongs to R n and f is a n cross 1 vector, this is the n cross 1 function vector. Any iterative method to solve this problem numerically can be written as x k plus 1 is equal to g of x k. Well it can be, so the old guess generates a new guess and this process is continued till difference between two successive solutions become negligible or norm of f of x goes close to 0. If you look carefully this is a non-linear difference equation, it is a non-linear difference equation. The index here is iteration index k. So, the new guess generated from the old guess g is the transformation. I showed you that all the methods that we are looking at iterative methods can be expressed in this form. Now, just like we had conditions for analyzing linear difference equations, earlier we had looked at equations of this type is equal to b x k and for this particular case we had derived necessary and sufficient condition for norm of x k to go to 0 as k goes to infinity. In this case we had a very very powerful result that is spectral radius of b is strictly less than 1. This was the situation for the linear difference equation. We had got this kind of a generic form while analyzing iterative methods for solving linear algebraic equations and this we could derive a very very powerful result here based on the Eigen values of matrix b. We wanted all Eigen values of matrix b to be inside the unit circle. Now, coming to non-linear equations, it is not possible to prove a so strong result. We can only give sufficient conditions. It is not possible to come up with necessary and sufficient condition for a general non-linear difference equation of that form. We have to come up with some kind of local condition. So, these local conditions I described through contraction mapping theorem or contraction mapping principle which forms the foundation of analyzing iterative schemes. And one special lemma that we saw was the operator g or which maps. So, g is something g maps a unit not a unit ball a ball around x naught of radius r 2 where r was a special radius which should be greater than or equal to a certain number that we had defined yesterday. If g is a mapping which maps a ball of radius r into itself then and if g is a contraction map. So, one simple way of finding out whether g is contraction map or u was to see whether dou g by dou x was strictly less than 1 or less than or equal to theta which is less than 1 for all x in. If the partial derivative of g with respect to x as a norm any induced norm strictly less than 1 everywhere then we know that map g is a contraction. If map g is a contraction then in neighborhood of x naught of radius r we were assured of existence of a solution we are assured that any sequence starting from any point in this region would converge to the solution. So, solution of this problem is x star is equal to g of x star x star is the solution and if this condition is met everywhere in this ball then it is a sufficient condition to say that any sequence generated by this difference equation will converge to this solution. Solution is x star is equal to g of x star just to draw the parallel I am writing this just to draw a parallel. We had a sufficient condition here that if norm of b is strictly less than 1 then also this condition holds that is x k goes to 0 as k goes to infinity norm x k goes to 0 as k goes to infinity. So, we said this is a weaker condition than this necessary and sufficient condition, but this condition helped us to analyze to come up with diagonal dominance and all kinds of other things which were other theorems which were used to analyze iterative schemes. Likewise analogous to this when I come here this contraction mapping principle tells us very very important things one is that if g is a contraction map g is a contraction map if it is local derivative has see if you take g of x to be b x then local derivative of g with respect to x will be matrix b and any induced norm of matrix b being strictly less than 1 is the condition that we are looking for there. So, they coincide and then this particular equation well only difference there was the solution or the point where we wanted to reach was 0 0 0 origin in this case we want to reach a solution x star is equal to g of x star. It is possible to make everything in terms of 0 0 0 if you redefine or if you shift the origin to x star then you can make the two problems almost equivalent, but that is not so important it is just matter of shifting the origin what is important is that there is a analogous sufficient condition here for non-linear difference equations. It does not help us here to look at the spectral radius of this matrix we it does not help here for the reasons which are difficult to explain as a part of this course, but we have to use only norm and a norm if the induced norm if any induced norm is strictly less than 1 in some region then you are guaranteed that there exist a solution to this difference equation in that region. The solution is unique the third point which was very very important start from any initial guess you will converge to that solution start from any initial guess in that region you will converge to the solution x star is equal to g of x star. So, these are very very important findings of this particular theorem in general it is more difficult to apply this theorem for a complex real problem nevertheless it gives us some insights. It gives us insights for example you can try and make the sufficient conditions meet by ensuring that dou g by dou x has induced norm less than 1 you can try to do this if there is some problem in some solving some non-linear algebraic equations we can see these are sufficient conditions remember that if these conditions are violated even then the convergence can occur these are not necessary conditions, but if this happens convergence will occur just like in this case when we were talking about linear algebraic equations. If norm of b is less than 1 spectral radius is less than 1 it is a sufficient condition but if norm of b is greater than 1 even then convergence can occur because convergence depends upon the spectral radius, spectral radius can be less than 1. Similarly, contraction mapping principle gives us a sufficient condition for convergence it is not a necessary condition but if you meet the sufficient condition you are guaranteed to converge. So, this gives at least some handle to understand how the convergence occurs from that view point this is important. So, those of you who are solving large set of algebraic equations as a part of your research M Tech or PhD and hit into problems you should look at the norm of the Jacobian I mean at least that much you should remember look at the norm of the Jacobian that will give you some try to tweak try to see whether you can make the normal Jacobian less than 1 then you know you have good chances of convergence. Just to illustrate this idea of contraction map I will just give you one example here I want to solve simultaneously z plus these are two non-linear algebraic equations which I want to solve simultaneously. So, if I write this minus this equal to 0 and this minus this equal to 0 then this is f of x equal to 0. There are two functions f 1 x f 1 z y and f 2 z y I want to find out a solution for this particular problem. I am formulating an iteration scheme here z k plus 1 is equal to 1 by 16 minus 1 by 4 y k square and I have just formed one iteration scheme this is not the only way to form iteration scheme I am just showing you one possible way of forming the iteration scheme. So, this is a Jacobi type iteration scheme what would be the Gauss-Seidel kind of iteration scheme if I were to use z k plus 1 here it will become Gauss-Seidel type iteration scheme this is a Jacobi type iteration scheme. Now, what I am going to do here is so I have this scheme which is y z is equal to g of y z where g is this right hand side function and then I am considering this unit ball I am considering unit ball let us say my x naught my initial guess is 0 1 and no no one minute my initial guess is x 0 is equal to 0 0 and I am considering this unit ball of radius 1 in the neighborhood of 0 0. So, I am looking at now what is what is this infinite norm g of I am taking some point x i and some point x j x here is x consist of y and g x is the vector consist of two elements y and g now I am looking at I am looking at this well what is the norm what is the infinite norm infinite norm is maximum of the absolute value of the two elements right see what I am doing is I am taking g x i minus g x j it has two elements I am just taking the maximum of these two absolute values will be the norm I am just using definition of infinite norm nothing else just this is just definition of infinite norm. So, you can show that this is less than or equal to max of I am skipping in between steps you should fill them up just go back and look at why this step comes from this you can prove these inequalities that is this particular difference is infinite norm of this difference is less than x i minus x j half times x i minus x j. So, actually the contraction constant is half see I just wanted to show that in this particular case you can show for this particular case you can show that well I am using here the fact that the elements are drawn from the unit ball elements are drawn from the unit ball. So, that is why this steps have been written and essentially using this inequalities what you can show is that g i minus g j using this inequalities you can also do analysis using the derivative of this and taking its infinite norm. You can also do analysis using derivative of this right hand side Jacobian matrix and infinite norm of the Jacobian matrix that analysis also possible. In this particular case we have found that if you apply g on any x i and x j then this inequality holds if this inequality holds what it means is that see this constant on the right hand side is less than 1. So, this is strictly less than 1. So, this g map is a contraction if g map is a contraction I am guaranteed that there exist a solution in this unit ball the solution is unique and starting anywhere in this unit ball unit ball this is with reference to the infinite norm. So, it will be a square not it will it will look like a square we have seen this how does the unit ball look like in different norms. So, starting from any initial guess within this the iterations will converge to the solution. So, this we are guaranteed because we are able to prove this inequality here for this particular x is equal to g of x. So, what is important here is that just looking at or just developing this inequality this is infinite and infinite norm just developing this inequality I am guaranteed that a solution exists in the ball I am guaranteed that I start from anywhere and I will reach the solution I will reach the solution and. So, this iteration scheme is going to work that is what I know from this analysis just right now just do not bother about this in between steps assume that this sequence is true because our aim is not to do this algebra you can work on this algebra later more important is that by doing this algebra I can show that infinite norm of g i minus g j divided by x i minus x j. So, for any i j I can prove this. So, I take any two points in this ball apply g on both the points the new points will have a distance which is closer than the original two points that is the main thing if that happens we are assured that a solution exist we are assured that starting from x naught will reach the solution moreover from any initial guess in this region if we start will still reach the solution that is the important point. It is difficult to do this analysis for a very large scale non-linear system. Nevertheless it is important to get this insight that how does one look at analysis of convergence of you know iterative schemes for solving non-linear algebraic equations because most of the times most of the times will be actually dealing with non-linear algebraic equations large scale in your work computational work because most of the chemical engineering problems 99.99 percent of them are non-linear problems reactions occur heat transfer occurs then turbulence and all these things will make the life very very complex. We have to work with set of non-linear algebraic equations what is it that governs the convergence we can get some clues if you look at if you can if you can show that the iteration scheme that you have formed actually is a contraction map difficult to show in general for a large scale system but this does give you insight which is very very important that is what you should carry. I want to stop here I do not want to get into too much details in the notes I have given some more detailed discussion on Newton's method. So there are special theorems for convergence of Newton's method and more than the proof and the theorem statement I have tried to give some qualitative insights as to how to interpret those theorems okay this go and I have not included the proof the proof can be found in any of the textbooks on non-linear systems like Reinhart and Ortega is one of the very well known textbooks. So you can find proofs there but the interpretation is quite important as to how do you make convergence occur. So typically if you have formed a iteration scheme well in this case I worked with directly worked I did not take a derivative but you could also try to see for this particular system you can work this out you can try to see whether dou G by dou x infinite norm if this is strictly less than 1 in the region of your in the region where you are trying to operate or trying to solve the problem or dou G by dou x 1 norm is strictly less than 1 okay. If these conditions are met then we are guaranteed that the solution exists and we will reach the solution okay. So these are some some why why why infinite norm and why one norm because they are easy to compute infinite norm and one norm are easy to compute okay other norms like two norms will require eigenvalue computation other than that one norm and infinite norm are easy to compute. So you can quickly make a judgment what is going wrong when you are solving the problem okay. So this brings us to an end of methods for solving non-linear algebraic equations we have looked at different concepts we have looked at how to solve them using different algorithms we just briefly test upon idea of condition number and also we very very briefly test upon the idea of convergence of iterative schemes we haven't gone deep into it but at least you know about you know what is the tool or what is the you know machinery that is used for actually looking at this problem. Let's move on let's move on to solving ordinary differential equations initial value problem. So now what I want to do next is before I proceed again we go back to our global diagram okay. So our global diagram was so we have this original problem then we use approximation theory to come up with transform problem. So we have been calling it transform computable forms and then we said there are four tools one is ax equal to 0. So this tool set we have which will be using and the other tool set was f of x equal to 0. So solving non-linear algebraic equations solving linear algebraic equations this is the second tool set that we have the third tool set that I am going to look at is ODE IVP because in many cases the transform problem is an ODE initial value problem okay even the I will talk about a method later on how do you transform a boundary value problem into initial value problem actually not just one initial value problem a series of initial value problems which are then solved iteratively and the third and the fourth tool is stochastic methods but we are not going to get into this. So right now we have done this how to solve ax equal to b we looked at many many many methods we looked at many issues that are associated with this we have looked at f of x equal to 0 and now I am moving to ODE IVP all this after all is going to give us approximate solution okay this is going to give us an approximate solution to the original problem. So moving on to solving ordinary differential equations initial value problem okay so general form of this the general the types of equation that I am going to look at is of this type dx by dt is equal to f of xt okay where f is the function vector and x belongs to Rn okay what I am what I am given is apart from this differential equation model I am also given initial condition I am given initial condition at time equal to 0 now before I move on let me explain one notational difference that we will have in this case if we are dealing with vectors we will have to deal with three different attached indices with the with the vector so suppose x is my vector okay here at element of the vector will be given by x i okay this notation we have been using even earlier bracket k will indicate kth iteration okay now additional complexity comes in we have time so time will come here so there are three things attached to the vector in some cases you will have ith component of the vector okay you will have time t appearing here okay and you may have kth iteration in some cases we do not need i and k we just might work with xt xt means vector x at time t okay vector x at time t so now a third dimension comes into picture here when you write in the notation okay sometimes there are schemes which are iterative and you need index sometimes you need to refer to ith component so you need x i and t is time okay now if now what kind of equations I am worried about what kind of equations that I am going to look at you might say that well what is written here is only a first order vector matrix equation right dx by dt is equal to f of x I am writing only a first order equation only first order derivatives and in your engineering problems you have often come across models which are second order third order fourth order and then when you did your first course in in the no differential equations you had nth order differential equations right and then you had methods of solving nth order differential equation so why am I doing things only for the first order differential equation though the difference here is a vector differential equation earlier you are looking at scalar differential equation what I am going to show that any nth order differential equation can be converted into n first order differential equations okay so this form which I have written here is very very generic very very generic okay so let us begin by looking at this conversion so let us say you have this let us say I have this differential equation in the scalar variable y so y is a scalar y is some mass fraction or y is some temperature or whatever whatever is the case you have some differential equation let us say this is nth order differential equation in general a non-linear differential equation we do not know I am just writing a generic form could be anything this is in one variable and independent variable is time okay so what I am going to do now I am going to define new state variables okay so my first state variable and when what I am given together here to solve this problem say initial value problem so what do I need to solve this problem I need a differential equation and I need the initial conditions initial conditions are given for y0 dy by dt at 0 so we are given initial condition we are given initial y0 initial derivatives up to order n-1 these are required to solve this differential equation with this differential equation together with this initial condition will be initial value problem right solving ordinary differential equation initial value problem this is this is what I get okay now what I am going to do now is what I am going to do now is to start defining a new state of variables okay so my new variable x1t is equal to yt okay x2t is equal to dy by dt okay x3t is equal to d2y by dt square up to x and t is equal to dn-1y by dt n-1th derivative I am defining new variables x1 to xn is equal okay now you can see that these variables are related through first order differential equations okay I can very easily say that dx1 by dt is equal to x2 tx2 by dt is equal to x3 so I have such n-1 equations so this is my equation number 1 equation number 2 and this is my equation number n-1 I have n-1 such one relationships between the variables okay all of them are first order differential equations okay the last one is now just the equation that we have okay so the last equation nth equation okay this is dxn by dt dxn by dt is this is nothing but d by dt of dn-1y by this is this is my definition this is my definition okay this is equal to fx1 x2 xnt I have an nth order differential equation which got converted into n first order differential equations this is my first equation second equation n-1th equation and the last equation came from the original nth order differential equation okay x1 x2 x3 xn are the new state variables that we have defined okay so what I have actually done is a scalar nth order differential equation I have converted into n first order differential equations in new variables okay so if I have nth order equation I can convert it into n first order equations okay if I have two simultaneous equations one nth order in one variable other mth order in other variable first one will give me n first order equations okay second will give me m first order equations you can stack them together into a bigger vector you will still get this form so this is a very very generic form I am not doing any compromise any nth order equation or any set of nth order equations nth mth order equations can be combined into finally this form this is a very very generic form so do not worry about why are we looking at only first order vector differential equation okay so all the books on advanced books on nonlinear differential equations will worry only about this generic form because anything can be converted to the generic form that is the first thing to understand so all the methods that we will develop are for this if you have nth order equations okay you know how to convert them okay you know how to convert them into n first order equations and write it like this so what will be f of in this particular case what will be the f vector let us go back and write that see in this particular case in this particular case my f vector after the transformation actually my equations are d by dt of x1 x2 x3 xn is equal to x2 x3 xn okay and f x1 x2 xn t this is my f of x this is my f of x okay this is the transform problem this is my f of x and I am given the initial condition okay so I am given initial condition x0 which is whatever this is y0 d by 0 by dt all these are given to me right this is my x0 this is my x0 this is given to me okay this is my f of x this is my f of x the original equation will appear as one scalar non-linear function in your function vector this is my function vector okay and this is the transform problem so I do not have to worry about nth order equations I am not going to do separate methods seen in the first course of differential equation you have second order differential equations one chapter on second order differential equations then you look at you know nth order equations we are not going to separate we are just going to look at n differential equations which are coupled okay if you are trying to solve dynamic trying to do dynamic simulation of a chemical plant there will be thousands of differential equations which are solved simultaneously together okay in fact they might be differential and algebraic equations not just differential equations so we are worried about right now to begin with solving large number of differential equations simultaneously together in one shot that is my aim okay and this form is very very generic applicable to any set of other way of getting these kind of equations we have already seen where do you get these kind of equations in problem discretization where did we find them finite difference method orthogonal collocations of partial differential equations that involve time and space we discretize in space we got differential equation in time we got n differential equations they were first order okay so if those are of second order you can convert them into first you know two first order equations all that is possible that is that is not difficult so converting nth order equation into first order equations is not a problem we are going to look at the generic form this could be arising from any of the sources it could be arising from the one which you have done right now it could be arising from discretization of a PDE okay it might be arising from some other some other context the context is already you know we have we already have studied about in what context this kind of problems will come we will look at only how to solve this abstract form okay of vector differential equation the other thing which you might worry about is that where does this time t come into picture okay most of the times the differential equations that you get as an exercise I have given you to solve differential equations for one particular system and I have given you a program which solve differential equations for a CSTR okay I suppose you remember to submit assignment soon that equation is of this form dx by dt is equal to f of x u there are some free variables x are dependent variables and there are some free variables okay like feed flow coolant flow coolant temperature inlet concentration all these are this u variables so in that particular problem CSTR problem x corresponds to concentration of a and temperature and u corresponds to inlet flow rate cooling water flow rate inlet concentration cooling water cooling water temperature at inlet and so on so these are the free variables but if you go back and look at the problem statement these manipulated variables or input variables have been defined as a function of time say this is sinusoidal this is whatever we have defined these as some functions of time okay once these are given as functions of time we can substitute them here okay as some function of time and then once these are these are specified once these are specified functions of time then only we can solve the initial value problem and for that particular for those specified functions of time this problem is then transformed to dx by dt is equal to f of x t because u will be function of only time some specified function of time okay a RAM function step function sinusoidal function or whatever random function whatever you want to study the dynamic dynamics of the particular system for so you are specified these free inputs and then this becomes a problem which again is this this generic form okay so these parameters or these input variables we assume that we are we already know them okay and then we want to solve the problem for the known inputs how does the dynamics evolves in time that is what we want to solve okay so that is why we are looking at in general dx by dt is equal to f of x t okay how this is specified as a function of time how these are specified how these are specified as a function of time let us not worry about that right now it could be an operator who is giving this values it could be a controller which is finding out these values it could be some environmental conditions which define the cooling water in the temperature okay we do not bother about that right now we want to solve the problem when this is specified how do you how do you actually find out how do you find out x as a function of time so I want to find out given this given these input trajectories in time I want to find out x trajectory that is concentration trajectory starting from time zero to whatever finite final time you want and temperature trajectory as solution of this problem solution of this problem is going to be not one vector see when you are solving non-linear algebraic equations you got one vector as a solution fixed point right now the solution is going to be a trajectory in time okay it is going to be a trajectory in time trajectory in time over the finite time if you are solving over finite time or whatever t goes to infinity if you want to look at now linear equations linear differential equations of this type you probably have already looked at in some other course wherever we need them we will visit them while those of you who have not done the other course on analytical methods in chemical engineering I will just briefly mention those results which we need here but we are going to look at the problem when this f of x on the right hand side is non-linear not when it is linear okay that is very very crucial we will use the results for linear later on to get some insights into the convergence properties under what conditions the methods that you have proposed will converge that is where we will use some linear system results but in general what we are going to look at is methods for solving non-linear ordinary differential equations given initial conditions how do you get trajectories in time or it could be trajectories in space okay we have seen that for example method of lines for you know converting Laplace equation it you discretize only in one spatial direction the other one is treated as a differential equation so you get a set of differential equations in time or space so you want to integrate these differential equations so T here in general need not be time alone T here treat it as independent variable in some context it could be space so maybe I should write a generic form that neta so neta is some independent variable okay could be time on space depending upon the context and initial condition at neta equal to 0 is given okay and you want to integrate this set of differential equations the way we are going to proceed is will briefly peek into the issue of existence of solution very very briefly and then move on to the different methods of doing numerical integration okay again what is going to help us is Taylor series approximation and polynomial approximations okay so we are going to meet our old friends Taylor and Weistras again and use them repeatedly to solve these problems okay what I want to stress here is that the same ideas are used again and again to form the solution methods okay there are few fundamental ideas which if you understand those ideas and if you know how to apply them you can almost do everything from scratch okay same idea is repeatedly used if you get this view point then you know I think you have learnt a lot in this course so next class onwards will begin with how to solve ordinary differential equations and algorithms and then finally we will move on to the convergence properties under what conditions these converge try to get some insights into relative behavior of different methods and so on.