 So in my last lecture we looked at least square approximations, method of least square approximations was used to discretize a boundary value problem. Now unlike orthogonal collocation or finite difference where we took some finite number of points and force the residual to 0, in this case we said that some of the square of something similar to some of the square of errors that is here integral square of the square of the residuals. Now we wanted to minimize that instead of setting it equal to 0 in the domain we wanted to minimize the square of the integral over the domain. So this was conceptually different from what we have done earlier and we looked at a specific problem. So the problem that we looked at was a linear problem. So we looked at solving this, so this was a boundary value problem with two homogeneous boundary conditions equal to 0 and then we wanted to get a solution for this problem. Now we constructed an approximate solution not exact solution. This approximate solution was u cap z was where u1 cap to um cap are some known functions and idea was to find out idea was to find out unknown coefficients alpha 1 to alpha m such that so we want to if we say theta that is equal to alpha 1 alpha 2 alpha m if we define this vector theta then we wanted to find out theta least square that was minimize with respect to theta norm of l u cap z minus fz 2 square. So this is nothing but the residual this is nothing but the residual square or integral of the residual this is minimize with respect to theta inner product of l u cap z minus fz. So we wanted to find out least square estimate such that well we had chosen the basis functions in this particular case such that the boundary conditions are met. So we had a special case and then with this we derived the normal equation we derived something that look like a normal equation. So basically idea was to minimize this with respect to theta. So here between any f and g or say h and g inner product is defined as 0 to 1 f tau. So this gave us this gave us the equation that was needed to solve this particular problem. Now to minimize this or to get the solution if we take this as phi if we take this objective function to be phi then we use the condition dou phi by dou theta is equal to dou phi by dou theta is equal to 0. This particular condition can be used and you can obtain alpha 1 to alpha m numerically but in a special case where l is the linear operator we could get the so called normal equation we could get the solution analytically. If it is not a linear operator we will not be able to solve alpha 1 to alpha m analytically we will have to use some numerical optimization to get the solution and then whether there will be a unique minimum and all that is not guaranteed but in this case when l is a linear differential equation we could actually find solution to this problem that is we could find theta l square analytically using equation that look almost like normal equation. So this was the way we went about doing this. Well there is a small modification suppose here what you do if you want to take a general case and see what if this is alpha what if this is some a and this is some b they are not equal to 0. Suppose this is equal to a and this is equal to b and so well in that case what we do is we do a simple linear transformation and then try to map it to 0 to 1 in such case we define instead of uz I will call a new function say vz which is I will define a transformation I will define a new variable vz which is uz plus b minus a times a plus a what will be vz you know in terms of this new variable in terms of this new variable I will get it is very easy to see that I will get l just check this we will get z1 and z0 so we can do a little bit of a transformation can do a transformation here at 0 at uz at z equal to 0 okay you will have this is 0 okay sorry this is a this is a this will be okay so now uz equal to vz plus so now it will fall out now it will work out so uz is equal to vz plus b minus a times z and plus a so we take let us take a specific operator that we looked at yesterday so instead of l let us look at the specific problem that is d2u by dz square minus uz is equal to 1 this was the problem that we looked at okay if you use this transformation then the transform problem will be d2v by dz square minus v is equal to 1 plus b minus a z plus a so this gets transformed to this and you can check here boundary conditions get boundary conditions in terms of u will become in terms of v will become v0 is equal to 0 v1 is equal to 0 just just check here so at 0 at 0 we have at 0 we have this nicely following and at 1 well will it be vz is equal to at c equal to 1 so if I put 0 here if I put 0 if I if I put b here so b equal to yeah 0 0 will cancel and bb will cancel and you will get 0 v0 so the problem gets transformed like this so this is one way of dealing with the problem the other way of dealing with the problem is modifying the definition of inner product so now we have to work with instead of working with the space of twice differentiable functions we have to work with product spaces and define a inner product slightly differently so this is one way of dealing with non-homogeneous boundary conditions you could do a transformation and then solve this problem so when you take derivative of this second derivative of this this will disappear right I am taking second derivative of this second derivative of this will disappear this part will disappear what remains is still d2 v by dz square and then when you substitute this for u here when you substitute this okay it will be minus of this which on this side will become plus so the problem gets transformed to this problem and the solution will still work but there is more general way of this is this looks like a fix by which you know we have done some transformation and we got this a better way is to change the definition of the inner product itself so what we can do now is that this equation which we are looking at this equation that we are looking at L is a map from L is a map from twice differentiable continuous functions to set of continuous functions over 0 to 1 cross r cross r this we have seen so this is the product space and now we have to define the inner product in slightly different way so if I define the inner product see this I have to define the inner product on this space and if I define a proper inner product on the inner product on this space then I can solve the problem in slightly different way so what I do now is I have to define a inner product I want to define a inner product of a function say h ht then a scalar c1 and a scalar c2 with inner product of with gt c3 I am defining inner product on this product space I am defining inner product on this product space now this inner product will be modification of this inner product will be modification of this plus w some positive weight which is w1 times c1 c3 plus w2 times c2 c4 the inner product now the inner product now is defined as integral of ft gt inner product is defined as this is ft gt plus w1 times c1 into c3 w2 times w1 and w2 are positive weights okay so w1 and w2 are positive weights okay so with this with this what I can do is earlier how did we how did we derive how did we arrive at the optimality criteria we defined the residual square we defined the residual square and then we took derivative with respect to theta remember that we took derivative with respect to theta theta was nothing but also our alpha to the multiplying coefficients with the basis functions okay so now with this with this modified definition of the inner product what what happens is the residual square that is 2 log of residual this is equal to residual okay actually residual will be will have to be defined so I should say I should say residual and okay now I am taking three residuals one residual is over the domain okay now that is that will be taken care by integral over 0 to 1 and the two residuals at the two ends two boundary points okay I am defining two residuals and now I am going to take square of this so my objective function which I minimize is going to be this plus this plus this so the way it will change is this integral it will be it will be so this will be equal to 0 to 1 okay see now just look at this this was already there earlier we are minimizing this earlier right in addition now I have two terms I am minimizing difference between u0 and a b0 and u1 and b okay now when I actually minimize this so how do I get how do I get theta what is my approximate solution my approximate solution is my approximate solution is u cap z is alpha 1 okay and my theta is this vector alpha 1 to alpha m okay and here this residual square okay this residual square I am going to call as phi and then how do I get the conditions to how do I get my conditions to solve the problem it is dou phi by dou theta is equal to 0 but now phi is going to be this so there are two additional terms okay now the trouble with this approach is that the boundary conditions will not be exactly met that boundary conditions will be met in the least square sense this equality may not be there you know well you can make it you can make it you know go closer and closer by increasing this weights you put more weight to this the optimizer will try to squeeze in and make sure that u cap 0 and u cap 1 are closer to each other okay now by this approach I do not have to choose the functions which are giving this equal to this and this equal to this okay let us say I am choosing shifted degenerate polynomials the two boundary points are not you know a and b but by this approach I can choose the coefficient such that some of the square of you know difference between the solution and the exact boundary condition is minimized and together with you know the differential equation is obeyed in the least square sense not exactly equal to when I minimize this when I minimize this this term will not be exactly equal to 0 because I am minimizing I am finding a least square solution okay true solution for this problem will be some you know infinite dimensional space I am taking a finite dimensional approximation okay so that is how I get a solution here that is that is one way to solve this problem now there is one more variant of this method okay this is the method of least squares and you will get of course that equation by which you can get analytical solution of theta if l is a linear operator then you will get a close form solution for theta and once you get that close form solution alpha 1 to alpha m will if you put those values back you will get this least square solution of okay so by this approach now I could choose a polynomial expansion nothing stops me I can choose a polynomial expansion here so this functions could be you know one t t one z z square z cube up to z to the power m and then I can minimize this objective function with respect to alpha 1 to alpha m and get my solution okay so earlier particularly for Taylor series approximation and for approximations using orthogonal collocations we were only considering you know interpolation solutions we are mainly considering the polynomial approximations here too if you want to work with polynomials in principle you can work with polynomials not an issue okay so that will not be a limiting factor so but we have choose we might we can as well choose some convenient basis like sine cos or shifted general polynomials and work with that okay so this almost now brings us close to end of this discretization there is one more point to be discussed now this is this is a method called Galerkin's method and I am going to just briefly touch upon this method in Galerkin's method we do not attempt to minimize we do not attempt to minimize this okay in least squares method we try to minimize the sum of the square of errors okay Galerkin's method is just an extension of just an extension of the idea of projection okay just an extension of idea of projections now I am not going to derive it in any way I am just going to propose this method and what is the basis for this method should be now clear to you when I write the equations the basis is idea of projections okay so what I am going to do in Galerkin's method what is different here is that we do not put this objective function we do not put this objective function or we do not put any of the minimization problem and so on but we use one fact see what we know in projections that the error should be orthogonal to the subspace span by the basis okay whatever is the subspace which are given so here if I take a set if I take a subspace S which is defined by span of u1 cap u2 cap um cap okay then I could derive I could derive an approach let us say I have I have defined my inner product like this okay I have defined my inner product like this I am not interested in minimizing the square of the residuals but what I am interested in doing is just using the fact that when you do projections the error is orthogonal to the subspace okay so what I am going to do there is I am going to say that u1z u1z is equal to rz this is equal to 0 I want the error I want the error to be orthogonal to the subspace span by okay this u1 to um so I am just taking this basis vectors and I am forcing the condition that this to be equal to 0 okay this I have to use together with my two boundary conditions to solve the problem now you may not be able to force it equal to 0 for all the points in some cases because in some situations where you have boundary conditions you may want to force it force this equal to 0 only at m-1 vectors and then two points come from the boundary conditions because two conditions will arrive at the boundary conditions I will look at a specific problem to just give you an insight let us go back to the tram problem now this particular condition this particular approach of solving will reduce to the square problem when the operator is linear okay when the operator is linear the square method and Galerkin's method will become identical but when the operator is not linear okay so you will get a solution which is which is different so this is this can be applied Galerkin method can be applied to any other operator which is not necessarily a linear operator it could be a non-linear differential equation and you are trying to solve it and basically what we do is we do not get into minimizing the residual we just say that the error between the solution and the subspace okay that error is orthogonal to the subspace by the basis functions of the solution that is the simple idea which is used so this is let us go back to the tram problem and now that you are solid it using orthogonal collocation and finite difference you will be able to appreciate this third method how do you choose this basis functions you know how do you choose them to be orthogonal there are different ways of choosing basis functions here we actually get into Galerkin method is belongs to the class of finite element methods and you will because it involves this particular method would involve integral over the entire domain this methods tend to give very accurate solutions and I am not going to get into the details of how do you choose these basis functions you know there are some continuous basis functions which are like no cap function and so on so if you want to know more about this well it is there in the notes you can check this also book by Gilbert strand okay his latest book on applied linear algebra and his first book on linear algebra with applications both of them give you very very good introduction I think the latest book on computational science and engineering here is the latest book on computational science and engineering and the other one is on linear algebra the second one gives very detailed introduction to this topic but this becomes little more involved I just want to mention this more for the sake of completeness so if you go back to our TRAM problem so this is 1 by Peclet number into d2c by dc square so this is the problem with two boundary conditions dc by dc is equal to Peclet number into c-1 at 0 and dc by dc is equal to 0 at z is equal to 1 you already know about this problem and what I am going to do here now is my solution here is going to be this is going to be my approximate solution okay and to get the equations what I do here is to get the equations so this integral or this inner product this inner product is equated to right hand side is 0 anyway right hand side is 0 for i is equal to 2 m-1 so we equate this we equate this to 0 for i is equal to 2-1 now this is you know even if you take some nice functions just remember that I have to take second derivative of that first derivative of that and then square of this function and then calculate all the integrals okay it is a fairly involved job in terms of computing the coefficients finally what you are going to get is a m-1 equations in m-1 unknowns sorry m-1 equations in m unknowns we will get m-2 equations because we are starting with 2 and going to m-1 okay so putting this will give you m-2 equations m unknowns okay m unknowns are alpha 1 to alpha m okay the rest 2 equations will come from boundary conditions so these 2 equations these 2 equations together with m-2 equations will give you m equations in m unknowns whatever m unknowns alpha 1 to alpha m so these are non-linear equations and the terms that appear in the coefficients of these non-linear equations will be all integrals which you have to integrate which you have to find out okay so numerically this particular scheme is very very involved but the dividends are very high okay you get good solutions that is why you know the finite element methods FEM methods are so where is this finite business comes when you construct this basis functions you divide the domain into a finite number of elements and on each one of them you define some nice functions which are orthonormal and those functions are used to see for example one could use one of the one of these functions one of the popular functions are I mean you might wonder why where is the discretization where is the finite element business coming here okay so you define functions over this domain which look like this and so on you divided into finite domains so this is divided into say 3 or 4 domains so this is 1 this is second this is third so you have this cap functions okay which are continuous at these points in this region so you have to divide the integral from integral over the domain 0 to 1 you have to divide between this point to this point this point to this point and this point to this point you divide the integral into 3 parts and then evaluate each integral okay and then likewise you have to do it for every one of them okay these are not differentiable so there are ways to construct differentiable functions I am just giving you just the idea so you can construct differentiable functions you can construct smooth functions which are okay and then those smooth basis functions can be then so you can construct smooth functions not an issue so just to give you an idea you can construct a basis which looks like this over finite domain so this function basis over only some domain it will be non zero value it will have zero value elsewhere okay that is why it makes it into finite so because the function value is zero from here to here you do not have to evaluate the integral okay you just have to evaluate between this point and this point and because of this special class of functions that you consider what happens is that finally the equations which you get have some kind of a sparse structure okay and you can exploit that to solve some big problems so I am going to stop this Galerkin's method only here I just wanted to connect everything into place you know I am not getting more into Galerkin's method because it if I have to now expand on the finite element methods it will be fairly complex it will not be as easy as orthogonal collocations and finite difference finite difference and orthogonal collocations are very finite difference is the easiest to understand but I would in terms of understanding the complexity I would put orthogonal collocations next it is not it is not that difficult to understand it is basically interpolation and then you know you are just transforming the problem into set of non zero algebraic equations here too you will finally get non zero algebraic equations the coefficients will be integrals and those integrals will be fairly complex to evaluate they are quite cumbersome so it is not that if you have to write a program for this so that is why you get now commercial programs which can actually do all these integrals and solve them okay so all those together will give you a set of equations which have to be solved simultaneously to arrive at the solution you can do that yeah m that m is notional see you can have the problem is that here these equations can be forced only at the internal segments not at the boundary points at the boundary points you will need to enforce the boundary conditions okay so unless you use some trick what we did earlier we had modified the inner product definition right that trick could be used you know you could modify the inner product definition to include the two end points then you know then the boundary conditions are satisfied in the least square sense not exactly okay so you could play all those tricks of modifying the inner product definition and then including the two end points and then all that is possible so now see because this why that is possible because each of these functions if you look at this function here this function is defined over the entire domain okay so this function is defined over the entire domain so you inner product can be modified to include this point this point and integral over this point okay so then this this inner product here will get modified with two additional terms for the end points okay and then you can have you don't have to have your force this you can just force M1 to M and be done with it that is also possible okay but in that case see if you do this these two will be exactly satisfied if you do it the other way you know where you include those two as some of the squares in the inner product definition then they will be satisfied in the least square sense not exactly exactly satisfied okay so we have looked at now let me sum it up what we have looked at is you know method of discretizing problems from so what happens here even in this case you start with a problem which is in the infinite dimensions you construct an approximation which is finite dimensional and then finally what you are going to get here after you do all these integrals and everything what are you going to get here you are going to get M equations in M unknowns in this case there will be nonlinear equations okay if L was a linear operator if the square wasn't there you will get linear equations and you can solve them very easily okay but we have this square here so because of that you will get you know all kinds of square terms alpha 1 alpha 2 and you will have to do complex integrals of you know u1 with u1 square u2 u2 square in all kinds of all kinds of terms will appear in the integrals now because because you get because of that you will get nonlinear equations but what is happening if you realize is that a problem which was originally in the infinite dimensional space is transformed into a finite dimensional algebraic equation solving problem okay so I am transforming a problem which was originally solving boundary value problem differential equation is getting transformed into problem of solving M equations in M unknowns okay so the transform problem looks completely different from the original problem solving nonlinear algebraic equations is completely different problem from the original problem okay so there are some issues like errors in discretization so when you actually solve the problem okay see there are variety of errors that creep in see we said that we wanted to solve y inverse problem y is equal to t of x where x x belongs to some space x and y belongs to some space y we wanted to solve this problem okay but we are not able to solve the original problem in most of the cases in you might wonder well I am doing a course on solving partial differential equations and linear partial differential equations okay I can solve them analytically you know I can do all these series expansion why do I need all this just go back and check when you can solve those problems analytically you can solve those problems analytically when the boundary conditions are very nice geometry is a simple okay when the geometry is simple if I ask you to solve Laplace equation to be formulated for this room and if you do not approximate walls to be smooth walls suppose I want to say well there is a you know small lot here and then it comes out and then okay my boundary is no longer my boundary is no longer you know straight wall then suppose I have a problem see if I if I take the reality that well the you know the conductive the convective heat transfer from this wall is different at different places okay in some region there is wall okay so the they will not be heat transfer they will be you know they will not be convective heat transfer there will be some something like you know insulation okay but in between there are windows so if I take all these realities into account my boundary conditions even for the simple Laplace formula linear operator will be very complex and I will not be able to get those closed form solutions those closed form solutions work they are very nice they give us they give us insight okay and when you can approximate geometry to be spherical cylindrical okay or perfect you know square or some parallelogram you can actually solve those problems very but those you should look at them as some kind of limiting conditions approximate in this room okay with smooth walls and you know only one kind of boundary conditions across this wall and all these walls are exactly constant temperature is a simplification and probably that was relevant you know 40-50 years back when computing was difficult now you can compute you can say that well there is a small notch here and I want to compute for this what happens here okay so when even for a linear problem when the analytical solution can be found for nice boundary conditions they may not be computable when the boundary conditions become weird so even for even for linear partial differential equations you would need to solve them numerically the class of problems which can be solved analytically is very small most of the problems you know in real life have to be solved using numerical methods so you better understand this numerical methods so what we are doing here is we are actually transforming this as I said into yn is equal to t tilde xn where xn typically belongs to some finite dimensional space some finite dimensional space and yn belongs to some finite dimensional space well do not confuse this x this y and this x this y okay probably maybe I should use some other notation x and some different kind of y okay so this is some finite dimensional space this is some finite dimensional space so you have taken the problem and then transformed it okay for example finite difference method you are not able to force acetyl 2 equal to 0 at all the points you know you have to take finite grid points how to save it say that is equal to 0 so actually this problem is an approximation of the original problem this is an imposter this is not a true problem this is just you know this looks like this but not this is not equal to this okay and further see how many errors we commit first of all we are not able to solve the original problem we transform it into something which is computable then you know you say that this is this is belongs to some Rn let us say this is the transform problem is in n dimensions and this is in some m dimensions so let us say this is Rn to Rn okay n equations in n unknowns you have got and this T cap is a different operator all together we solve a differential operator these are algebraic equations you know something else now when you go to computer okay you cannot solve using real numbers you have finite precision you do everything using rational numbers okay and then see nonlinear algebraic equations you cannot solve them exactly you use Newton Raphson so you further approximate this see you started from here you approximated this from here you again approximate because you know Newton Raphson is required Newton Raphson requires tell us the approximation again okay so there are series of approximations just imagine so what you get finally okay so there is an approximation for you end up with you know you end up with yn tilde xn tilde suppose suppose the true solution of this this problem transform problem was actually y star or let us say yn star xn star and the true solution here is y star x star so the true true solution is y star x star then you have a transform problem which has a true solution suppose you forget where it came from okay that was a true solution so that will be yn star xn star okay but nonlinear algebraic equations suppose you get you are not able to solve them again okay so you get a approximation to this so that is yn tilde xn tilde okay what would be of interest to you is what is the difference between y star minus yn tilde well first of all can I compute this sometimes in many cases you just cannot actually define this because you know your true solution will be a continuous function and this finite element method for finite difference method you got some finite points so what is the error between this and this this is defined only at finite points this is defined everywhere in the in the domain so you know this this animal is difficult to even think about okay so this you start with something you want to solve something you actually transform it that also you cannot solve then you re-transform it okay what you will see that when you go from here to here now these lectures I'll start post midsem when I go from here to here again I have to use approximations again when I when I solve this problem I am going to again go back to Taylor series I am going to go back to we are stars theorem polynomial approximations interpolations same idea okay so Taylor series and you know interpolation gave me this but again I am not able to solve this so again Taylor series again you know the interpolation kind of approximations and then solve that problem okay so finally what we get and what we intend to do is completely different hopefully you know they are close and that is where your insight as an engineer comes into picture are these numbers which computer is throwing does it make sense is this close to this well in many situations many situations you do not know what is true through why or you do not know what is true x but as an engineer you have gut feeling that what is true what it what should true x look like okay you know that you know if there is a pfr the concentration of the reacting species will reduce as z you know this okay so if that is not happening here okay you know that there is something wrong computer is giving me garbage so that is where that is where in spite of in spite of all these you know advanced techniques in spite of availability of you know very very powerful computing tools we are still in business because your intuition as chemical engineers is required otherwise you know computer would do everything you wouldn't add you and me will not be required okay but fortunately that is not the case okay we still get our jobs because you have to make a comment whether this though you cannot define this difference whether this why and tilde which you get finally does it make sense okay is it a good solution so that is where we come into picture now to do all this business you have to solve many things approximately to solve approximately you have to give initial guess how do you initial guess only if you are a good physicist engineer chemist chemical engineer you can generate a good reasonable in initial guess and then you can solve the problem otherwise you will not be able to solve the problems okay so this this brings us to end of problem discretization so what we have seen till now is that first of all we have seen that a problem can be represented most of the problems in chemical engineering or engineering literature can be represented by this generic form where why their inverse problems we are given y and t we want to find out x okay we cannot solve them in most of the cases so we transform them to this problem so we have come up to this point okay now post mid-sem I will begin how to go from here to here and how to compute the solution okay so now I will get into solving tools like solving linear algebraic equations solving sparse matrix equations solving them you know using some iterative methods all kinds of things nonlinear algebraic equations we have looked at one method Newton there are many weeks enhancements how do you do those enhancements and then the second thing which we have to look at is only initial value problems because many problems get transformed to only initial value problems so how do you integrate differential equations subject to initial conditions all those Rande Kutta methods predictor character methods all their integration everything will come in that so post mid-sem we will look at tools till now we have looked at problem transformations okay