 So, last class we have discussed that if you have a given a functional j is equal j dot is equal to t 0 to t f and v of t x v is a function of t and x t and x dot t. And if you consider the one point is fixed and other point is free then in order to have in our job is to find a trajectory optimal trajectory x t such that this functional is x has is its extremal values either minimum or maximum. So, the necessary condition we have derived is this one. So, the first condition is the Euler's Lagrange equation this equation is Euler's Lagrange equation. So, you have to solve this one you have to solve this one there are two conditions a boundary conditions are there this condition when t f is free when t f is free then you have to use this equation. And x f is x t f is fixed then out of this one only this is you have to use it when x t f is free, but t f is fixed then you have to use in addition to the Euler's Lagrange equation. When both are free that means x t f is free and at t f is free then we have to solve Euler's equation in addition to the two boundary equation boundary condition. And this two boundary condition is called the transversality condition agree that we have seen it. And we have consider the different cases if you recollect we have consider the different cases case a b and c different cases and correspondingly Euler's Lagrange equation you have to solve it and the corresponding boundary condition. Further we have consider the next is we have consider the case 4 when the end point is restricted on a curve which is the function of t agree this. And in this equation that you will get a what is called transversality condition one is that x dot of t instead of x dot of t minus g dot of t at t is equal to t f this equation you have to solve. So, this is we have to now after solving this equation that we will get the trajectory whether this trajectory is what is called optimal this trajectory will maximize or minimize this functional or not that we have to check it by using the what is called sufficient condition. We will now discuss that sufficient condition for the functional j is to be optimized what is the sufficient condition for the functional j will be optimized mean either maximum or a minimum value of the function. So, our sufficient condition sufficient condition it is same as what we discuss in case of what is called the static optimization for problem for multivariable optimization problems the same condition will consider here also sufficient condition. The sufficient condition will give you whether the functional is maximum or the along the trajectory if you use in the functional that whether it will be maximum or minimum value of the functional that will give you the you will get it from sufficient condition. So, if you see this our second variation of the second variation of the incremental what is called functional that we have denoted by del j square and that is equal to if you see the Taylor series expansion of what we have discussed earlier that means that incremental functional value the second we have we will consider up to second derivative of the Taylor series expansion. And that second derivative we will consider now as a what is called the second variation of the functional that is equal to integration of this t 0 to t f and then del x of t that whole transpose del square v dot del x square of t this is a this is a matrix because we are finding the gradient of v with respect to x again we are differentiated with respect to x. So, this is the matrix so that matrix multiplied by delta x of t vector. So, this and second term the second term is twice delta x of t whole transpose then delta square v differentiate gradient of v with respect to x then x dot or reverse order x dot this you calculate along the trajectory of this gradient multiplied by delta x dot of t. Then plus delta x dot of t delta x of delta x of dot of t transpose delta x dot of t transpose this delta square v dot delta dot square of t this compute at the this is the matrix computed along the trajectory star means along the trajectory delta x dot of t that this bracket and this bracket is complete this and this complete and then differentiate with respect to t. So, this quantity inside the bracket of this one is a scalar quantity so we can write it in terms of quadratic form that you know quadratic form is nothing but a x transpose p x form we can write it. So, how we are writing this one you just see this one this integration of t 0 to t f then we have a delta x of t then delta x dot of t. So, this is a matrix form you write it vector in case then you write it this del square v dot del x square of t then del square v dot del x of t into del x dot of t then del square v dot del x of t into del x dot of t del square v dot del dot square of t that is whole matrix agree this you evaluate along the trajectory multiplied by delta x of t multiplied by delta x dot of t. Now, see this one this is a matrix of dimension if x is a dimension of n variables then this is the matrix of dimension n cross n this is n rows n column this is also n rows n column n rows n column and similarly the whole matrix dimension will be twice n into twice n if you see similarly this one is a n rows n column this whole thing. So, now in order to this whole thing differentiating with respect to dt with respect to dt this whole this is a vector matrix multiplied by column vector then differentiate with respect to this whole quantity is a scalar quantity same as this one. Now, in order to that we know that sufficient condition of a second variation of the functional must be if it is greater than 0 if it is greater than 0 then the functional value along the trajectory what we got it should be a negative sorry it should be a minimum value of the functional. If it is a delta square v is less than 0 then functional value along the trajectory will be a maximum value of the functional. So, how to decide this one if you see it is now in quadratic form. So, our this matrix compute this matrix whose dimension is twice n by twice n along the trajectory if this matrix is positive definite matrix this matrix is positive definite matrix that indicates that del square del j square means second variation of the functional will be positive value means the functional value along the trajectory is minimum. So, our condition is if delta square j is greater than 0 if the matrix delta square v delta x square of t delta square v delta x t and delta x dot of t delta square v delta x of t delta x dot of t this order may be changed delta x dot and delta x this value will be same. So, it does not matter for this one we have discussed this thing in details while we have discussed the static optimization problems. So, this x dot square of t compute this matrix along the trajectory and if it is a if you want to delta square is greater than 0 implies this must be greater than 0. So, we will check this matrix whose dimension is twice n into twice n where x is the n variables of the what is called functional if this matrix is positive definite implies that delta square j is greater than 0. That means the functional value what will get again if you put this x star that means optimal trajectory in the functional values then we will get it minimum value of the functional and if you can write delta square j is less than 0 if that same matrix that this matrix this matrix is less than 0. This implies that the functional value j star of this is maximum value along the trajectory. Similarly, this implies the j star of this is the minimum value along the optimal trajectory you can write optimal trajectory means x star of t this is x star of t you can write it. So, this is the sufficient condition for this one again. So, now let us solve one problems to get how one can solve such type of optimization problem dynamic optimization problem using the calculus of variation principle. So, example taken example, so find our problem is find the extremal for the functional j is equal to t 0 to t 0 is equal to 0 to t f and this functional is given x dot square plus 24 x of t into t d t. So, this functional value j is to optimize extremal whether minimum or maximum that we have to find out and corresponding what is the trajectory for this one. So, where left where the left end point is fixed end point is fixed in other words x of t 0 is equal to 0 at x of t 0 is 0 and t 0 is equal to 0 here t 0 is 0 that means you can write it t 0 is 0. So, this point is fixed point you can assign this point as a this point is fixed point and the right end the right end point t f is free, but x t f value is equal to 2 is fixed x t f value whatever the t f value is the free, but x t f finally, this trajectory should reach at time whatever the time is there at 2 units it may reach. So, corresponding to our problem if you represent that one it is something like this. So, if you consider x star of t and this is x star of t near about delta near about the optimal trajectory there is another trajectory which you call x star of t never would of the optimal trajectory is this. We are calling it a point and this point is b if you say that point is t f is free, but x t f what is the x t f is fixed that is 2 x t f is equal to and we have to find out x star of t the optimal trajectory for this for the functional for which the j will be extremal whether minimum or maximum that for that we have to check the sufficient condition. So, if you just follow our methods what we have to say first we have to consider our step solution. So, first you write the v function v function is what in your case 2 x dot square of t see this is our v is a function of x t x dot of t and t. So, the v function is what function of v is x 2 24 x of t into t this v. So, write the Euler's equation if you recollect the Euler's equation del v del x of t minus differentiation of del v with respect to x dot del x dot of t partial differentiation of this one. Then this equal to 0 and since x is a in this case dimension is 1. So, this dimension will be 1 cross 1. So, I will just put the value of v and differentiate with respect to x that is our next problem. So, v is then if you differentiate with respect to this x I will get 24 t first part of this Euler's equation this is called the Euler's Lagrange equation. Now, second part of this one you see we have to differentiate with respect to x dot. So, that will be a your 4 x dot of t is equal to 0 then 24 t is equal to 4 x double dot of t is equal to 0. Then your case is x double dot is equal to x double dot of t is equal to 6 t which equation number 1. This is a simple differential equation. So, I will advise you to recap the solution of differential equation in presence of forcing function that you recollect this one. Since this is a simple case one can find out the solution of this one by using what is called double integration. If you do it first time you integrate it will get one constant term second term you integrate it will get the another constant term all this. So, you can this in this case it is very simple. So, I can write the general solution of equation 1 the general solution of 1 what you can write a general solution of this one x of t is equal to c 1 t plus c 2 plus t cube for your convenience. Just I show you the x double dot of t is equal to 6 t integrate both side integrate both side x double dot t d t with respect to t t integrate 6 t d t then you will get x dot of t is equal to 6 into t square by 2 plus c 1. Then again you integrate this one once again if you again integrate that x dot of t again integrate with respect to time t then 3 integration of t square d t plus c 1 integration of what is called d t. So, if you do this one that solution is x of t is equal to t square t cube by 3 it is cancel 3 3 cancel the t cube we got it then you have a this c 1 into t this one and another integration constant you will get c 2. So, this is the general solution of that one let us call this is equation number 1. So, now let us try if you differentiate this one once again then what you will get it that c 1 into t differentiate 1 this c 2 is constant this is 0 then 3 t square let us call this equation is 3 equation 3. So, now this is the Euler's equation from Euler's equation we got the what is called trajectory optimal trajectory of this one but still it is with us that how to find out c 1 and c 2 that is and that two constant we can find out from what is called the initial that of a boundary condition using the boundary condition. So, let us see that how one can find out that c 1 and c 2 of equation 2. So, at time t is equal to t 0 that is 0 let us see what is our trajectory t 0 value. So, this is equal to c 1 into t 0 t is equal to t 0 plus c 2 into t 0 cube t is equal to t 0 then c t t 0 is 0 this is 0 this is also 0. So, c 2 value is 0 the constant c 2 value is 0 now initial condition from the initial condition we got the condition of this then next is at time t is equal to t f t f is unknown is free then from equation 2 this is from 2 we can write it x t f is equal to c 1 t f plus c 2 value is 0 we need not to write it here t cube this. So, c 1 that this value is given to the problem is 2. So, now our c 1 value is what we can write it c 1 value is your t f c 1 value 2 minus 2 take it this side t f you take it this side then t f cube divided by t f. So, this is the value of c f, but that depends on the final time t f because t f is free for this. So, let us call this equation is equation number 4. So, now this is the condition that means c 1 value we know c 2 value we know. So, the description of our optimal trajectory is known provided if t f is known that what type of t f that will reach to the our x t f is equal to 2. So, now cost condition is coming transversality conditions transversality conditions. So, our x t f if you recollect this our terminal conditions that one x t f is fixed that means delta t x t f is 0. So, this condition will not come into the picture only this condition will come Euler's equation and that this condition will come into the picture because here x t f is fixed not free, but t f is free that a t f means delta t f is not equal to 0 this implies that this boundary condition must satisfied that we have explained earlier. So, let us see this one f x t x dot of t of t minus and if you recollect this one see I have not put it star you can put it star because whatever solution you will get it that is only optimal trajectory star. So, it not necessary put it star that if you get the solution of Euler's Lagrangian equation you will get the optimal trajectory x star of t or you can write x of t this one. So, I am just not writing if you like you can put it star also there. So, this and del v the transversality condition when t f is free, but x delta x t f is fixed delta x t f is fixed means delta x t f is 0 then this is equal to x dot of t whole this I should write it transpose if x is a vector, but it is a scalar. So, not necessary to write transpose for this one. So, this into x dot of t, but do not forget to compute this one as t is equal to t f you have to put t is equal to t f in the transversality condition see Euler's equation transverse t is equal to t f. So, do not forget to write t is equal to t f when the t f is free. So, this now write this equation the delta x is 0. So, this is the v of x is equal to x dot square of t plus 24 x of t into t. Now, this you differentiate with respect to x dot that means this is the v we have differentiate with respect to x dot means this is a 4 x dot bracket 4 x dot of t into that other term is 0 because I am differentiate with respect to x dot. So, it is a x dot of t is equal to put t is equal to t f is equal to 0. Now, see this one that is a 4. So, there is a 2 you divide by this into 2 then it will be x dot t is equal to t f. I am now putting the t values t is equal to t f plus 12 x t f into t f minus 2 x dot t is equal to t f square this is square I missed it. So, this is equal to 0. So, finally therefore, x dot square t f is equal to 12 x dot t f is equal to 12 x t f into t f I am correct. So, this is a 4. So, this is I am writing x 2 x dot square 24 x t f so 24 t f. So, 4 that is your x dot 4 x dot t into x dot. So, it is a twice of this one. So, if you take it this that side left hand side. So, it is a minus of this equal to 0. So, this is minus. So, now I know this value of what is called t f x value x solution of this is here solution of x dot you see solution of x dot I know that c 1 3 sorry c 1 x dot is x dot. So, x dot t is c 1 3 t square as t is equal to t f as t is equal to t f x dot t f c 1 3 t f square. So, I will write x dot value is c 1 plus 3 t f square whole square minus 12. So, x t f value is what say x t f value is 2 x t f value is given 2. So, this is 12 into 2 into t f is equal to 0. So, if you simplify that one c 1 square c 1 square 9 t f 4 x dot 4 plus 6 c 1 t f minus 24 t f is equal to 0. So, what we get c 1 square 9 t f 4 plus 6 c 1 t f 6 c 1 t f square here t f square then 24 24 t f you got it this is equal to 0. So, now c 1 expression already we have derived the c 1 expression if you see in terms of t f. So, I will put the value of c 1 expression 2 minus t f cube by t f in this expression. So, if you put this it is in this expression then what we will get it just see this one I am just putting this in this expression. So, c 1 is what 2 2 t f cube by t f whole square this is c 1 this value is c 1 this value is c 1 plus 9 t f 4 plus 6 c 1 is what 2 minus t f cube 2 minus t f cube divided by t f into t f square minus 24 t f as it is minus 21 t f as it is look in the if you see this one this is the equation of this one I am just putting the value of c 1 and where c 1 is c 1 is 2 minus t f cube by t f this is c 1 minus c 1 this value c 1 I am putting in this expression and then I got it. So, if you simplify this one if you simplify that one finally, we will get the expression. So, I am skipping few steps for the there is a just is some academic exercise you have to do it to arrive in this expression. So, t x this t f 6 4 t f cube plus 1 minus 1 is equal to 0. So, now, this is the equation which one can solve for t f let z is equal to t f cube. Now, if it is this one I can write it that this t f expression that z square minus 4 z plus 1 is equal to 0. So, z is equal to it has a two roots quadratic form in fact, it is a you will get it how many roots is here for the time being z I am getting it minus b means 4 plus minus root over b square means 16 minus 4 a minus 4 a minus 4 a minus c divided by twice a. So, that will be 2 plus minus that is root 3. So, it is a 12 2 is taken out then this. So, that z has a roots of z 1 has a root of 2 plus root 3 1 which value is 3.732 another is z 2 is equal to 2 minus root 3 and that value will be 0.2679 and ultimately I have to get it z t f how t f is related z 1 is related t f 1 cube. So, our t f is related t f 1 from z 1 if you see z 1 is equal to t f cube t f 1 cube is equal to what we got it that is 3.732. So, t f 1 is equal to 1.551 this one and z 2 another case is z 2 z 2 is equal to t f 2 cube which is equal to 0.2679. So, t f 2 is values is your 0.6446. So, there are 2 final time is there 1.5 times the state will reach x t f is equal to 2 another time is 0.2679. So, t f 2 is values is your 0.6446. So, there are 2 final time is there 1.5 times the state will reach x t f is equal to 2 another time is 0.26.6446 the our final state will reach to 2 units agree. So, once I know the t I can find out the c 1 for t is equal to note c 1 is equal to 2 minus t f cube by t f and for t is equal to for t f is equal to t f 1 that is your 0.15 second c 1 value you will get it 2 minus t f 1 cube divided by t f this is t f 1 agree t f 1. So, that value will come minus 1.117 this is equal to 2 minus t f 1 plus for another case for t f is equal to or you can say what is our trajectory x t of this is equal to c 1 t plus t cube and I know c 1 value c 1 value 1 case is minus 1.117 t plus t cube. So, this is the optimal trajectory for this one when at time t is equal to 1.551 second this x t f 1 will reach the state is variable is 2 units. This is one solution and if you plot it let us call your advice to plot this x t versus that time agree and that trajectory at time that trajectory let us call is some name you give it curve 1 that trajectory is called curve 1 curve 1. Now, next situation that is what we will see that is for t f is equal to our time is that for t f is equal to t f 2 that is our time is what if you see that t f is equal to t f 2 that is what we have got it that 0.6446 second agree then c 1 this implies c 1 is equal to twice minus t f 2 cube divided by t f 2. So, this value will come 2.6871 constant. So, corresponding trajectory optimal trajectory that x t is equal to c 1 t plus t cube. So, it is c 1 is 2.6871 t plus t cube agree. So, it is c 1 is 2.6871 t plus t cube agree. So, you are once again you are advice to plot t versus time and call this curve that this optimal trajectory curve is curve 2. You can easily plot it this t versus t is 0 to you know how long you have to plot it this is valid for t is equal to t f that value is your 0.6446 second you plot it up to this t is equal to 0 to this. And that one you plot earlier one t is equal to you have to plot this plot you do it t is equal to t f 1 which is 1.1551 second agree. So, you have to plot t there are 2 trajectories are there. Which will give you optimal value or external value of the function? Both are what is called optimal trajectory. One case you justify your think 1 case will give the maximum value of the functional. Other case it will give you the minimum value of the functional out of this 2 trajectories. One will give you the maximum value of the functional other will give you the minimum value of the function both the trajectory reaches is final value at x t f is equal to unit is 2 units, but different times this one. So, I adjust advice you make your comments make your comments on the curve on the curve 1 and 2, which I asked you to plot it and make your comments which one is giving the optimum in the sense maximum value of the functional which trajectory will give you the minimum value of the functional values. So, the whether minimum or maximum value functional will give you you can find out how that our sufficient condition sufficient condition you use it. What is the sufficient condition if you recollect this one that our matrix that Hessian matrix if you recollect in static optimization problem we have discussed as an a Hessian matrix. The Hessian matrix is del v dot del x square of t del square v dot del x of t into del x dot of t del square v dot del x of t into del x dot of t del square v dot del square del x dot square of t. This value you find out the trajectory x is equal to x t is equal to x star of t what is x star of t one case we got it x star of t. If you see first case when t f is equal to 1.5 second we got it minus 1.117 t plus t cube when t f is equal to t f 1 is equal to 1.15 second. So, you put this values because in this expression you differentiate with respect to x gradient v gradient you find out with respect to x. Then again you once again differentiate and whenever you got it x you put this values of that x is equal to t f then you x is equal to t f you put it and find out the what is the whether it is a positive definite matrix or not. If it is a positive definite matrix if this is positive definite matrix then functional j star is minimum value we will get. If this is less than 0 then j star is we will give you the maximum value of the functional. So, this way you can check it. So, this we have shown it how to solve a problem dynamic optimization problem by using the calculus of variation concept. So, now we will see slowly we will go to application of what is called calculus of variation to a control problems. The control problems means you have a dynamic system any system one can model into a mathematical model as a transformation model or state phase model or different model. Our problem is concentrated to describe the dynamic system in terms of differential equation. In terms of differential equation let us call if the dynamic system is described by nth order differential equation one can convert by selecting the suitable variables into convert the differential equation which is order of n can be converted into a nth first order differential equations. So, that is called is a state phase representation of a dynamic systems whether the dynamic system may be linear or non-linear, but we can apply the what is called the calculus of variation principle to find out the optimum value of the functional or objective function optimum value. Let us see how to do this that one. So, now application of you can say the application of calculus of variation approach to optimal control problem solution. So, that is our problem let us call consider our system or plant consider the system the dynamic system or plant is described by x dot capital X. I am now using to represent if the variables are more than one then I am represent with the capital X. So, let us call that variable is n cross small n cross one n variables are there. So, that is a function of X t U t and function of time. So, this function also we have a n function that there is a vector. So, let us call this is equation number one and if you see that vector dimension is n cross one and we assume that we have a m inputs are there to the system. So, this is the time. So, this is a description of what is called our dynamic system in terms of variables and how many variables are there n variables how many inputs are there m inputs. So, our problem is that you find and corresponding performance index or objective function corresponding performance index or objective function is j is equal to s function of X t of t and that X t is equal to t f X t f or you can write it here better you write t is equal to t f the terminal time that this function below this is a scalar one because objective function is a scalar one. This is also scalar one it can be a quadratic form agree, but it is a scalar it is a function of X t and t and then we have a functional t 0 to t f v is a function of X t is a function of U t and t let us call it d t agree this is the function of that one. So, and this we call this term first term is called terminal cost terminal cost and this s is a function of X of t and t it is differentiable with respect to time and t that is and that function is a continuous function and this term is called the cost integral cost function and that j is your performance index. So, our objective is if you see our objective is to find a control signal in the system suppose we have a system is there that system and we apply the input U of t and the output is let us call the states at the output. So, we are accessing the states our problem is to find a control input U of t such that this performance index what is the performance index that performance index is indicating that it is indicating 0 to t f and it is a function of X t U t and t U t is the control input once the control input is that is that regulates the state X of t agree. So, this plus the terminal means X time t is equal to t f what is this function value that is called the terminal cost. So, that means we want to drive the state at final state at particular point agree and that represents in terms of a quadratic form agree it is we cannot write the X t f only it is quadratic, but that quantity must be a positive quantity or 0 or 0. So, our problem is find a control input which is which will control the state of the system X t such that this performance index is minimized not only that control input and state because control input state will satisfy this equation. That means this you find out the control input U in such a way this functional is minimized or maximized control input control problem is minimized let us call and then subject to this constraint and that constraint is the equality constraint. So, our problem is now becoming a what is called a constraint optimization problem and that constraint is equality constraints are there. So, I just write the statement of the problem objective find U of t whose dimension is m cross 1 number of input says m that means control law only that minimizes performance index P i means performance index. Let us call we just write this equation is equation number 2 agree that performance index 2 subject to plant dynamics subject to equation 1 plant dynamics system dynamics subject with this such that our problem is find a U only that minimized the performance index subject to the constraint that means whatever the control input we generate and x generate this must satisfy the equation number 1 that is a such in this type of problem is called such in problem optimal control problem is known as Bolgia problem. So, this is our problem now let us see if you look at this things what is difference from the earlier problem we have a same thing here earlier problem also we have to what is called integrand v is there we have to integrate 0 to 2 e f and that functional we have to minimize, but in addition to that there is another term is there which will called what is called is the terminal cost and that count physically represent the final state of the system agree and in terms of it quadratic form this and that quantity is must be a positive on that one. So, this I mean not only this there is another constraint is there it must satisfy this equality constraint and this equality constraint is nothing, but a what is called that dynamic equation of the system this equation is the dynamic equation of the systems. So, by some means if you can push it this term into the integral part of this equation that equation then our problem is similar to the earlier problem let us see this how one can do this one. So, note because our job is how to push this term into a integral part of this functional agree. So, now see this one we can write it now t 0 to t f d s x t which is function of time and d t we have just mentioned earlier that this is the continuous function which is differentiable with respect to x t as also t. So, this I can write it this into d t is nothing, but a that if you see this one this nothing, but a x of t comma t t is equal to t f minus x you can write it x t of t t is equal to t 0 which is nothing, but a x x t f t f minus x x t 0 and t 0. So, in place of this one I can write this term and this term. So, this limit is an integral limit of this one is same t 0 to t f. So, I can push it in equation number 2. So, if you push it is let us call this is equation number 3 agree using 3 in equation 2 using 3 in equation 2 what we get it just look at this expression that this is this minus this one I am using this one. So, if you do this one j of dot is equal to integration I am just keeping one step this v of x of t u of t then t d t that one plus t 0 to t f d s x of t t d t whole bracket d t plus s of x t 0 and t 0 this. And this is the constant term because you know that t is equal to t 0 what is that initial condition of this one you know this one this is the constant term. Now, our whole problem is now if you see our problem is now this one minimize this one in place of minimizing this one minimize this term let us call equation 4 agree this is equation number 2 3 and this is 4 minimize equation 4 is same as minimize equation number 2 agree minimize equation number 4 such that minimize in the you find the optimal value of control you such that this is minimize. And as well as this equation is satisfied since this is a constant term that optimal trajectory will remain same if you consider the minimization of this part only. So, instead of whole thing minimization this part minimization whatever the optimal trajectory you will get it agree for u transpose n x transpose that will minimize that as well as it will satisfy this one. So, we will stop it here next class we will continue this lecture in detail.