 So, last class we have taken a numerical example to illustrate the solution of optimal control problem using the calculus of variation. And you see this is the simple circuit is there our initial condition of the circuit is given like this way voltage across the capacitor is given 2 volt and current flowing to the inductor is 0 compare. So, I have to find out the control law u of t means current flowing to the main circuit in such a way. So, that at time t is equal to t f the current voltage across the capacitor will be 0 and current flowing to the inductor will be 0 find out the corresponding control law u of t. In other words I of t such that the loss in the circuit is minimum that was to the least possible energy power loss in the circuit is minimum. So, this thing we have converted this description of this circuit is converted into a dynamic equation and x 1 dot is equal to x 1 dot is equal to x 2 and x 2 dot is equal to x 1 minus x 1 plus u of t. This is the state phase depth description and it is a general structure is x dot is equal to f function of x t u t d t which is in this particular case it has become into a state phase form x dot is equal to a x of t b u of t. So, we know the our dynamic equation of this one our problem is to minimize this performance index that one. So, this we are solving this problem by using it is a constant optimization problem we have converted into a what is called unconstraining optimization problem. And using the what is called in state of Lagrangian function we are using the what is the Hamiltonian function that necessary condition for the Hamiltonian function is once you form the Hamiltonian function that necessary condition for the Hamiltonian function is first is del h del u is equal to 0. Then you have a del h del lambda is equal to x dot then next is del h del x dot is equal to minus lambda dot that three necessary equation necessary condition we have to solve it in addition to the what is called the boundary conditions. And our boundary condition if you see we have shown you earlier that our boundary condition that h del h that is what we have written see this one this is the boundary condition. And we have to use this boundary condition corresponding to our problems since that what is called del t f because we know at what time this that final state of the circuit that means voltage across the capacitor and current flowing the inductor will be 0 that t f is fixed. So, and now so we know the final state of this one is we know x voltage across the capacitor is 0 and current flowing to the inductor is 0. So, you know the final state of the system that means delta x f is equal to 0 that means delta but t f is not 0 you find out the t f for which the system final state which reached to the origin mean 0 0 position. So, we have seen by using the three necessary conditions as well as the boundary condition we have come to this point that what is called lambda 2 0 cos t f lambda 1 0 sin t f these are the three unknowns are there, but this equation 18 contains the three unknowns lambda 1 of 0 lambda 2 of 0 and lambda sin t f t f which is not known, but our main aim if you see our problem is to find out the u control law by solving that equation. So, u 2 you need the description of lambda 2 of t lambda 2 of t. So, now first lambda 2 of t if you see the expression for lambda 2 of t you got it lambda 2 of t expression that this is the expression lambda 2 of t expression. So, here unknowns are lambda 2 0 lambda 1 0. So, this we must know this one. So, we know this equation and another two equation because three unknowns are there another two equation we get from the equation of the state equation trajectory that x 1 t expression and x 2 t expression. The 15 corresponding to x 1 t and 17 corresponding x 2 of t if you put the t is equal to t f we will get the expression for that one what we are going to write it here. From 15 t f t is equal to t f from 15 and 17 at t is equal to t f from 15 and 17 at t is equal to t f we will get x 1 t f 15 is the expression for x 1 t expression. We are putting t is equal to t f in that expression 15 expression. So, twice sin t f plus half lambda 1 0 sin t f minus t f cos t f minus half t f minus t f minus t f. So, lambda 0 of t t f then sin t f see the equation 15 just now I told you that equation 15 you just see this is the equation 15 last class we have seen equation in that expression we are putting t is equal to t f t wherever t is that t is equal to t f final time this one. So, we got it is similarly from equation 17 this one I will put t is equal to t f t is equal to t f in the left hand side and right hand side both expression this. So, if you write it this one x 2 t f is equal to twice cos t f plus half lambda 1 0 t f sin t f minus half lambda 1 0 t f sin t f minus half lambda 0 t sin t f plus t f cos t f that is the at t is equal to t f that x t f final because we know the final state of the x 1 x 2 is 0 that may current flowing through the inductor is 0. We want to find out a control law e u in such a way that resistive loss in the circuit is minimum. And not only that current flowing at a time t is equal to t f that current will be 0 through the inductor is 0 that means current flowing through the inductor is 0. We want to find out a control law e u in such a way that resistive loss in the circuit is minimum and not only that current flowing at a time t is equal to t f that current will be 0 through the inductor and voltage across the inductor capacitor will be 0. So, these things now we have a let us call this equation is 19 and this equation is 20. So, now see this one there are 3 equations 18 to 20 3 equation 3 unknowns, but unfortunately this equation are the non-linear algebraic equation. So, you have to solve it by numerical techniques you can solve it. So, you are writing solving 18 to 20 one can get one can get the solution for lambda 1 of 0 lambda 2 of 0 and t f. Once you know that lambda 1 of 0 and lambda 2 of 0 then our problem is solved because we can find out our control law. If you see our control law u of t is nothing but a minus lambda 2 of t lambda minus lambda 2 of t u of t minus lambda. And lambda 2 of t is nothing but a lambda 2 of t you see the expression for lambda 2 of t here lambda 2 of t is lambda 2 of 0 sin t lambda 1 of 0 cos t sin t. So, this values are now known. So, lambda 2 of t we know intern u of t we know once you know u of t and lambda u of t then we can get the response for optimal trajectory for x 1 t x 2 of t see x 1 of t is what we got it. So, our x 1 of t this. So, I know lambda 1 of 0 lambda 2 of 0. So, I know x 1 of t trajectory of that one similarly x 2 of t putting these values I know the response of this one. So, these are the solution for this we can get it. So, or one can use to solve the equation 18 to 19, 18 to 20, 18, 19, 20 that is a set of non-linear algebraic equation one can solve by using a software generally used in MATLAB software what is called MATLAB toolbox you can use it to solve this set of equation. So, by using MATLAB toolbox that is a one solver is there f solver agree you can use this one to get these 3 values 3 unknown values lambda 1 of 0 lambda 2 of 0 and t f we can solve this one. So, now what is the optimal control law just now we have seen our optimal control law is equal to optimal control law u of t is nothing but a minus lambda 2 of t c from equation 13. So, lambda 2 expression we know. So, lambda 2 expression we know that lambda 2 0 cos t minus lambda 1 0 sin t. So, that this is the lambda 2 expression see this expression. So, first is this is equation number 4 this is equation number 4 and this is expression lambda 2 expression from 13 see the equation 13 this is the lambda 2 expression. So, once you know this one this is known all are known by solving this set 3 set of equations. So, and we have to find out from t 0 to t f we know at what time t f we know at what time t is equal to f this final state of the system. That means, voltage across the capacitor will be 0 and voltage across voltage not voltage current flowing to the inductor will be 0 at time t is equal to t f if you solve this 3 set of non-linear algebraic equations. So, if you see the block diagram of this one what we got it this we have a see this one our system x dot x 2 dot of t we have integrated the output of the integrated is x 2 of t and this is initial condition of this x 2 of 0 is equal to 2. Then this x 2 do is nothing but x 1 dot of t x 1 dot of t is nothing but x 2 integrated to realize that one is x 1 of t. So, this output is coming to here if you see the basic equation x 2 dot if you see our basic equation x 2 dot is equal to minus x 1 of t plus u of t. So, minus x 1 of t so there is a another is b this is our b and this is our u star and where from u star is getting this is u of t and this is b is in your case 0 1 this and what is this this is our optimal control law optimal control law. And how what is this expression u of t is equal to just now we have seen it u of t is that one. So, what is this one lambda 1 of 0 sin t I am writing from this one u of t sin t minus lambda 2 0 cos t. So, you see this control law is the open loop control law. So, it entirely depends on the solution of lambda 1 0 and lambda 2 0. So, this so optimal so it depends on the initial value of say u u depends on initial value of u of t. So, this is the open loop it does not depends on the state information to control the systems. So, it is a open loop what is called system and the control law is generated by with this expression. And we have seen how lambda 1 0 and lambda 2 0 are obtained from this solution of this one. So, lambda 2 0 if you see this one then when we are solving this equation. So, this initial condition lambda 2 how you are solving this equation lambda 2 expression is this one alpha 1 plus alpha 2 we know this lambda 2 of 0. So, whole control law depends on the initial condition of that alpha 1 0 and alpha 2 0. So, it is a open loop solution of this one and this is dependent on the initial condition. And in practice it is not acceptable that control law. So, what is this our sufficient condition and sufficient whether the system that objective function is minimized or maximized along this trajectory or whatever the control law we have generated you start whether the corresponding performing index minimized or maximized that has to be checked with this sufficient condition. So, this is equal to x square of t minus del square h del x of t del u of t similar del square of h del x of t del u of t then del square h this del x of t del x of t del x u square of t. This you find out along the trajectory that whatever the solution you got it x of t is let us call that solution you got x star of t and u of t you got let us call u star of t this is capital X u star of t. And if it is greater than 0 for then you conclude implies that j minimum we got it if it is less than 0 for j maximum that you have to check that things. So, you know how to solve this problems of that one next we will come to the solution of what is called linear quadratic regulator problem with finite time. So, next is before that we will discuss the what is the performing index says performance indices. Performing index is a measure of system performance is a measure of system performance first suppose a our objective. So, it is a measure of system performance first suppose that an objective function that our objective objective is to if our objective is to control dynamic response of the system of the system that described by x dot is equal to A x of t plus B u of t and initial condition is known or given let us call equation number one and our output equation u of t is equal to C x of t this is the output equation. So, this is the equation what is called state equation and this is the this is our output equation so, that output equation on a fixed interval fixed interval t 0 to t f. So, that one what is our objective the component of the state variables the component of the state variables x are small our objective is to get the component of the components of the state variables are small. So, for that one so, our problem is you design a control law u such that our component of the state variables are small for that one what should be the choice of our performing index. Since, component of the state variables are small the performing index what we have selected in quadratic form that means, sum of the error square must be small. So, a suitable suitable performance index a suitable performance index performance index is selected as so, our performing index j 1 is equal to t 0 to t f because our over the time interval t 0 to t f that our component of the state variable should be small. So, that is can be represented by x transpose t into x of t d t should be small that this indicates over the time interval t 0 to t f that this and nothing but a what is called a clean and norm or distance of the vector that is nothing but a each component if it is x 1 x 2 dot x n the dimension is x n is nothing but a x 1 square plus x 2 square plus x 3 square dot dot x n square this represents. So, this should be a small so, you have to find out the control law in such a way so that j is small means each component of the state variable is small. So, that for that one what should be our control law. So, if you want to design a control law there is no restriction on the input. So, input magnitude can be very high which is not acceptable in practical situation. So, this is one of the our performing index we consider if our each component of the state variables if you want to make it small that corresponding performing index is that one. Now, this or one can write it this also one can write it by giving weightage on the state suppose our q is the diagonal matrix it indicates if you expand this one if q is the diagonal matrix then it is nothing but a x 1 square multiplied by small q 1 and diagonal elements of q if it is a small q 1 q 2 dot dot q n. Then if you expand this one you will get x 1 square into q 1 x 2 square into q 2 that means we are giving weightage on this performing index with this one that is also could be a one of the one of the performing index when our objective to make the each component of the state variable is each component state variable to make small. This is the either one of this will be our choice second performing index if our motivation is to each component of the control variable that means you want to make it what is called large not too large if you want to make it each component of the each component of the input variable not to make too large then what should be the choice of our performing index. So, the component of the input variables u of t and u of t dimension is m cross 1. So, we have a m components that each component when to make it not to make very large and not too large then our suitable choice of performing then a suitable choice of performing index performance index performance index is j 2 is equal to t 0 to t f over the interval and 0 to t d t t transpose this d t d t. And this is nothing but a one component of once again it is nothing but a i u is a control afford just like in your previous example it is a current. So, it is nothing but a scalar case is nothing but a i square is nothing but a energy. So, u 2 t u transpose u indicates the energy or control afford. So, what is the energy is delivered this indicates physical interpretation is what is the energy delivered to the system to achieve our objectives our objective is now in this case to make the input variables are each input variable not too large again that is we have to make it or one can select this one t 0 to t f u transpose giving some weightage on u. So, this u of t d t this is also may be our choice of performing index. So, our problem is like this way find out the control law u in such a way this performing index is minimized subject to the constraint the x dot is equal to a x plus b u, but here we do not have any control on the state. State magnitude may be a very large all these things again. So, here we do not have any control on the state. So, now question is so this is the let us call this is 3 this is 4. So, now we make it comment that what is the minimization of 4 means the minimal energy to the system because this is because this is because input energy is related to the input generality related to the because this is because because input is generally related to the energy consumed in the circuit or in the system. So, that u transpose u u transpose u again this indicates nothing but a norm of u of t square. So, it is nothing but a control effort or control energy involved in the system again control energy is supplied to the system this one this indicates. So, if you see first case when the system is in equilibrium position what we expect we expect our state should be in equilibrium position in linear system the generally our equilibrium position is 0 0. So, if you give a perturbation with the initial condition the state will deviate from the equilibrium position and the corresponding deviation you see our x minus this is the our state x minus x 0 is the our equilibrium position this transpose into x minus x t minus 0 this is the our equilibrium position from there we have given some perturbation. So, this interpreted x transpose of x. So, this is to minimize this one, but you see whenever minimizing this one whenever minimizing that quantity then we do not have any control on input whenever minimizing that second part then we do not have any control on the state. So, there is a what is called so remark it may be pointed out we cannot simultaneously minimize the performance index indices 1 that is we are consider 1 that is j 1 of dot and 2 j 2 of dot. That means this is nothing but a t 0 to t f x transpose of t x of t d t and this is nothing but a t 0 to t f u transpose t u of t d t we cannot simultaneously minimize because why you cannot do it. Because the minimization of j of j 1 results large control effort or control signal control effort also you can control signal while minimization of while minimization of j 2 results minimization j 2 results small control signal. So, this is the minimization of j 1 because we do not have any control. So, if you give it physically if you give a pumping more energy to the system that quantity will be minimized. Second part of this one is when we are minimizing that one when we are minimizing the results small control signal minimization of this one results small control signal. So, these two things simultaneously we cannot minimize because it is a contradict of one another. So, what you have to do this one we have to make some to this dilemma we have to make some compromise. So, the compromise we can make it by using the convex combination of these two performance index. So, to solve this dilemma we could compromise between two conflicting objective functions objectives by minimizing by minimizing the performance index performance index that is performing index that is a combination of that is a combination of that is a convex combination of convex combination of j 1 and j 2. So, our resultant performance index is j dot is lambda into j 1 dot and plus 1 minus j 2 1 minus lambda is j 2 and lambda varies from 1 to n 1 1 to n when lambda is equal to 1 then this is minimization of this one when lambda is equal to 0 then minimization of j 2. So, you are taking the convex combination of these two things. So, one can go one can get good results good response of the system by making trial and error giving more weight is here or less weight is here depending upon the situation with lambda value we can get the tune the system response. So, let us now see how to solve our control problem, but what do you mean by the linear quadratic regulator problems on finite time linear quadratic regulated problem and regulator you mean optimal control. So, linear quadratic regulator problem what we understand. So, describe the system equation like this later that the system is described as x dot is equal to A x of t plus B u of t. So, that is this one and y is equal to y of t is equal to c x of t and this is the initial state x t of 0 is equal to x t of 0 either initial state is given to you this is equation number 1 this is equation number 2 and this dimension is n cross 1 this input is m cross 1 and output is p cross 1 p outputs are there. So, our problem is to find a control law the statement of the problem or problem is to find a control law u star of t such that the associated associated performance index. So, the performance index j which is a function of u t x t all this thing half x transpose t f f of t f x of t f plus half t 0 to t f x transpose of this q x of t plus u transpose r u of t d t. Such the associated performance is minimized over the interval t 0 to t f. So, our problem is to find a control law u such that this the performance index is minimized and our state is moved the initial state the control law will drive the initial state from x of t 0 to from initial state x t 0 x t 0 to drive the final state x is equal to t f near to the what is called origin or near to the 0. So, that is our problem. So, once again I state the problem like this way that our problem is find out the control law u of t in such a way. So, this performance index is minimized. In other words you can say you find a control law u that will drive the state from the initial state x to the final state x t f which is close to the origin or 0 for that you find out the control law u of t. And what is the performance index is there similar to this one is a terminal cost similar to earlier discussion it is a terminal cost and this whole part is a integral cost functional this is the integral cost functional. Now, this t f f of t this is called the terminal waiting matrix you can call this is the terminal f of t this f of t is a matrix whose dimension is n cross n is a terminal is a terminal waiting matrix terminal cost waiting matrix and which is a positive semi definite matrix which a positive f is a positive semi definite. And this is the symmetric this is also symmetric q is a symmetric positive semi definite that q is equal to you can write q transpose and which is a positive semi definite r is the waiting matrix with the control f out or control signal r is the waiting matrix associated with the control signal. And q is the waiting matrix associated with the state vector and this r also symmetric matrix r is equal to r transpose and it is a only positive definite matrix not positive semi definite matrix. So, this indicates that this performance index you find out the control law that will drive find out the optimal control law that will drive the state from initial state this to a final state x t f which is near to the origin by minimizing this performance index. And each term of the performance index has a physical significance because our original system is at rest whatever the position is there and we have given the system perturbation x t is equal to t 0 something. So, that state will deviate from the equilibrium position and this indicate the sum of the deviations squares with some weight is q. And this is the control f out that is u transpose u if r is the identity u transpose is a control f out. So, simultaneously we want to minimize both in such a way that we find you minimize that u you minimize that performance index j such that that control f out u will drive the state from x t 0 to x t f which is near to the origin. So, this is called finite time regulator problem within a finite time the state should reach to the origin near origin. So, this if you see it is equivalent to our original statement of the problem s this is x t f comma t f plus whole t 0 to t f v x t u of t comma t v of t. Now, only difference you should there is a half in present in this expression this half is intentionally we kept it we know if the objective function if you multiply it by a scalar quantity the optimal point does not change it. But, optimal value of the function value will change it. So, just I multiply it by half the optimal value at which the objective function will be maximized on minimize that value will not change optimal point at which the function value is minimum maximum will not change. So, why I made it half you will see when you will do the that find out the necessary condition of this objective function there are two terms a two will come from the when you will differentiate with respect to x all with respect to y all these things with respect to lambda we will do just like a your what is called necessary condition what you got it the del h del u the two term two will come into the picture and this two term and half will cancel. If you do not keep it half the two will be carried out throughout the derivations that is only this. So, intentionally we make it half in the expressions. So, let us why we have introduced half we just write it here remember the half in the cost function is associated mainly to cancel to that would have otherwise otherwise been carried on throughout the that result or derivations. So, if you just see our main problem of this our the statement problem our problem our aim this is the important thing our aim at driving the initial state x of t 0 is equal to x 0 to the smallest possible value near to 0 in the interval t 0 to t f, but without but without spending too much control effort too much control effort. Means, you would transpose you would transpose you to achieve the goal to achieve the goal. So, this is our statement of the problem if you see this one our problem is find out the control law u of t such that our original state will with the application of this control law our original state will drive to a what is called at time t is equal to t f x t x t f which is near to the origin will drive not only this it will satisfy our original system dynamic equations. So, this is our so now this problem is solution of this problem is easy the same as what we have described earlier now our the state equation we have now described in state in state of what is called that dynamic equation is expressed in in terms of state equations. This is only difference is this one now that physical significance of this weighting matrix this is called state weighting matrix q is called the state weighting matrix. The q is whose dimension is n cross n q is a q transpose agree and which is a positive semi definite matrix and it is called state weighting matrices. Then r is equal to r transpose which is greater than 0 when positive definite positive this is positive semi definite is called control weighting matrices agree. So, if you want to keep the state small in the expression if you see in the expression if you want to keep the state small then choice of q must be positive definite matrix. So, our implication you can say implication of various weighting matrices and what is this f of t t f is the this is positive semi definite matrix and this is the terminal weighting matrix terminal cost weighting matrix. This dimension is m cross m and this dimension is n cross m because f is associated with the x. So, it is n cross n so next is implication of various weighting matrices. So, first implication q of t state weighting matrix this is a state weighting matrix agree to keep the state small agree because initially the state small the system state at the equilibrium position if you give a perturbation to the system the state will deviate from the equilibrium position. So, if you want to keep the state small then the choice of then the integral of the expression half x transpose q x of t this should be x transpose q x of t this should be non negative non negative small implies that q must be greater than 0. You want to keep the this integral of this one small this should be a non negative small quantity the q must be a positive definite matrix. So, how you can make it see this expression how can you make it this q this part is small by giving more control effort to the system the u magnitude should be that u star u t of u t multiplied u t transpose of u t should be large then it will drive the state to small value of this one. But, when you will give the control effort large then what will happen this control effort is not permissible to act into the system because it is a physical system. So, you cannot make it too large of this one second implication of that is R R with R R of t the control weighting matrix to keep to keep the control effort to keep the control effort small the integral part integral of the expression half x transpose u transpose this R u of t should be should be positive small why positive small this quantity that that what with this quantity is you are getting this quantity. If it is 0 then u transpose u t if it is a 0 that there is we are not applying to any control effort to the system. So, it will not be able to control the state of this one. So, this should be a positive definite. So, this input positive implies that R should be greater than 0. So, another important issue is most of the practical control problems you will find there is a restriction on the control input. That means the control input has a minimum value and maximum value of this one. So, this is most of the practical problem we have a restriction on the control input, but for the time being we will discuss that there is no restriction on the control input. The control input u of t there are no constraint imposed on the control input u of t in your statement of the problem. We have not mentioned any control constraint in the statement of the problem which is very uncommon in real practice or which is very important in practice to design the control systems to design the close loop optimal configurations. First we will design a linear optimal control linear regulator problem without constraints. Then we will take that if there is a constraint in the control input constraint the input as well as in more practical problem constraint in the input as well as constraint in the state. So, that problem we will discuss later. So, the last one is what is called infinite time infinite time infinite final time that means t f tends to infinity. When the final time t tends to infinity the terminal cost tends to infinity the terminal cost when final time t tends to infinity that terminal cost does not have any sense because at time t is equal to infinity x of t f will approach to 0 or it will go to asymptotically it will go to 0. So, in that situation the terminal waiting function has no what is called sense at all. So, we will put that f f t f is equal to 0 when t tends to infinity. So, it is called this type of problem infinite final time this type of problem is called what is called infinite horizon problems infinite infinite horizon control problem. So, another two points are there we will discuss less next class of this one. So, infinite time regulator problem t tends to infinity you see the state will reach to the origin means near equilibrium position at that situation the terminal waiting function has no sense. So, we will assign that t f is equal to 0. So, that type of problem is called what is called infinite horizon control problems. So, we will