 the expression for solution of LQR problems. Let us describe briefly what is LQR problems. So, our problem is to design a controller, to design a feedback controller, feedback controller or control law which is described by u of t is equal to minus R inverse of t b transpose of t p of t into x of t. We assume the states are accessible measurable for feedback purpose. This control law will minimize that minimizes a performance index. What is this performance index? Half x transpose t f f t f x t f then this is the call is terminal cost and also t 0 to t f half then x transpose t u of t x of t plus u transpose R of t u of t whole bracket d t. So, our problem is to find a design a controller u such that this performance index is minimized subject to the condition subject to the equality condition subject to the condition condition x dot of t is equal to a of xt a function of xt plus b u of t. This is our problem and we have derived the expression to find out the control law u of t and q of t this is q of t which we assume is a positive semi definite mean and R of t with the input matrix input and this q is wetted with the state vector and R is wetted with the input vector and that is positive definite matrix and we have derived the expression to find out the expression for p of t for a finite time interval. Now, we have seen to solve this control law we need to solve a what is called dynamic matrix Riccati equations dynamic matrix Riccati equation that we need to and this is nothing but a set of dynamic equation set of dynamic equation which is coupled to each other and in general it is a non-linear differential equations. So, let us write the what is called the algorithmic steps that how one can solve the finite time regulator problems the algorithmic steps for l q r finite time that is finite time l q r problem finite time l q r problem. So, first step step one solve the what is called dynamic matrix Riccati equation step one solve matrix dynamic Riccati equations what is that matrix Riccati equation if you recollect this one p dot is equal to minus a transpose of t p of t plus p of t a of t minus p b p of t b of t r inverse b transpose of t b transpose of t then p of t plus q of t this equation we have to solve it. Since we have considered p is a symmetric matrix one can easily see from this expression that since p is symmetric matrix one can see from this expression then we need to solve n into n plus 1 by 2 differential equation first order differential equation n into n plus 2 by 2 since p is a symmetric matrix agree. So, this differential equation are dynamic in what is coupled in nature to agree and not only this they are coupled and in general it is a non-linear agree. So, this set of different non-linear differential equation we have to solve it that is the first step this with the boundary condition that p of t f is equal to f of t f and f of t f is nothing but a the terminal cost waiting function and this is the cost cost is what is cost is integral cost functional this whole part integral cost functional. So, with the knowledge of this boundary condition we have to solve the p dot of that is expression this expression to find out p of t and that we have to do what is called backward integration in time we have to do backward integration in time this one. So, next step once you solve this one next we can easily implement the what is our control law control law u of t is equal to minus r inverse of t agree then b transpose t p of t and x of t and this if you can write it this whole thing is k of t and x of t it is just like a state feedback you can say that information of state feedback and we are generating the control law and that k current feedback control law is varying with time and this is the second step we can implement this one and I as I told you this solution you have to do backward integration you have to store all the information of rickety matrix p and then in second step offline this is the offline computation you have to do it then store it in a matrix and then you complete u of t online then step 3 is the then solve the optimal state trajectory x star of t by using this equation x dot of t is equal to a of t x of t plus b of t u of t u of t we can complete from step 2 so this way you have to solve it and now question is suppose the matrix a of t b of t is time invariant that means this parameter does not change with respect to time and also weighting matrix q state weighting matrix and also input weighting matrix does not change with time then you can write in place of a of t in place of a of t you can write a in place of b of t you can write b in place of q of t you can write q and in place of what is called r of t you can write r in this expression but still it is this expression is valid for this one now one may interest one may be interested to find out the what is called the optimal cost so one if you want to obtain the optimal cost obtain the time obtain the optimal performing index value then j is j star is optimal value is half x transpose t 0 p of t 0 and x of t 0 if you know the initial condition of state and since we have solved the rickety equation matrix rickety equation agree and we have solved it in backward in time then store the value of p of 0 at p at time t is equal to 0 this store then you can compute the optimal cost of the performing index over the interval 0 to t what is the performance index that performance index is that one the value of j you can compute over the time t is equal to tf what is the expression for 0 to tf this expression you can find out with this expression so we know the algorithmic steps of this one now we will see the what is the sufficiency condition to get the nature of optimality sufficiency condition sufficiency condition for the optimality to check the optimality means whether the function the objective function is optimized in the sense is maximized or minimized to check that one it is sufficient condition you have to consider to check the sufficient condition let us if you recollect that our Hamiltonian matrix is this is formed from the system matrix knowledge and also performing index state and waiting state waiting matrices and the input waiting matrices expressions so this is the half x transpose of t u of t x of t plus u transpose of t r of t u of t this is the first part which is form form performing index plus lambda of t transpose e of x t x of t plus b of t u of t and you recollect it may recollect that the what is called Hamiltonian matrix is form a function which is free form it is the x dot derivative agree this so this Hamiltonian function to examine is the nature of optimal control law to examine the nature of optimal control law what does it mean to nature of the optimal control law means whatever the control law we obtain whether the performing index what we have considered earlier this one is minimized or maximized to know that one we have to check the sufficiently condition so what is that condition without deriving in details because we have already derived earlier in some few situations so we have to form what is called this matrix h of this del u of t then this will be symmetric matrix as we have already seen del x of t del u of t plus del square h del u square of t so that matrix dimension if you see this is x dimension is n and u dimension is m so this dimension will be n plus m that is the symmetric matrix so this is called what is called a matrix which is a symmetric matrix and that matrix we have to check whether it is a will give you sufficiently condition and max what is sufficiently condition will give you whether it is a maximum or minimum if this matrix is positive definite then we will get the what is the minimum value of the function if this matrix is a negative definite matrix then we will get a what is called positive value what is called maximum value of the function let us see this one we know since we know the expression for h we can easily compute this term this term and this term and this term each is a matrix of proper dimensions so if you compute this this this this one now you see this this I will get it q of t this I will get 0 because I'm differentiating with respect to you and then you will x this is also zero and this is we will get r of t and that this is r of t and in order to become in order to become the function that is objective function is maximum or minimum it depends on the nature of this matrix. So, this nature of the matrix if you see this matrix we have consider if you consider this is a positive semi definite this is a positive definite matrix this matrix is always is positive definite matrix. So, if this matrix is positive definite matrix and which is obvious this again and we have selected the q state weighting matrix and what is called input weighting matrix based on some physical considerations this thing. So, this will be greater than zero. So, the objective function value that will get it that is will be equal to what is it will give you the minimum value of the function obviously. So, this implies that this must be so this implies that matrix that this matrix must be greater than equal to zero that del square h this del square of t del square h del x of t del u of t and this is the symmetric of that one that star I can write it then del square h del u square of t must be positive definite for minimum value of the functional minimum value of the of j star of j right j. So, this is the necessary condition. So, if you select q is positive definite and r is positive semi definite r is positive definite this ensures that we will get minimum value of the functional. So, let us take an one example and illustrate that and illustrate and if you see this expression it is enough more stress you can give it in order to become a positive definite this is enough r must be a positive definite this is enough. So, let us solve this problem by using what is call a numerical example see how to solve the finite time regulator problem by solving a by considering the numerical examples. So, you consider our example is this x one dot is equal to x two and then x x 2 dot is equal to minus twice x 1 of t plus 2 x 2 of t plus 2 u of t with initial condition that initial condition x of 0 is equal to x 1 of 0 plus x 2 of 0 which is given to you 1 minus 2. So, our problem is to find out the controller u such that the performing index and the performing index this performing index is minimized this find the controller u such that this performing index is minimized subject to the condition of that dynamic equation. So, this is given is x 1 of square of 6 plus twice x 1 of 6 x 2 of 6 plus twice x 2 square of 6 this is the terminal cost thus 6 indicates at time t is equal to 6 the terminal time and initial time is t is equal to 0 is given condition plus half that integration 0 to 6 then twice x 1 square of t plus 3 x 1 of t into x 2 of t let us call this half is not given we have to formulate this problem keeping half. So, let us call it is not half is not given. So, this is equal to then your 0.5 that is 2 x square then 2 x 2 square of t plus 0.5 u square of t whole dt this is the performing index. So, if you consider this is in our standard format that means half you have to keep it here and multiplied by 2. So, this will be 4 6 4 1. So, we can just write it in short compact form x x transpose x has a 2 component x 1 and x 2. So, this we can write it if you see this one we can write it 1 1 1 and this is 2 is x of 6 this is we have written in quadratic form x transpose p x form plus the I told you multiplied by half that is divided by 2 multiplied by 2. So, half 0 to 6 this will be 4. So, the 4 then this will be a 6 3 is here 3 is here that we have expressed earlier how to convert a polynomial quadratic polynomial into a matrix and vector form and this is 4 4 is here multiplied by x of t here is I missed this one here is x transpose of t this one plus this is multiplied by 2 multiplied by 2 divided by 2 that means it is a u square of t whole dt. So, immediately you can find out you see from this expression if you write it in that this 2 equation in compact form I can write x dot is equal to 0 1 minus 2 plus 2 into x of t plus 0 2 u of t. So, this is our if you see this is nothing, but our a matrix this is nothing, but our b matrix agree and this quantity is nothing, but our f of t f this weight is given at time period and this matrix is nothing, but our q of t which is equal to q because it is not a function of time and this r is is nothing, but a equal to 1 here weight is in the matrix that means weight is in the control vector is 1. So, I can write it now is like this way. So, you can write this is into this form note f of t is nothing, but since f which is nothing, but a 1 1 1 2 just say from the last equation then our q is we got it 4 3 3 4 and our r is equal to 1 because it is a scalar input this and if you look that this expression if you look this expression the matrix open loop system when there is no control f out is applied to the system that open loop system that a matrix is unstable. If you want to find out the note Eigen values of the matrix you will find the Eigen values the Eigen values of a lambda 1 lambda 2 is equal to 1 plus minus j this. So, this is the unstable open loop system open loop Eigen values are this then matrix a is matrix a dimension 2 by 2 is unstable and this matrix is what if you see 1 1 minus 2 2 this matrix a matrix is this one agree. So, it is unstable naturally we have to use and control law so that the closed loop system response is stable 1 and not only this that we have to restricted that our terminal cost is given. So, within a time t is equal to 0 with the initial condition this the final time at t is equal to t f is equal to 6 the state should come near about the that value near to 0 this one. So, now let us solve this problems how you will solve this problems this so you used r of t that our optimal control law this is nothing but a r inverse b transpose p of t x of t this is our control law and we know r we know b and the state you can find out by simulating the system or from the system you can get the information of x t. So, now what is this how to find out p the solving the matrix dynamic rickety equation or dynamic matrix rickety equation you just write it dynamic matrix rickety equation. So, this equation what is that equation if you see this equation r or you note it r is equal to 1 b is equal to 0 to this and our rickety equation p dot of t is equal to minus a transpose p of t because our a is a constant matrix. So, you omitted that symbol a of t plus p of a minus p b are inverse b transpose p of t this is function of t p is a function of t that p of t plus q agree this is the matrix rickety equation dynamic matrix rickety equation this you have to solve with the knowledge with the knowledge p of t f is equal to f of t f. So, this value we know it from here f of t is this one. So, let us say how to solve this equation. So, our p of t since p is a symmetric matrix I can write the each component of p like this way p 1 p 1 1 dot p 1 2 dot of t since it is symmetric I can p 2 1 I can write same as p 1 2 of t dot and then p 2 2 dot of t this left hand side is equal to minus then you write it that one a transpose a is our 0 1 minus 2 2 the transpose will be 1 0 1 minus 2 2 this is nothing but our a transpose then p p 1 1 of t p 1 2 of t p 2 1 of t p 2 2 of t since p 1 and p 2 p is symmetric matrix I will write is p 1 of t this is a transpose p of t then p of t p 1 1 of t plus p 1 of t plus p 1 of t p 2 2 of t into a a is if you recollect our system matrix is minus 2 1 0 1 minus 2 2 then minus p p 1 1 of t p 1 2 of t p 2 2 of t p b b is 0 2 if you see our b is this is our b 0 2 then r inverse r is 1 r inverse is that is so this is a a this is p of t this is p of t this is b this is this one is r inverse then r inverse b transpose 0 2 which is 0 b transpose then p and p is 1 p 1 1 of t p 1 2 p 1 2 t p 1 2 t p 2 2 of t plus q our q is if you see this is p of t our q is if you recollect that our q is 4 4 3 3 symmetric matrix 4 4 3 3 this and whole bracket so now you just write you say this is a symmetric matrix of this one as I mentioned earlier if the matrix dimension is n then how many differential equation you have to solve which are coupled each other n into n plus 1 by 2 that you have shown now ours n is in the present case is 2 so 2 into 2 plus 1 by 2 that we have to solve three equations p 1 1 dot 1 expression will get from the left hand side and right hand side p 1 2 another expression p 2 2 dot another expression so three equation we have to solve so so this with p t f is equal to if you see 1 1 1 2 which is nothing but a f t f value p t f value is same as f t f value so with this knowledge we have to solve it so now we write this after solving this one because you just multiply you multiply the right hand side matrix matrix all these things and equate the position the left hand side 1 1 position and right hand side 1 1 position if you do this one then you will get it you will get p 1 1 dot of t is equal to 4 p 1 2 square of t plus 4 p 1 2 of t minus 4 and we know the our initial condition for this one we know initial means not initial final terminal condition of p we know that I can write it p 1 1 t f is equal to 1 then p 1 dot of t we can write but this I am getting I just I told you you are getting by equating left hand side and right hand side by simplifying this whole right hand side expression into a matrix form equivalent to a matrix form this 2 by 2 matrix form then you will get p 1 2 dot is equal to twice p 2 2 of t plus 4 p 1 2 of t plus p 2 2 of t minus twice p 1 2 of t minus p 1 1 of t and that value and minus 3 which p 1 2 of t is equal to 1 then p 2 2 dot of t you will get it 4 p 2 2 square of t minus 4 p 2 2 of t minus twice p 1 2 of t minus 4 with p 2 2 of t f this is t f t f is equal to 2 okay so this equation you see there are 3 first order differential equation but they are coupled to each other and we have to solve for solving this one we must know the boundary condition and we know the final terminal condition of the matrix p or each element of this one p 1 each element of p we know this one so we have to do backward indication and not only this this set of equation if you see these are the non-linear differential equation dynamic differential equation so let us call this is equation number 1 this is equation number 2 and this is equation number 3 so solving non-linear differential equations 1 2 3 backward in time in time with the given with the given with the given final condition means p t f is equal to f t f is equal to in your case is 1 and our final condition is what 1 1 1 2 so 1 can solve this thing either 2 is either analytically which is very tough to solve this one or one can solve this one by using numerical techniques so I leave this problem as an exercise solve this problem what is called numerical method to find out the trajectory of p 1 find out the trajectory of p 1 1 of t p 1 of p 1 of t and p 2 2 of t so I leave it this exercise this one to find the trajectory of this one starting from t f to t 0 means t 0 means t 0 is equal to 0 value this trajectory if you solve these problems by numerical methods the nature of this response agree what you may get it from this one let us call this is the time and this is I am plotting the p 1 1 of t p 1 2 of t then p 2 2 of t I am plotting this one now see the value of p 1 1 of t f value is equal to 1 as you know from the terminal cost this is p 1 1 p p t of p t f is equal to f t f is equal to this one so p 1 1 t f is equal to 1 p 1 2 t f is equal to 1 p 1 p 2 2 t f is equal to 2 so our p 1 2 t f is equal to 2 p 2 2 t f is equal to sorry this is 1 this is 2 so let us call this is if it is this is 1 this is 2 this is 3 this is 4 and let us call this is 1 2 time 1 2 3 4 5 and 6 7 and this way now you see this one the our final condition on p 1 1 is where it starts from here agree and let us call the solution of this one if you do the backward in time integration and the solution of this differential equation if you do by numerical methods that means by using the what is called your run out a method for solution of differential equation and if you use this terminal condition let us call this you are getting like this with the solution this is corresponding to p 1 1 t and p 1 2 t p 1 2 p 1 1 2 t also starts from here p 1 2 also starts from here let us call this solution is something like this and p 2 2 starts from 2 here and let us call this solution is something like this and this is I am showing the nature of the solution is not the solution corresponding to the problem what we are now discussing this is not the solution of this does agree so this is the your solution for p 1 2 and this is the solution of p 2 2 p 2 2 of t so if you solve this set of equation that what we have considered now here in backward integration backward in time then trajectory of p 1 1 you will get like this way and trajectory of p 1 2 you will get like this way the trajectory p 2 this way is not the I am showing I have shown the nature of this one is not corresponding to the problem what we have considered so I have given you the exercise that you solve it the p 1 1 p 1 2 p 2 2 dot using the final terminal condition agree using this final time condition condition as well as use the numerical method say the run out of 4th order method or simplest way you want to solve if the sample step size is very small you can use the what is called the others method also for solution of this three differential equations so once you get the solution you store how you store this one let us I am storing the P of T how you are storing it and P is having how many elements there are four elements are there for this particular example so you store the four elements of this one that is that is you see here at time you store it this one P 11 6 P 11 P 12 6 P 2 P 12 6 and P 22 6 with the knowledge of this this is given with the knowledge of this one with the knowledge of this one then you find out P 11 5 P 125 P 2 P 125 and P 22 5 so this next you store it P 11 4 P 12 4 P 12 4 P 22 4 and so on and last what will get it P 11 0 P 12 0 and P 12 0 P 22 0 in this way you just store it again so this competition this competition you have to do offline competition and then you store it in a memory whenever you need it let us call I want you of 0 when to complete minus R inverse B transpose P of 0 into X of 0 so you you retrieve this information P of 0 information from this stored memory from this one and then get it this one now how to find out why you have one is again R inverse B transpose P of 1 X of 1 so P of 1 you retrieve this information from the stored memory so this you this is P of 1 story and then X of 1 how you get it by simulating that one X of this or from the system you are getting the information of X 1 of T so X 1 of T then in this way you just solve this problem to get it that one so in general that this is the procedure how to solve the finite time regulator problems next will come the what is called stability analysis of finite time regulator problem stability analysis so next is stability analysis and next is stability analysis of finite time alcoor problems will be using using Lyapunov direct method this analysis is very straightforward if you see let us consider our system equation X dot is equal to a of T X of T plus B of T U of T this is our system state equation state equation and if you use the value of U of T then you will get it is nothing but a X of T B of T and U information is minus R inverse of T R inverse B transpose P of T into X of T so this and this if you club together you will get a of T minus B of T R inverse of T B transpose of T P of T into X of T now see this is our if you say this is our closed loop systems dynamics closed loop system matrix again now I want to study the stability of these systems let us consider let us consider the Lyapunov function as B of X of T is equal to X transpose P of T X of T and say P is what is P you have considered is a positive definite matrix P of T let us call this is the Lyapunov function we have considered for any this function you say when P is greater than 0 this function value is always positive for any value of T all these things now this nothing but a energy function in Lyapunov function this is because this energy function value is either positive or 0 it cannot be negative this one so with this one if you if the system is stable that if the system is stable the energy should decrease with time agree that means V dot must be negative value and suppose the system is excited with the initial condition this whether the system will be stable or not you can say the if the energy contained in the system due to the initial conditions what if the energy that energy should decrease with time that means V dot must be negative so this X of X dot of T we can write it now if you differentiate this with respect to time T then we will get the three terms associated with this one X transpose of T P of T XT plus X transpose T P dot of T XT plus X transpose of T P of T X dot of T and we know the expression X dot of T expression of the closed group system let us call this is equation number one we know X dot expression is that one we replace wherever when we will get the expression for X dot you replace by equation one so using and using P dot expression is that A transpose of this P of T let us call this is equation number two plus P of T A of T minus P B or inverse of T P B then star B transpose P of T plus Q of T use this expression P dot expression and X dot of T is equal to X dot of T is equal to we got it that one X dot expression what we got it just now we have seen it A of here X dot expression this or you write it X dot expression A of T is equal to B of T or inverse of T plus B transpose of T P of T into X of T so using this expression and this expression in two in this expression we will get finally after simplification we will get finally that one you see using P dot X dot in equation two using P dot and X in equation two we get using P dot expression in two we get that one minus X transpose of T then P of T B of T or inverse of T B transpose of T and P of T plus Q of T whole into X of T this must be negative definite in order to become negative definite because B dot if it is a system is stable then B dot must be a negative definite so in order to become negative this must be a positive definite because it is preceded with the minus sign this must be a positive definite so Q is if you see the Q is greater than equal to 0 positive definite and this R is positive definite this implies R inverse also will be a positive definite matrix so clearly from the expression of three clearly from the expression of the three that whatever the logic we have put it Q is a positive definite matrix positive semi definite matrix R is positive definite matrix so R inverse also positive definite matrix it is preceded with a matrix and this transpose post multiplied by its transpose and pre multiplied by that matrix so this matrix also will be a positive definite that means we can say M transpose P M this matrix if P is greater than 0 this implies that M transpose P of M also greater than 0 means positive definite this we have used it here so clearly one can say clearly if R is greater than 0 and Q of T is greater than equal to 0 then the upon of condition the upon of condition is satisfied and the LQR controller or the close group system is stable so this so the stability now this is the stability of them now let us call if you are interested to find out what is called the as I told you if you use the what is called our finite time regulator problems that the way we solve it here it is obvious that you have to check it what is called the sufficiency condition that means whether the control law will give you the minimum value of the functional or maximum value of the functional that you have to check it and that one can check with this matrix and this is called the Hessian matrix Hessian matrix that Hessian matrix must be a positive definite for functional value to be a minimum and we have seen that it is enough to check let us call if you select the R of T and Q of T is R of T is positive definite and Q of T is positive semi definite that it ensures that function value what a functional value you will get it minimum and stability analysis also shows that this R of T greater than if it is there clearly the closed loop system or LQR controller designed stabilize the our original systems because if you see the our original system is unstable now with this controller we can able to stabilize the systems so next question comes so as I mentioned is in the algorithm if you see this algorithm step 4 of this algorithm that the optimal value of the cost function is that one so how to prove that one this is next we will consider competition of the optimal cost since we have got the what is called that our control law U of T that must be satisfied our what is called subjective that our statement of the problem is like this way design a controller U such that the performing index what is we consider will be minimized that is the our problem find a control law U such that the performing index is minimized and this U must satisfy this our its own dynamic equation that means it must satisfy the constraints so naturally the trajectory what will get it corresponding to the optimal control law that must be that A of T X star of T plus B of T U star of T must satisfy that equations that means this implies that X dot of star of T minus A of A of T X star of T minus B of T U star of T U star of T must be equal to 0 since with the with the with the knowledge of the optimal control law U the optimal trajectory we got it and that must satisfy this our own dynamic equations this is our condition based on this one based on this one we will derive the optimal value of the function and that will discuss next class that how to derive to get the value of optimal value of the function that I have considered that j optimal value of the functional j of this is equal to half X transpose of T 0 and P of T 0 X of T 0 that portion will derive next class now look this if you know the initial state of this one which you know from the our statement of the problem which you know from the statement of the problem and if you know P of T 0 I know P of T F T is equal to T F final value then P of T 0 we can find out by from the solution of matrix rickety differential equation in backward in time and store it P of 0 then this product of this one is a scalar and that will give you the optimum value of the functional so we will stop it now