 So, last class we have discussed the solution of finite time regulator problem. The basic steps of solution of finite time regulator problem is first you have to solve the what is called dynamic matrix Riccati equation. Once that you have to solve by what is called knowing the final time state weighting matrix, final time state weighting matrix. You have to backward integration the matrix Riccati equation which is a n into n plus 1 by 2 dynamic first order differential equation and coupled. So, you have to solve it backward integration and store the values of this information of the matrices at different instant of time from t f to t 0 you store it then you implement the control law u of t. Once you implement the control u of t then question comes then what is the our optimal cost value the objective function value what is our objective function value. That means we have a terminal cost plus integral of the quadratic terms associated with the objective function that you have to compute from 0 to t is equal to t 0 to t f. So, that we are discussing since we have obtained the optimal trajectory for control input which in turn we can get the optimal trajectory for the state trajectory and it must satisfy the solution of that our control law in turn the state trajectory must satisfy our that constraints what is that constraint x dot is equal to x plus b u that constraint you have to satisfy. That means if you take this is that side this equation must satisfy and we can show it that optimal objective function value is nothing but a half x transpose t 0 p of t 0 x of t 0 x t 0 is the initial time of the state and p t 0 when you do the backward integration of the Riccati matrix Riccati equation dynamic matrix Riccati equation from t is equal to t f to t is equal to t 0 then t 0 value what is the Riccati what is the value of p matrix that you have to use it here to get the optimal cost function value. So, this is let us see how we have achieved this one. So, that is today I will discuss how we have achieved that optimal cost function is equal to j star is equal to half x transpose t 0 into p of t 0 into x of t 0 how you have achieved. So, let us see this one and if you consider that one that our equation that basic equation what we consider x dot star of t minus a of t x star of t x star of t minus b of t u of t is equal to 0 let us call this is the equation u star is the our optimal trajectory for control input optimal trajectory that we have found out by optimizing the our performing index and correspondingly state trajectory we can get it. So, this equation that is constraint that x dot is equal to x plus b u must be satisfied by this optimal trajectory x star as well as the input optimal trajectory u star. So, this is the basic equation. So, our performing index expression is like this way this of t is equal to half x transpose t f star along the optimal trajectory that f of t which is nothing but a p of t f this is nothing but a is equal to terminal state weighting matrix this f of t which is nothing but a p of t into x t of t star this is the our terminal cost plus integral of the what is called quadratic terms associated with state and input vector is that one this 0 to t 0 to t f then this one state weighting matrix x t u of t x of t plus u star of t plus r of t u star of t plus lambda star of t into a x a function of t x star of t minus plus b star b u of t b of t u star of t this is b of t u star of t minus x dot star of t this quantity is 0 0 multiplied by this. So, objective function values will not change even if you multiply the linear multiplier factor it will not change. So, now let us see this one actually our objective function value this plus that term we have added this one because along the trajectory this values this is 0. So, it does not it will not change the objective function values of this one now let us call this is the equation number 2 as we know from the derivation of the finite time regulator problem when we derived this one we have we have seen that we know that lambda dot star of t is equal to minus q of t x star of t minus a transpose of t lambda star of t which if you take this term in the left hand side this will be a transpose of t lambda star of t is equal to minus q of t x star of t minus lambda dot star of t and which we can write it that we can write it lambda dot if you just make it this is the transpose both side left hand side right hand side you take the transpose left hand side and right hand side if you take the transpose then you will get lambda star transpose of t into a of t minus x star of t transpose q of t actually q transpose of t since q is a symmetric matrix then I can write q of t next is this is equal to lambda dot star transpose of t. Let us call that is the equation number 3 this is the equation number t this is the equation whole equation is equation number 2 and this is the equation number 3 this one. So, this is the one expression we got it further we can note it that u star note that our control input that u star of t optimal control input is minus r inverse v transpose p of t p of t actually then we have a lambda star of t when we have derived this one u star of t c at the derivation we got it minus r inverse v transpose lambda star of t. So, this we can write it is if you take this both side I multiplied by r if you both side multiplied by this is r of t u star of t is equal to minus v transpose of t lambda star of t take the transpose of both side take the transpose of both side then it will be a lambda star transpose of t v of t is equal to minus u star t transpose of t into r of t transpose since it is a symmetric matrix r transpose is equal to r. So, you can write it since it is a symmetric matrix r transpose is equal to r. So, this due to the symmetric matrix. So, this is the equation number 4 using equation number 3 equation number 3 and 4 in equation number 2 you just replace that one lambda star transpose of t into a t replace by this expression and here once again lambda star transpose v of t lambda star of t u express by this equation 4 using 3 and 4 in 2 we will get it that j star j x star of t of t is equal to half first term terminal cost will be as it is there is no change in here p of t f x of t f then half t 0 to t f then this will be a first term is x transpose x star transpose q of t then x of t transpose plus u transpose t r of t u transpose u star of t that is that first two terms as it is now I replace lambda star t transpose of t into a t by this expression. So, if you replace this one by that one you will get it plus I am replacing that one by minus x transpose of t then your q of t minus lambda dot star of t the whole thing multiplied by say lambda star t transpose a t I am replacing by this one that is what we have written then multiplied by x star of t then this lambda star transpose v t I am replacing by plus lambda star v t I am replacing by that quantity minus v star lambda star v I am using minus u star transpose of t r of t into u star that u star is there this u star. So, u star that I am replacing by this one and what is left with this one this two terms left is lambda star transpose x dot star. So, it is left with minus then minus lambda star transpose x dot star of t the whole thing d t now see this one this you push it x in the bracket then this and this cancelled this and this cancelled agree. Then what we can write it this one this is nothing but a integration of left is integration of minus lambda dot star transpose x star of t minus lambda star transpose x dot of transpose these two things commonly we can write it into this form rewrite this equation that half x star transpose t of f p t of f x t of f then minus because both are minus term here minus term is there you see. So, I am taking minus half then this I can write it t 0 to t f d of d t lambda star transpose of t x star of t d t this. So, this two term commonly this two terms commonly I can write it into this form. So, if you take integrate this one I will get it first term will be as it is constant depending on the value of x t f and p t f this term is constant then minus then what is the limit of that one I will get it this x transpose but there is a one term this is this lambda star this x dot. So, I will get it this. So, if you do this one what will get it that one lambda star t f transpose x t f star that lower limit if you take lower limit if you take minus is there minus plus half x star transpose of t 0 x of star t 0. So, this quantity lower limit is the upper limit I have written then lower limit I have written. Now, see this one we know lambda t f is nothing but a our p x t f but they are related it is lambda t is related to p t into x t with this one. So, if I use t is equal to t f if I write it that one t is equal to then take transpose I will get this x star t f p t f x t f that minus half lambda star transpose first will come x star this is the star x star transpose t f into p t f t is equal to t f into x t f star. So, this term is this one plus half that will be x transpose t of 0 star then p t of 0 then your what is called then x t of 0 star. So, this this cancel So, our optimum value of j star that our optimum value of j star will get it this half x transpose of this t 0 into p t 0 plus x t 0 star. Now, you see initial value of this system says this is known to us. So, this is nothing but a x star t 0 is nothing but a x of t 0 is known initial known value known and this p 0 we have completed the solution of matrix dynamic matrix Riccati equation backward integration backward in time we have integrated and we have lined up with a t is equal to 0 this value also known. So, you can we can now find out what is the optimal cost of this functional or objective function corresponding to the what is called finite time regulator problems what is the optimal cost this one this is the important relation that is your half x transpose t 0 p of t 0 x of this is the now next we consider that what is called finite time regulator problem we have considered next we will consider that is what is called infinite time linear quadratic regulator for linear time invariant systems. So, next we will consider that infinite time regulator problems infinite time linear quadratic regulator for linear time invariant systems time invariant systems. This is linear quadratic regulator problem infinite time regular linear quadratic problem also called that that horizon infinite horizon finite time regulator problem horizon infinite horizon LQR problem infinite horizon previously we have considered finite horizon linear quadratic regulator problem now infinite time LQR problem also called infinite horizon LQR problems. Then what is the problem is this different from this one previous case finite time regulator if you recollect the our final time is t is equal to t f is finite. That means at time t is equal to t f our state from initial state x t of 0 will drive will go from initial state to a final state which is close to the origin or desired position of this one by using a what is called suitable control effort we have to drive the state near to the as small as possible near to the desired values that is. So, here we consider an infinite time regulator t f tends to 0 when you are talking about t f tends to 0 final time tends to 0 by using the control law what will see that our state will go to the equilibrium position equilibrium point and equilibrium point is nothing but a linear system case is a origin. So, at t tends to infinity x t f is equal to 0. So, if you consider the terminal cost if you see the terminal cost is not coming into the picture because at t is equal to t f when t f tends to infinity this state will go to the equilibrium point which is nothing but a origin of the coordinate systems. So, that terminal cost is not coming into the picture and integral part of this the integration of the quadratic term associated with the performing index that limit will come from t 0 t is equal to t 0 to t is equal to t f where t f tends to infinity. So, that will be the our corresponding performing index. So, let us redefine that our problems of that one let us consider the plant as x dot of t is equal to a of x t plus b I we are consider in the problem we have consider linear time invariant. So, a is not a function of time now if it is a function of time still we can write it this one in general form. So, a of t b is replaced by a and b since it is linear time invariant and your initial state is given x t of 0 is given. So, and the corresponding performing index as j is equal to half 0 to infinity t 0 is equal to 0 to infinity t f is equal to infinity because t f now in infinite time is infinity then x transpose of t q x of t plus u transpose of t r of t u of t whole multiplied by d. Now, you look at the terminal cost is not coming to the picture because when our t f the final time is t f is infinity it indicates the with the control f out the our state will go to the equilibrium point and equilibrium point in the our system. So, the terminal cost x t f will be 0. So, the terminal cost term terminal cost term will not come into the picture and our t f is now is infinity. So, you have to solve this optimization problem if you recast this or problem in it will be like this way or problem is design a control law u such that this performing index is minimized subject to the condition that this dynamic equation of the system is satisfied that is our problem. So, if that what I have just mentioned it that I will just write it here for your convenience if the final time t f tends to infinity then x t f should approach to the equilibrium state either 0 or origin of the coordinate systems or in the all states will drive to the 0. So, naturally this terminal cost will not come into the problem the terminal cost in j is thus no longer necessary is no thus no longer is necessary. So, this problem is known as this problem is known as infinite time time l q r problem or infinite time, infinite time horizontal problem. So, infinite time then q r problem so our previous formulation for finite time regulator problem previous formulation for finite time regulator problem is still valid only there is an additional assumption is made that the system a and b must be controllable or at least it should be a stabilizable. So, this is the additional condition is imposed there that means in this case compared to the our finite time regulator problem a and b must be controllable then the our performance index will be a bounded performance index this one because still if it is a a and b are controllable completely controllable or stabilizable the state by using the control law that can be stabilized the state. So, our assumption from the previous one that finite time regulator that a b pair pair a b must be completely controllable or at least stabilizable. So, this condition this assumption ensures a bounded performance bounded performance because time t t is equal to 0 this performance will be bounded. That means this for whenever find out the objective function that is our optimal control if you use in this expression t is equal to time t is equal to 0 to t infinity then this expression value will be bounded if this assumption is considered that means the state trajectory of this one x of t will be stable one. So, this is our basic assumption keeping this thing in mind these two things it can be shown that performance index 2 the let us call this is the system equation this is our system equation let us call equation number 1 this is the performance index is the system number 2. So, if you see this solution of this one that means it can be shown that the performance index 2 and the assumption x t is equal to t f tends to infinity is equal to 0 the matrix the matrix limit t tends to infinity p of t f will be a p which is a constant matrix. What does it mean that if the assumption is considered a b and this t f is tends to infinity this value is reached to the equilibrium point of the system that means origin then it can be shown that our solution of p is constant when t is tends to infinity this solution of this one constant. If you recollect the finite time regulator problem that our solution we have seen it let us call t f is very large then we have seen t f is finite case we have seen it if you see these values are nature of the p is like this way this is a p of t expression value this is the t f and this is t is equal to t is equal to t 0 then this period from here to here this period is the transient response of this our solution of our matrix p dot p dot is the matrix. So, this is the transient period this from here to here is a transient period that means during this period the value of p is changing let us call up to this after that the value of p is constant even for finite time regulator similarly now if you put it in limited case t f is now increasing and approaching to the infinity. So, the solution will still will get same nature of this for short period of time it is a transient period is there p of t value will be changing and then after that it is a constant. So, if it is so then our the finite time regulator expression if you see p dot is equal to then our p dot expression is this p dot is equal to in finite time regulator problem is a transpose because our system is time invariant system means it is not a function of time this is equal to p plus p of t p of a plus p b are inverse b transpose p plus q is that one. Since p is constant you see see this values when t tends to infinity that actually this is equal to t if you write it our p of function of t most basically p of function of t if you write it and if you consider the our state weighting matrix q is constant which is which is also positive semi definite matrix and our input weighting matrix are is also what is called constant and assume it is a positive definite matrix. So, since t f is infinity then most of the time after the after the transient period is over this is constant. So, during this period from t 0 to if you see from t 0 to sufficient long time of this one p is constant. So, this term will be a 0 then I can replace that one is a transpose this p of t is constant p then p of t is constant a then p b are inverse b transpose p plus q this. Therefore, a transpose p plus p a p b are inverse b transpose p plus q is equal to 0. That dimension is n cross n n is the n is the order of the order of the dynamic systems systems and this is called this whole equation is called algebraic Riccati equation non-linear more more specifically going to write non-linear algebraic Riccati equation because is no dynamic equation is involved p dot is not involved for infinite time regulator problems. So, now question is if you must know how to solve that one if you can solve this one then our problem is solved. That means we know our that u is equal to minus r inverse b transpose p p is constant because most of the from t 0 to sufficiently long term this is constant only short period of time this value of p is changing with respect to time. So, this is the transient period and from here to here is a steady state value of p steady state value of p this is the. So, that u of t is equal to 0. So, u of t is nothing but a r inverse b transpose p into x of t and this x is known we are assuming that states are accessible from the system is a known. If it is not known we have to by some means we have to estimate or you have to get the information of the states which will call estimation or observer we one has to introduce the observer to estimate the state of this system. So, this and this is nothing but a if you use k r inverse b transpose p is k then it is a this k is the this k is called controller gain controller gain. One can obtain the controller gain is nothing but a r inverse b transpose p and this p is constant and p is what the solution of this non-linear algebraic equation. So, our first step is solution of non-linear algebraic equation this is the non-linear you solve it you will get p. Once you get p then you know what is k r inverse b transpose p is k. Once you know k I know what is u and then you just recursively you solve this one or in real time what you will get you get the information of x multiplied by k. k you have done you have solved this equation it gets an offline and you know the information of k. So, multiplied by x. So, it is a state feedback but controller k is designed based on minimizing the what is called our objective function this which is called a infinite time regulated problems that due to some initial state disturbance this controller will drive the state to a equilibrium position or equilibrium point. So, if you see this one that what is our algorithmic steps algorithmic steps for finite time infinite time l q r problem or horizon l q r problems infinite time horizon control problems or you can say infinite time l q r problem solution for this one. So, first thing we have made the assumption we have to check it the assumption the pair a and b first assumption is pair a and b is pair a is controllable and a and c is detectable. So, what is controllable you might have read it in your first course of linear control theory that with the if the system is controllable by using and control f out we will be able to drive the state at time t is equal to 0 to a desired state at t is equal to t f at desired state we will be able to drive it if the system is controllable or at least it is a stabilizable and this a and c must be a detectable of this this one. So, this is the first assumption if this assumption is valid then and also we have made the assumption if you recollect that our state weighting matrix this is the q is the state weighting matrix state weighting matrix associate in the performing index that must be a positive semi definite and the input weighting matrix input weighting matrix must be positive definite. So, this thing our first step with this assumption if the system is not controllable then we will not proceed further that means we are the system must be controllable or at least stabilizable. Step one that compute the open loop system may be unstable, but it must be controllable or at least stabilizable. So, compute the solution of p form the algebraic edicati equation. So, a transpose p plus p a minus p b or inverse b transpose p plus q is equal to null matrix whose dimension is n cross n and if x is the number of states which number of states is n small n order of the system is n then a is the dimension n cross n b is the dimension depends on the number of inputs u. If u is m the dimension of b n cross m the dimension of b n cross m and the dimension of a is a n cross n order of the system then this immediately this dimension is n cross n the dimension of r is a m cross n it depends on the number of inputs and one has to solve this one. We will discuss later then how to solve the algebraic edicati equation which is a non-linear algebraic edicati equation what are the methods are available to solve such type of algebraic edicati equation. So, once you compute this one from this step you can compute that one the step two once I compute the solution of edicati equation solution of our equation then step two compute the controller gain compute the feedback gain matrix k is equal to r inverse b transpose p you know this is known by solution of this is nothing but a solution of algebraic edicati equation. So, this is known once you know this one you can compute step four step three you can compute the controller u of t is equal to minus k x of t assumed all the states are measurable agree all the states are measurable. Then once you know this one once you know this one then immediately you can find out that what is called the our cost objective function value that compute the optimum or compute the minimum value of the j that means j star I can compute j star is equal to half x transpose of t 0 m t 0 is equal to 0 then this then solution of the edicati equation p then x of 0. So, this you can compute. So, this is the steps of how to solve the what is called infinite time regulator problems this is the solution. So, if you see the stability analysis of the closed loop system it is exactly same that what we have consider in the finite time regulator problems. So, stability analysis of the closed loop C L closed loop system bracket for infinite infinite time L q r infinite time regulator problems that is in short it is called L q r infinite problem. So, this stability so let us see the stability of the system our system description is a is x of t a is the system is linear time invariant systems. So, b of t and our system state is given initial state is given if you see our control law u of t is equal to minus x of t where k is we have r inverse b transpose p this way you have generated p. So, what is the closed loop systems closed loop system is if you just write it use the value of k in this expression u k x in this expression then it is a closed loop system x dot closed loop system x dot is equal to a x of t plus b what is u minus k x of t. So, this is if you simplify adjust simplify this one a minus b k whole x of t. Now, let us call this is the equation number 1 and this is the equation number 2. So, this is our closed loop system in short we have denoted by a suffix c closed loop of x t you have denoted by this. Now, look at this one what is the what is the how you check it whether the system is stable or not one way of checking it since you have calculated the k put the value of k here find out the Eigen values of this one. And analytically how we will check the I told you exactly same way what we have described for finite time regulator based on the Lyapunov function technique we can check it the stability of the system. So, now consider the simplest Lyapunov function is like this way v of x t is equal to x transpose of p. And this is the quadratic form this is energy function is always what is called value is greater than equal to 0 when x is 0 will be when the states are all 0 then it will be a 0. And this value will be always greater than 0 greater than equal to 0 because p is a positive definite matrix p is a positive definite matrix. So, we have selected the energy function in like this way now v dot let us k. So, if energy if due to some initial condition the energy contain in the system is that one. Now, I will see whether this energy is decreasing with respect to time or not that how to v dot we have to find out with time as time is increasing whether the energy is decreasing or not that means v dot must be negative definite. So, what is v dot differentiate this thing with respect to time then you will get x dot t of t p x dot of t plus x transpose of t then p is constant we are not differentiating this p is constant. In earlier case p was a finite time regulator p was a function of we got a three terms, but here is two terms. So, put the value of x dot from equation two use equation two in equation three in three then we got v dot is equal to x of t is equal to a minus c this is our equation two x dot is equal to this. So, x dot transpose so it will come x transpose a minus a minus v k whole transpose. So, I am writing x transpose a minus v k whole transpose then p multiplied by x t this is the first term I am written here then next is this term I am writing here this term I am written here. So, this term is x transpose of t p then what is this one a minus v k into x of t this is x of t. So, this is this two terms I have written x transpose p and this is our x dot I have written. So, if you just simplify this one x transpose of t is nothing but a a minus v k whole transpose p plus p a minus v k this into x of t. So, the v dot v dot is a scalar function this will be negative definite provided this is a negative definite matrix this is a negative definite matrix and p is the solution of the Riccati equation this. So, this matrix if you can prove it that this is the negative definite matrix then our stability analysis proved that for infinite time regulator problems. Let us call this equation is 4. So, recall our recall that algebraic Riccati equations. So, you recall the algebraic Riccati equation that our a transpose p plus p a minus p b are inverse b transpose p plus p a minus q is equal to 0. So, now I will just see what I am doing it here that b p b are inverse I am taking p common from the right post and inverse b transpose then whole multiplied by p. So, it is a transpose p this term is coming p b are inverse b transpose p it is getting then I am writing p then your writing is that one is a then p b b not p are inverse then b transpose p. Then you see a transpose p b are inverse b transpose this is also one then p a this is also one, but there is one additional term p minus p b are inverse b transpose. So, I will just add another term of that one p b are inverse b transpose p plus this term. So, this our equation just remain same now what we can write it this one you see a minus what you can write it for this one a minus if you just write it the transpose of that one. So, a minus b are inverse b transpose p whole thing transpose into p. So, first term you see as it is. So, if you take the transpose p transpose means it is symmetric matrix p transpose p b are inverse b transpose this term I am getting. So, this is same as that one plus p then a minus b are inverse b transpose p and this equal to take this is that side minus q then minus will be plus because minus I am taking common. So, p b are inverse b transpose p. So, if you see this quantity is nothing but our k. So, it is nothing but a minus b k the closed loop system transpose p plus p a minus b k is equal to minus q plus p b are inverse b transpose p. Now see the left hand side I can easily replace that let us call this is equation 5 using equation 5 in 4 in this equation using equation 5 in 4 if at this term I can replace this term is same as this one I can replace by the right hand side. So, equation 4 write it here not here equation 4 equation 4 can be rewritten using the right hand side. Equation 5 as what you can write it that equation 4 you see this equation 4. So, v dot equation 4 is v dot. So, v dot x of t of t this one is x transpose of t then I am writing minus. So, minus time I am taking out q of t plus then your p b are inverse b transpose p plus x of t. So, this equation I replaced this left hand side is replaced by that one in equation 4. So, I got it that one now this change in Lyapunov function will be must be negative in order to become the system is stable. So, is a specific with a negative sign. So, this matrix must be this must be positive definite I can in the similar manner what we have shown in the finite time regulator problem same thing we can use here. So, since our assumption q e is q e is our greater than equal to 0 q e is greater than equal to positive semi definite r is positive definite matrix. Say if r is a positive definite matrix r inverse also positive definite matrix and it is specific with a some matrix and pre multiplied by some matrix post multiplied by same matrix transpose. So, this is the matrix you can say this is the matrix. So, r inverse is multiplied by pre multiplied by some matrix the combination of p and b and post multiplied by transpose of that one. So, if it is so then we can write we can also say the combined matrix is also positive definite sin r is the positive definite matrix. So, resultant of this one since is the positive definite matrix and since it is a positive semi definite matrix resultant will be positive definite matrix. So, this includes that v dot of t of t is less than 0 because this is positive negative sign is less. So, this implies that our the closed loop system is asymptotically stable. In the sense at time t is equal to 0 the states will come back to what is called equilibrium position even though there is initial condition initial disturbances given to the system at t is equal to t 0. So, the system is asymptotically stable. Next question comes in mind that how to solve that what is called a algebraic rickety equations. There are different methods are there one some methods are what is called iterative method to solve the algebraic rickety equation. Some methods are used for numerically more reliable method to solve the algebraic rickety equation. So, next class we will discuss the methods for solution of methods for solution of algebraic rickety equation. So, this is the method how this algebraic rickety equation is solved. And also we will discuss suppose a problem is given that whatever we consider the finitely regulator in state of t is equal to 0. Now, t f sorry t t is equal to t f is equal to finite time now t f is equal to tends to infinity. And our then correspondingly our rickety matrix rickety equation matrix rickety dynamic matrix rickety equation will be converted into a algebraic rickety equation. And we have to solve the algebraic rickety equation. And if you get the solution of p is positive definite we have to solve this rickety equation in such a way. So, that p must be a positive definite matrix. And then correspondingly what the solution will get it that controller gain that controller gain will stabilize the system or the states will bring down to with the controller for the states will come to equilibrium states again. So, we will stop it here now.