 Hello, welcome to yet another session of our NPTEL on nonlinear and adaptive control. I am Srikanth Sukumar from systems and control IIT Bombay. I warmly welcome you to the week number 11 of our course and I hope, I really hope that we've had a very good journey together. We have already learned several methods, several algorithms that will help design robust adaptive nonlinear laws that drive autonomous systems such as the SpaceX satellite that you see in our background. With this sort of an introduction, I want to talk a little bit, go back a little bit to what we were doing in the previous week. So we actually started off talking about what is robustness and what is the need for robustness in adaptive control. And we got a pretty good idea that in the presence of disturbance, a disturbance that would not really trouble a non-adaptive loss, what happens in adaptive control setting is that this disturbance can result in our parameters going unbounded and subsequently the control also becoming unbounded in order to maintain a nice bound on the tracking error. This was of course rather undesirable and we first talked about tackling this using the parameter projection method in which we assumed some prior knowledge of bounds on the parameter. And this prior knowledge of bounds on the parameter, we designed adaptive laws using projection functions like hyperbolic functions which will allow you to keep the parameter values within this given bounds. Now one of the, of course, we did the stability analysis and we figured out some nice properties for such projection-based adaptive control laws. One of the sort of issues or well, I mean one of the sort of concerns that may be raised on in such solutions is that you require knowledge of the bound on parameters though it is a very reasonable assumption. But if you do not have such a knowledge, then there also exist solutions, they are called the sigma modification and the epsilon modification. The basic idea being that the standard adapt certainty equivalence adaptive law is augmented with a damping term which was not present in the technical certainty equivalence law. And what this damping term does is it creates a residual set in both a tilde that is a parameter error and the packing error. Now one of the issues, a couple of issues, one of the issues is that even if the disturbance is zero, we only get bounded performance with such kind of analysis, which may more or less be okay for most practicing engineers, but may not be okay with some theoreticians. They would say that if there is no external disturbance, why am I still getting sort of poorer performance or not ideal performance. The other issue is that because of the structure of ahad dot, when e goes to zero, that is e becomes very close to zero, you see that this term starts to dominate significantly and whatever parameter value has been learned ahad, suppose ahad has gotten rather close to the true value a, but that gets unlearned because this term will push every ahad to zero, right? So you unlearn the true parameter value and this can of course cause a deterioration of the packing error performance also. And this is where the epsilon modification came in, where instead of a constant gain sigma, the gain is now in absolute value of the error itself. Therefore, when the error goes to zero, then this term also goes to zero and whatever parameter value you've learned until that point remain as it is, so there is no change, all right? So that's ah sort of what you would do, the best you can do, some sense, right? Again, of course, this all suffers from the same issue, like the residual set expressions are exactly same as before. So even in the absence of the disturbance, we will ah get only uniform entertainment performance, all right? But in any case, these were rather useful solutions, I hope, yeah, I mean, for for realistic applications, these are rather useful solutions and ah, either projection or sigma epsilon modification are more than sufficient to impart robustness to your adaptive control against unmodeled ah, ah disturbances or unknown disturbances, right? So as long as things, the disturbances are bounded, you are good to go. Now, we have, by this time, seen covered more or less ah, large breadth of the adaptive control. Of course, it's ah, like any other field, it's ah, it's huge and and ah, always more to learn, more to do and more remains to be taught, but ah, of course, we can't expect to do everything, but we have ah, covered a very large portion of adaptive control and most importantly, all of you have been enabled ah, to do your own design, yeah, own adaptive law designs ah, you know, as you go along, yeah. Now, what we want to do, henceforth, is more modern ideas, you know, or ideas that are becoming more popular now in adaptive control, right? One of the first ah, you know, sort of topics that we want to do and this is where we will start with, is basically initial excitation based adaptive control. Now, as you can see, this is ah, not too far from, you know, pretty recent work and so, one of the sort of issues in adaptive control is parameter convergence, right? Or parameter learning, as you would say in modern terminology, right? Now, this parameter learning requires, ah, persistence of excitation, this is well known, right? Now, persistence of excitation, as the word says, is persistent, right? So, it has to exist for all time for you to be able to do good parameter identification and this may not always be possible, right? This may not, especially when you are sort of close to being done tracking, you want to move on and probably do another tracking problem. So, you may not have enough persistence, yeah, in your signals for you to have learned the parameter value perfectly. So, a lot of research went into relaxing this persistence condition, right? And this is what is called initial, well, this one of these methods is called initial excitation based adaptive control and the authors here are from IIT Delhi, so, very local, they are being vocal for local, yeah. But very interesting bit of work, interesting sequence of work on relaxing persistence excitation, right? So, this is sort of a reference point. So, what we do is we will look at a simple version of what they propose. Of course, they are proposing all of this for, as you can see, Euler Lagrange system is essentially a standard model for robot systems. So, we will of course look at slightly simpler, you know, version of things for classroom illustration purposes. You can very easily extend this to, you know, once you understand this, you can very easily extend this to the robot problem or any other, you know, control problem you might have, all right? So, let's begin. This is where we start our this week's lectures, right? Lecture 11.1, right? So, as usual, let's continue with a single integrator setting. So, I have a single integrator model. This could have been nonlinear or linear, it doesn't matter. Nothing much changes there, it's okay. As long as you have this kind of a linear parameterization, A. So, A is unknown, there is a control. I mean, it's, you know, more often than not, A would be positive because otherwise this is stable and nothing much to do. And the typical objective is to track a reference signal R. And so, as always, we construct an error signal X minus R. But before we go on to do any control design, we start to look at the parameter design, okay? So, this is going to be similar to the noncertainty equivalence method that we saw in projection and algorithm, in the projection based adaptive control algorithm. And so, this also has a similar feel in the sense that the parameter update, yeah, is sort of done separately from the Lyapunov analysis, okay? And we'll of course talk about it a little bit more later on. So, what we do is we want to express our system, any system for that matter, in this case a single integrator, but we want to express any system in a standard regressor parameter form. And what is the standard regressor parameter form? We want to write our dynamical system as y theta equal to e, yeah? If there's an unknown connected to you, that's a separate problem that can be dealt with also. But right now we only deal with the problem that there is the control gain is not unknown. In fact, it's identity. So, we write the system as y theta equal to u. This is what we do for all dynamical system, not just for the single integrator. So, in order to do this for the single integrator, you notice that I have constructed a regressor and a theta. Now, the interesting thing to see here is that because I want to write it as y theta equal to u, I have no choice but to augment the a with a 1 in theta. Otherwise, this dynamics and this will not match. This I have to do and it is common in this initial excitation based adaptive control. We have to do this, that we have to augment our parameter a with a 1. So, there is an over parameterization. We are trying to identify something that we already know. One is not unknown, but we will try to identify it, yeah? So, already you see one, if you may have one disadvantage if you may have this method, all right? So, you have y theta equal to u, okay? So, where y is the regressor, which is basically x dot and x and theta is 1 and a and u is of course retained as such, okay? So, that is what we mentioned here. There is over parameterization as 1 in theta is not unknown, all right? Now, we start with the process of defining filters. Again, this should be reminiscent of non-CE projection, projection based adaptive laws, yeah? This is should be reminiscent of non-CE projection based adaptive laws. In fact, why it is so reminiscent? Because most of these authors have followed an earlier author's work, yeah? So, Slotine had done, had sort of proposed these idea in 80s, yeah? You can look up his this reference, yeah? I mean you can find his reference. So, therefore, a lot of authors are using similar ideas. So, even the projection based adaptive control the non-CE version was by Akella and they also are motivated of course by the Slotine's work in the 80s, yeah? That's the idea. Now, what's the idea again? In there also, if you remember, in this equation, you essentially filtered everything that was known to you, all known quantities got filtered, right? So, what is known? The regressor is known and the control is known. So, both of these are past career filter, a low-pass filter if you may, with some bandwidth sigma which is specific of course and the additional thing here is you ask for or you require zero initial conditions, right? So, in our earlier sort of result, the initial conditions were not specified, but here we are specifying the initial condition, right? To be zero. These are again filters that are implemented in theory, I mean in the sense that they are implemented on a computer, they are not real data coming from any real dynamical system. So, we don't have to worry about the initial conditions not being exactly zero. So, once we have assumed this sort of a filter structure, apologize, right? This sort of a filter structure and some initial conditions, I can actually solve for this, yeah? So, we are sort of trying to verify the implementability of this filter structure filter, all right? So, we just integrate this using standard variation of parameters integration. So, the solution of this for zero initial condition is just this e to the power minus sigma t integral from zero to t e to the power sigma tau y tau d tau, all right? So, that sequences the solution and now I can substitute for y which is just x dot and minus x, right? So, this is actually if you notice in this case y is actually in, right? It is a two-row vector, row vector of size 2, all right? So, now one of the, if you look at this first term here, there is a little bit of a problem. Typically in a state space system, x dot is not connected, right? You measure the states but not the derivatives of the states. So, this is the typical assumption in a state space system, all right? So, the question is, can we still implement this kind of a filter? And the answer is yes, yeah? Otherwise I am assuming the, I am sorry, I am sorry for that. Otherwise the authors will not have proposed it, I guess, right? So, if you look at this term, you want to see how to deal with this particular term. This term is of course implemented it, right? No problem, right? Because x is available. So, all I need to do is integrate this or whatever. I mean, I will just solve this differential equation with this x piece, no problem, right? So, that is what we do. We look at the first piece of the integral here which is this guy. They are not implementable because x dot is typically not available for measurement. This is very standard assumption in state space design, okay? All right. Now, how do we resolve this? We just do integration by parts on this scale. So, what is integration of parts by parts of this guy? It is just whatever, the scaling remains as it is. And then you have first times integral of second and then minus integral of differential of first times integral of second. All right? We are just applying the integral by parts formula. And now you see something nice has happened. What is this something nice? You see that this thing is now implementable. And it is, I mean, x is known. Anyway, this is implementable. We will anyway look at it further. And this quantity can be evaluated. So, this, if I evaluate this whole thing right here, you get this. You get x t, x at time t, minus e to the power minus sigma t x 0, minus sigma e minus sigma t 0 to e to the power sigma tau x tau d z, right? So, nice. Everything looks like an integral or integrable term. Everything looks like it is using measurements that are available, right? So, that is good. In fact, if you look at this particular quantity, sorry, this particular quantity, this is actually the solution of this system with zero initial conditions. All right? Very standard, very easy to verify, right? Excellent. So, that is what we do. We just write it in terms of this new variable h just for ease of notation. Otherwise, you can directly use this also, no problem. So, therefore, what do we have? We have y f is actually equal to e to the power minus sigma e to the power sigma tau x tau d tau. This is the second term, by the way. This is just the second term here, right? I hope I got the sign correct. I think this should be a, the second term is a negative term. So, I think this should be a negative sign, I believe. But this is minus e to the power minus sigma tau zero to p e to the power sigma tau x tau d tau. And then you have x t from here, right? Minus e to the power minus sigma t x zero from here minus sigma h from here. And if you now look at this term also, this is also in fact equal to h. This is also equal to h in this particular case. So, this actually becomes minus sigma plus 1 h plus x t minus e to the power minus sigma t x zero, right? So, the important thing to remember is that h is implementable because x is known. Therefore, y f is also implementable, all right? Pretty straightforward, because x is known and h is known. So, in fact, I can be more precise here and say that this is actually h evaluated at t, right? h evaluated at t, okay? Great. So, we have essentially constructed a filter, yeah? An unusual filter because we also have the derivative of the states in here that is being filtered. But the important thing to remember is that this filter is implemented. Excellent. Great. Because you don't want to design something that's not implemented, right? Right. Now, let's look at some properties of these filtered variables, right? If you remember, even in the, again in the projection based adaptive control design also we wrote, we sort of tried to relate the or write the dynamics in terms of the filtered variables also. So, we are sort of doing something similar, not exactly the same, but something similar. So, if you look at this y f dot, it is just minus sigma y f plus a y, right? So, if I multiply by theta on both sides, I get this. And notice that y times theta equals to u from our regressor parameter form. I have just substituted here. And now, because theta is a constant, this is possible only because theta is a constant. If I take the derivative there, if I take y f theta as a variable and I take the derivative, I get exactly the left hand side. And on the right hand side, I have minus sigma again times y f theta plus u. And y f theta at zero is zero because y f at zero is zero, right? So, if you compare it with the u f equation, I will write it here for your reference. If you compare it with the u f equation, what do you note? You note that these two, sorry, these two equations are exactly the same, right? These two are exactly the same equations with the same initial conditions, right? So, it's just different notation. So, what does it mean? It means u f is equal to y f theta, right? So, this is very similar to what we did in projection based adaptive control, right? Even there, we found an equation in terms of the filtered variables. And that's exactly what we are doing here too, right? Because the original equation was y theta equals to u in the regressor parameter form. And we notice that just like in the projection based adaptive control method, the filtered equation also has a very similar structure just that the y is replaced by y f and the u is replaced by u less, all right? And this is of course, a very important property that gets used subsequently, all right? So, remember, so that's what I would say, similar to originals y theta equal to u as in projection based adaptive control. Yeah, again, not a surprise. Most of us researchers are always motivated by some past work and multiple researchers could be motivated by one seminal piece of past work. So, this work by Slotkin in the 80s, who also proposed having filters is of course, something that a lot of people refer to, yeah? And so, of course, you have a very similar feel to this as opposed to as actually when you compare with projection based adaptive control, right? Now, one thing we know is that the performance of this system does include by adding filters. How do we know this? Of course, we didn't do anything, Slotkin did. So, they also showed that the performance inputs and also we did in the projection based adaptive control, you remember that because of this filtered variables, you got some kind of a attractive set here, yeah? You see, you got an attractive invariant set, yeah? Which is omega equal to 0, yeah? We got an omega equal to 0, which was an attractive invariant set. So, this is sort of what Slotkin also proved, right? And that's what you get when you have a filtered, you know, like a filter system, you can expect to get this kind of convergence to some attractive set kind of behavior, which improves the performance of the adaptive control. However, this does not allow you to get rid of persistence, yeah? I mean, even here, in this system, if you notice, all you are guaranteed to get is that omega goes to 0 and what was omega? Omega was some complicated function of the unknown, yeah? So, this quantity going to 0 does not necessarily mean the parameters are converged, all right? It's just that this quantity is going to 0 and you need some kind of persistence condition on probably XF or something to get parameter convergence, all right? So, that's the important thing to remember that persistence condition is not, cannot be done away with just because you filtered it. And this is the sort of lesson that was well understood. And in order to, you know, relax the persistence condition, these authors, that is Shubhendu and Shion and Indrakarth from IIT Delhi, they proposed the introduction of a second layer filter, all right? So, this is the innovation, okay? The addition of a second layer filter. So, earlier it was just filtering Y, now they filter YF. How? It has a very specific structure. YIF now is the new variable. YIF dot is just minus YF transpose YF, again with zero initial condition. And UIF dot is just YF transpose UF with zero initial condition, okay? So, this is the second layer filter which will in fact, help us to get rid of the persistence of excitation requirement, all right? Excellent. So, what did we look at today? We started talking about one particular method which helps relax persistence of excitation condition, which may be a rather steep demand on a dynamical system because having excitation infinitely is not a very feasible requirement. Therefore, in order to relax such a requirement, there is some recent work which uses, which relies on the filter ideas that we've already seen in projection based adaptive control. The only thing is instead of a one layer filter that you have in the projection based adaptive controller, which is again motivated by Slotine's work in the 80s, here we have a two layer adaptive filter, okay? And we will see how this will result in construction of an adaptive law without requiring a Lyapunov function. And subsequently, we will also see how it relaxes the requirement for persistent excitation, all right? So, that's really the idea and we'll see how this helps improve the system performance, right? So, we are looking at a single integrator system, of course, to keep things simple, but once we understand that once we can follow this, we can very well follow the work on double integrators and you know, you can also work with all the Lagrange systems, spacecraft, dynamics and so on, you know, things like again autonomous systems such as what you see in the background here, all right? Great. So, I'll see you again in the next session. Thank you.