 Hello everyone, welcome to yet another session of our NPTEL on nonlinear adaptive control. I am Srikanth Supermar from Systems and Control IIT, Bombay. So I warmly welcome you to this last week of lectures on this nonlinear adaptive control and we have already completed four lectures in this set of lectures this week. And we started up in this week looking at connections between adaptive control and learning. And we are now focusing on a specific problem of looking at a multi-layered neural network which is being tuned, which is going to be tuned using an adaptive controller. And this neural network, this three-layer neural network really specific is being used to design approximation-based controller for robotic manipulator. So we do hope that the algorithms that we have been looking at will help you design robust adaptive control algorithms for systems like the aircraft, drone, spacecraft, etc. that you see in the background. Now, where we were until last time was basically, we had already seen what the robot dynamics consists of in the joint space coordinates. And then we sort of also define this nice back-stepping variable which is used for analysis and this is in terms of, of course, the tracking error. And the entire dynamics was, of course, written in terms of this back-stepping error variable. And this dynamics is what contains the nonlinear robot function. And this is essentially what the neural network is used to approximate. This function is, of course, depending on some variables like ee.qd, qd.qd, double dot, which are, you know, collected together in this x and using an estimation of this function using a neural network, which we will see subsequently, we define a controller. And of course, it's not difficult to see. I mean, we're not doing that here. It's not difficult to see that if f tilde and tau b are, in fact, zero. So if these are not there, then this is a nice stable system, asymptotically stable system in R. And you can show that R goes to zero, which means that you can also show that e and e dot. We, of course, showed some nice relationships using this back-stepping error variable design. We show that, you know, that basically the e and e dot are bounded by the back-stepping error variable norm, as you would expect. Then we have talked about a few properties like the positive definiteness of m, the boundedness of vm dot, skew symmetry of m dot minus 2 vm, and the fact that the disturbances are bounded. We did not look at the passivity properties. So the idea is to estimate this non-linear robot function. This was the sort of plan using the neural network, the three stage or the three layer neural network. We clubbed the unknown weights, w and v, inside this new variable z. And then, of course, we made a few assumptions. The first one being that the ideal value of weights are bounded, which is a very reasonable assumption. We do that even in projection-based adaptive control design. Then we also assume that the trajectories are bounded. This, again, trajectory and its derivatives are bounded. This is, again, another assumption we did make while doing our standard adaptive control. And finally, using this, we could come up with the fact that the norm of x, that is where x is this e, e dot, q d, q d dot, and q d double dot, is also bounded by q d and the norm of x. Again, not difficult to verify once you have made these assumptions. So then we, of course, talk about linearizing the neural network because what happens is that you have this non-linear regressor parameter structure, which you cannot define adaptive laws for. And even if you do define some kind of adaptation law, you will not be able to prove any stability, which is why we looked at some Taylor series approximations. So we had nw hat by the estimates and correspondingly we defined the tilde, which are the estimation errors, and also the error in the activation function values. And using that, we sort of gave an estimate for this sigma tilde. So sigma tilde had this kind of an estimate. And further, we also state the fact which said that this square term is bounded in this way. So bound on the square term is, of course, obtained using this kind of an expression. This expression is simply obtained from 23. So from 23, you obtain this and from here you can actually prove this kind of a fact. So this fact essentially says that for sigma ideal basis function and tan hyperbolic functions, so these are the activation function examples we looked at, you do have this synchronized relationship. Now this is where we start this lecture. So let's mark it first lecture number 12.5, which is the fifth lecture of this last week. So here if you see the way we define f hat is again using just the estimates, we replace w by its estimate w cap and v by its estimate v cap. And that's it. And so v cap and w cap are the current estimated values. So this is what is your neural network functional estimate. So with this, you have this tau 0, which was the control we defined. And the true control. So until now we had only the tau 0, if you remember, which was just these two terms contain the functional estimate and damping nice PD type of a term, proportional derivative type of a term. But now we also add some kind of sort of a robustification input v. And this is what we look at what this v can be later on. But for now, we introduce this robustifications out of input v. So this becomes your true control tau. So yeah, so v is basically, like I said, is a function to be detailed subsequently that provides robustness in the phase of higher order terms in the Taylor series. So this is sort of the block diagram for this whole implementation, you can verify that this block diagram matches with the system equations, not too complicated. And of course, you've defined q under bar and e under bar as this e dot and q q dot sort of vectors. So once you have this modified control tau, we started with the tau 0 and we added a robustification or we subtracted a robustification control v. So you have the tau and then you substitute for that and you get this sort of a closed loop system. You already had this term, this is not a new term. And then this was the function and the true value of the function if you may, where you have the ideal value of the weight w and v. And then you have the estimate w hat and v hat and even the ideal value is somehow epsilon off from the true function. So the function is approximated as w transpose sigma v transpose x plus an epsilon. And you have this estimated value, then you have the disturbance and then you have this robustification term. So then we do a little bit of manipulations. What is this manipulation? First we add and subtract a w transpose sigma hat. So that sort of gets combined. So this you notice is w transpose sigma hat. So we already defined this notation. So this is actually equal to w hat transpose sigma hat in the notation we defined. So if you add and subtract w transpose sigma hat, one of it combines with this, the other one combines with this guy. So the combination with this will yield the w tilde transpose sigma hat because this is w hat transpose sigma hat and your w transpose sigma hat. So if you subtract one from the other, you get w tilde transpose sigma hat. And then this guy combines with this again to give you a w transpose sigma tilde. And so you get two terms by adding and subtracting the w transpose sigma hat. So then of course these three terms remain exactly as it is. And these three terms show no change. Great. So now again we do another layer of manipulation. And we now, so first we added and subtracted w transpose sigma hat. And now we add and subtract w hat transpose sigma tilde. Why? Because this term and this term is what we are looking to sort of deal with. So if you look at this term and this term together, these two will combine to give you a w tilde transpose sigma tilde. And you will of course be left with a w hat transpose sigma tilde. So that term remains as it is here. And these two combine to give you this term. Great. So now you had two terms here and you now have three terms because of this addition and subtraction. It is just a rewriting of these terms with your background. Now we start using all our Taylor series approximations that we did. So this Taylor series approximation is for this v tilde term. Sorry, it is for the sigma tilde term. So this is where we use the Taylor series approximation. So if you look at this guy, it is retained as it is. And then you have the w hat transpose sigma tilde type term. And then you write some terms as what we call the disturbance term or what the authors call as the disturbance term. So here, if you notice in the sigma tilde approximation, you had several terms. So sigma tilde looked like this. I mean, of course you have this term which had some bound. But then the key approximation was something like this. So this, this did we define any further notation? Let me see. This was defined as sigma hat transpose. I believe this is called sigma hat transpose this term. And then you have v tilde transpose x. So that is what is written here. The sigma tilde here is written as sigma hat transpose v tilde transpose x. And then everything else is sort of written as a disturbance, including this guy. So this also has this, this comes to this with the approximation. And then you have the second order terms which combine to give you this. And then you have epsilon plus tau d and the v is kept as it is. So if you notice, a lot of terms have been, so the second order terms of corresponding to this guy and this guy combine to give you this. Yeah. And the first order term of this guy is written here. And the first order term of this guy is also clubbed into the disturbance. Okay. So rather interesting. Just I would say, more or less arbitrary choice of what should be disturbance. Yeah. I mean, here this w1, I mean, it makes sense for these to be disturbance. And of course, also this, because if you think of the second order terms to be small enough, fair enough, this is also disturbance. But now thinking of this term as the disturbance is not very well justified. So anyway, so we, so they of course make some changes. Yeah. So unfortunately, using this error system does not yield a compact set outside which you have a negatively upon a function derivative. Right. So which is what you needed. You need the Lyapunov function derivative to be negative, at least outside a compact set. What is this compact set? This is what is the residual set. Right. So this what, what they call this compact set is what is the, how we have talked about the residual set. And this is what is the residual set. All right. So then they of course rewrite things a little bit, I guess. Yeah. What is it that they do? So this term is the same. Great. Then this term is again the same. Right. Yeah. This term is again the same. Then this guy is brought back in. And this guy is not put into the disturbance. So this term, I believe, is brought back in. So because, so this term is the same here. Then you have W tilde transpose sigma hat, which is the same here. And then this W tilde transpose sigma hat prime V transpose X is brought back in. Yeah. So this, this term is put back, which is what I also said does not make sense to keep it out. Okay. And then this term remains as it is. And then a V. And now the W1 becomes a W. Right. So this makes a little bit more sense. Right. And here I believe what is left. Let's see carefully. Ah, I see the whole term is not brought back. Right. Right. Let's be careful. This, what is brought back is only the, so here you had the W tilde transpose sigma hat prime V tilde transpose. Yeah. And what is brought back is only the V hat transpose term. Right. And what is left is the V transpose term. Right. So because V tilde transpose is V minus V hat. So the V hat term is brought in here with the negative sign. And the V term is left out here. Okay. Interesting. I guess the purpose of this is primarily to make sure the Lyapunov analysis goes through. And of course, if W tilde's are small, then this is sort of justified. Yeah. So if W tilde is small, then keeping this as disturbance is sort of justified in some sense. Okay. All right. And this is anyway a second order term. Right. So yeah. So these, these three being here was never a big question. But this, this being here and this being here does pose some question. But then you have to sort of think of W tilde having some kind of a nice bound. And then you can say some nice things because X is bounded and all these quantities are also possibly bounded. Yeah. All right. So of course, then we need to, you know, get a bound on W one to overbound W. Right. So this is what is shown in this next fact. So the disturbance term 31, which is this guy W is bounded in this way. Right. Or of course, I mean, it's, it's probably nicer simpler type of expression here. Right. So here you have some constant. Let's not worry about it. Then there is dependence on the Z tilde, which is expected because well, W tilde appears here, obviously as you bounded in Z tilde. And then there is bound correspond, there's terms that have multiplication of Z tilde and R. This also comes, I believe from this guy because this second order term is bounded by V tilde times R. So therefore the whole thing is bounded by Z tilde times R normal Z tilde times. So Z tilde is just V tilde and W tilde stacked. Right. So this is again another bound, you know, with respect to the state and the estimate bounds. Okay. Great. So now once we have all these bound, we have this sort of a closed loop system with whatever we have declared as some kind of a disturbance. Right. So this is our final system, you know, with this kind of a disturbance term. Right. And so what we now want to do is how to do the update propagation. Right. So this is what is the important thing. Right. We want to see how the updates are propagated. Right. So it is required to demonstrate that the tracking error is suitably small and then the neural network weights remain bounded for only then the control is found. Right. So this is important. Right. So if you assume a few things, this is where the ideal case the back propagation tuning, which is like the basic neural network tuning method. Here there is a lot of idealization. So the assumption is there's no net functional reconstruction error. This sort of means that epsilon is zero. Right. What's the next one? There's no unmodeled disturbances and no higher order terms. So essentially this second one means d is zero. Sorry. Tau d is zero. And the third one means that, you know, your unmodeled terms are gone. So you have whatever, not unmodeled higher order terms are gone. So o v tilde transpose x equal to zero. So there are no higher order terms. So if you make these kinds of assumption, so yeah, it's as good as assuming that fx is linear function. It's as good as assuming that fx is a linear function. And then of course things are much more simpler. You go sort of back to your typical adaptive control type domain. All right. So in that case, this theorem proposes a nice update law, which is this guy and that guy. Right. So, so in fact, in fact, this theorem is rather nice, because it looks directly at the w1 disturbance. Let the desired trajectory be bounded. So there's a lot of assumptions actually. Suppose the disturbance is equal to zero. Okay. And then you also assume that v is zero. So this is very nice and simplistic, I would say, because you're assuming this w1 is zero, which means you're sort of assuming a lot of nice things, not just this. So this guy's zero is okay. Right. This guy being zero is okay. But then you're also assuming that this quantity is okay. So this is of course, I mean, I would say pretty restrictive. Right. Then the tracking error is just go to zero and all the nice things happen because everything is nice and linear. So you expect all the nice things to happen. So, see, so let's sort of think about it. Okay, I see, I see. So the higher order terms are zero, then this being zero also makes sense. So this is not such a bad, I mean, once you assume that there are no higher order terms, that is, if f is linear, then this is zero. So this is fine. Actually, this term will be zero, because sigma hat prime, you know, is this this guy basically? Right. This sort of this guy. So, so the activation function that you have is, is there's no activation function per se. I mean, it is linear. Right. So this, this term is of course, going to be zero. All right. So this term is of course, going to be zero. So because if there is safe, if so basic point being that if f is linear, then there is no higher order terms, I have to assume, this become zero is the question. So this is the higher order term. If this is the higher order term, do we need an and this is zero, then what is the activation function that we need an activation function would be the sort of question. Yeah. So this is also an interesting assumption, I would say. And because you're taking the derivative of the activation function with respect to an argument and the activation function, if the if the overall function fx is linear, then the question is, do you need an activation function or do you even need the higher layers? Right. So that's the whole point. Yeah. I believe the activation function becomes the identity function in this case. Yeah. If the function f is linear, then the activation function is simply the identity function. So sigma of z is actually equal to z. So if I take a partial, it just gives me, it just take the derivative, it just gives me one. So this quantity should be just one. Yeah. I'm wondering if I'm wondering if that is how it will be. But if this quantity is just one, then this is basically v hat transpose x. So basically you don't have anything like a Taylor series expansion because this is, I mean, in the linear case, this I believe will simply become v tilde transpose x kind of a thing. Yeah, there is no sigma. I mean, sigma is just going to be the identity function is what I believe will happen. Right. And then, right, right. But then I wonder what is the, I mean, this, why will this still go to zero? Why would this still go to zero is not very clear. Suppose the disturbance can be, here the authors are pretty much making an assumption that w1 and 28 is pretty much going to zero. And so the question that we are trying to answer to us is, of course, this is zero by the assumption is no problem. But what happens to this term is a big question. What happens to this term is a big question. If it's linear, then the activation function is linear and the derivative is simply one. But then I still have v tilde transpose x. So sigma tilde is basically v tilde transpose x. And then I'll be left with w tilde transpose times v tilde transpose x. So this term, it's not clear why this would be zero. So this term, so I would say this guy only possible if w tilde transpose v tilde transpose x is also zero. Yeah. So if you assume that this is also some kind of a second order term in the parameter errors and the parameter errors are relatively small, then yes, this term also goes to zero. Yeah, I mean, this is an interesting assumption, I would say. This is an interesting assumption, is what I would say. Yeah, I'm not very sure of how that will be. But if it does happen, then of course, life is super simple. I take my usual Lyapunov function R transpose MR, which is what I would expect. And the second and third terms are basically these update law type terms. I hope you remember these kind of terms from your model reference adaptive control because the unknowns are matrices here. So you sort of take the trace with some gain matrix FNG, just positive definite symmetric matrices. And then you take the derivative, you get something like this. And if you substitute everything carefully with w10, all you're left with is just these guys. Yeah, w1 equal to zero, all you're left with is just these guys. You sort of have these terms, if you substitute for r dot, you will have m dot minus 2 vm, which is going to go to zero. And then you'll have the nice minus kv term because of this guy. So this guy becomes this term. So that goes to zero because of the skew symmetry property. Then you have this nice negative term. Then you have all the, you know, sort of the tilde terms, right? So sorry, you'll have this sort of a tilde term. And that, of course, gets combined here nicely using the trace functions. Right? And this, this is what essentially lets you choose your v hat dot and w hat dot. So from, from trying to make these quantity zero, right? So what I'm trying to see is which particular equation the authors have used here to substitute for r dot. So here I have, this is coming from just the derivative here, right? And this guy is coming from the equation. So here you have w tilde transpose sigma hat r transpose. So r transpose is of course from this. So w tilde transpose sigma hat, that's coming from here. So that's this term. So this term is due to this. Okay. Now, if I look at this term, this is w hat term. This is coming due to this guy. Yeah. So this two terms. And so w one is zero, v is zero. So none of these are there. So now if I choose this kind of an update law, everything is rather straightforward. You have a nice tuning function. I mean, you have this nice L dot is minus r transpose Kvr, which is of course, a negative semi definite, but you can prove that r goes to zero and you're done. Yeah, because r goes to zero, e and e dot also go to zero. All right. Great. So what did we see today? We started to write the system under the control where we added a robustness term to the control. Right. And then we sort of did some kind of nice clubbing of the disturbances. Some of it is based on inclusion and other one more or less to help your Lyapunov analysis go through. And of course, these are all based on some kind of approximation. So none of the results that you get will be like precise tracking, et cetera. It will all be boundedness results. But then we started with this back propagation type tuning, which helps you give, you know, this kind of, you know, it sort of makes a, I would say, not a very good assumption that all the disturbance are essentially zero. And, but then what it can also give you is you had zero tracking errors and things like that. So it's a very, very idealized situation. But we do, I mean, this is sort of a first step. You can use this back propagation tuning to still get some interesting results using this neural network that is tuned via an adaptive control. So we will, of course, continue with this in our subsequent final session also. So I hope you're enjoying our discussion and I hope to see you again.