 Welcome to our final session of nonlinear and adaptive control. I'm Srikanth Sukumar from Systems and Control, IID Bradley. I'm very, very thankful to all those who have attended this course. And I really hope I've been able to generate enough interest in the topic of nonlinear adaptive control. It is a rather rich area. And it is, of course, a rather powerful set of tools in nonlinear control, which is what makes it one of the most applied areas in nonlinear control. For fighter jets like what you see in the background here and also for spacecraft, these have even been tested in actual flight conditions, right? And of course, there are also companies that specialize in adaptive control for drones. So a lot of applications, again, I focus mostly on the aerospace applications or aeromechanical applications because of my own background. But there is quite a bit of work out there on electrical, biological, chemical systems also using adaptive control. So what we have been looking at in this last week is a little bit of what you see in the bottom right of our motivating image in the background, which is on neural networks. So as you can see here in the image, this is like a deep learning network, so essentially a multi-layered neural network. So it's typical practice in neural networks to have several layers now because again, we have a lot more computation and power and a lot more data available to train these networks. So the typical approach is to have an offline training where you input a lot of training data on the left of this network and you get the output on the right of the network and use this input output in order to tune the weights that appear in this sort of nonlinear function approximator. And then you use these weights in order to do actual experiments. I mean, when you go to the actual run, you design your controllers based on these weights. So this is what we have been looking at. We of course are looking at a rather special situation where there are a couple of things. The weight or the learning itself of the weights is happening online along with the control. And secondly, in order to make sure that this learning plus control mix is stable, we use adaptive control principles. So I hope you have seen by now that the adaptive control is essentially very, very closely connected to deep learning. And we also saw that deep learning is now prevalent also in reinforcement learning and other classification approaches. So over and all, I hope you understand that these stable adaptive control methods are of significant use in learning. So we are of course looking at a very specific case of this three layer neural network being applied as a function approximator to a multi-link robot. And because it's like a neural network approximator, we don't really care to write out these mass inertia functions and the Coriolis functions by hand for every single robot. If you bring in any robot, this is like a plug-and-play controller. So if you have a control module, you can plug it into any multi-link robot which has this kind of an Euler Lagrange system structure like you see here in equation 7. And you can start controlling it because your neural network using this adaptive law will automatically learn all these non-linear robot functions. So this is what is the cool aspect that it is agnostic to what kind of MVGF you have, the neural network function approximator is going to learn it. And how the learning happens via the adaptive update laws is what we saw last time. So we sort of added a robustification term and of course, we sort of wrote the dynamics in a rather nice way either in form 28 or in the form of equation 30. So it was either in the form of equation 28 with the disturbance terms w1 or in the form of equation 30 with disturbance terms labeled as w. So the first result then that we saw was on back propagation, which was a rather ideal situation where we assume that these disturbances are zero. We saw that it's not very easy to sort of claim w1 is zero. It's obvious if you assume that the second order terms are zero, so your function is a linear function and you don't have these, you know, it's an exact estimator and the disturbance is zero. So these terms will become zero rather easily, but we realize that this term will become zero only if you know you have this kind of w tilde v tilde type of a thing. So this is a sort of linear and parameter type of assumption on the function. If you want these to be zero and if this does happen, we did have this back propagation based tuning of weights. So this sort of math, why it's given this name is because this sort of maps with to the sort of weight tuning algorithms that standard nonlinear neural networks use offline. So this is the update law. I mean, and if you notice that, I mean, you have to remember that this Sigma hat was essentially, you know, Sigma hat is just defined as Sigma of v hat transpose x. So remember, this is just notation. So Sigma hat is just v hat transpose x. So this was a notation that was used. And so that notation, so you can see that w hat dot depends on v hat and v hat dot also depends on w hat. So they are sort of interdependent here. Yeah, and we also saw how the Lyapunov analysis goes, right, because of this ideal circumstance where you don't have this w one and we just make them zero here. And you look at this equation. So all you have is so this term goes to zero because of this Q symmetry, then this term contributes to a nice negative term. And these two terms are of course, cancelled using the update law. And then you have, of course, this L function, this Lyapunov candidate function L, its derivative is negative semi definite. And then you can use your Lassall invariance or power bloods lemma signal chasing to prove R goes to zero, right? Excellent. So, so this is sort of where we were going to mark our final lecture here, right, starting here. Like I said, you can use bar bloods lemma signal chasing or you can use the saline variance to claim that R goes to zero. And once you have R going to zero, you know that you also have error and the derivative of the error tracking error going to zero. And that's essentially what it is. So few important points. The first in the introduction itself, the authors mentioned that most of these neural network based laws require very specific or very good weight initialization. So that problem is does not arise at all. Right. In fact, if you see, if you take the initial conditions on the weights, a W hat, we had to be zero, you see that the right hand side of this guy and this guy becomes zero. So it's an equilibrium. So this entire update goes away. So you will stay at W hat and we had to be zero for all time. So basically, it takes the neural network out of the circuit and has the only has only the outer tracking. Right. So, so of course, basically, if you, I mean, you can do some kind of a local stabilization using this KB term. Right. So if you have a large enough KB and if the filtered error is R is small enough, then you're fine. Right. So, so anyway, so of course, it's not possible to choose such large KB for several practical reasons. Yeah. So, so anyway, so this is, so this is the connection to the back propagation algorithm. So this is equation 33 and 34 are matched with 37 and 38, which is what is known in neural network theory as the back propagation results. Right. Great. So theorem is when it reveals that, you know, when, when only this back propagation alone will suffice. And this is when this term is exactly equal to zero. And, and this is what I was saying that this first term in 29 reveals that it's a rather strong assumption than just in linearity of effects. Right. So even if epsilon is zero tau d zero f is linear, this is not guaranteed. Yeah. So, and of course, fx is of course only linear in the one link arm place. So there's a lot of assumptions. Right. A lot of assumptions. So this is a, I mean, so w1 being equal to zero is a very strong assumption. Right. So, so of course, then there is, you know, additional results that are in this article also. So that's what is there in theorem 3.2 it reveals the failure of simple back propagation in the general case. Right. Basically, it can be shown. The authors themselves. I'm not sure if they're showing it themselves. Right. So we'll not worry about that right now. But the point is it can be shown that the weights become unbounded. Right. I mean, all of this is because of the assumption that w1 is equal to zero. Right. W1 is not really zero even for the linear fx case. And even the linear case happens only if fx is only the robot is a single link to it. Right. So, so this is what the authors say are generally not bounded under the hidden layer who bases stringent persistence condition. Okay. Great. Now, therefore there is a need to look at the more general case. Right. So, so of course there is need to modify the tuning weight. There is the need to add a robust defined term. Okay. So, both of these are there. So, we already added a v because in the previous result we assumed that v was exactly zero. Yeah. And we gave a, I mean, set of weight tuning rules. Now, of course, the authors will give a slightly different weight tuning. Okay. Great. So, so this is what you know is going to be detailed in the second theorem. Right. So, of course, this, I mean the disturbance tau d and the reconstruction error make it impossible to show that L is L dot is non positive for all R. So, therefore, you can only show that it is L dot is negative outside a compact set. So, this is what is as usual R. Right. The residual set type result. You are used to this. Right. Because if there is disturbance, there is an epsilon error in the function approximator, then you are used to the, I mean, you have to have some kind of a residual set performance rule. You cannot have exact convergence. So, one should not even expect any exact convergence. All right. So, this is the first important thing to remember. Yeah. Okay. Right. So, see, so this is the sort of robust defined term that the authors introduce. Right. So, this is essentially the robustification term. It contains of the, I mean, honestly, not a smooth term. It's a non-smooth term, if you may. And this term is like a norm of Z hat. Obesity is norm of Z hat. And then, of course, you have some state dependent terms. And then with some gain, which is larger than some constant C2. Right. And this constant C2 is known in equation 32. It's coming from the bound on W. Okay. So, the idea is sort of to take care of this sort of term. Right. So, we see how that happens. Okay. And then the neural network weight tuning is changed a little bit. If you see, the second term is not very different from what we had. So, you had GX sigma hat prime transpose W hat R transpose. And this is exactly the same. GX sigma hat prime transpose W hat R whole transpose. It's exactly the same. But the first term was F sigma hat R transpose in the back propagation term case. And here it is modified to include two more terms. Again, there is a non-smooth term here. Yeah, but it's a nice continuous term, but a non-smooth term. Because it's a norm R type of term. Okay. And this term is also written in terms of a norm R type of term. I think this is not equal to this. This is what I'm doing. Let's see. Let's see. This is the second place. Just give me a moment, please. I think that term is not quite right. I believe this is not equal to, this is actually a minus term. And this is not an equal to, but this is a minus term. Yeah. That's a typo here in the paper. Right. So, that's fine. So, this also, the V hat dot is also changed and the W hat dot also changes. The ideal version. And then you have these new terms. Okay. And now, how you, and essentially what you can prove is that for large enough KV and the filter tracking error R, these become uniform, ultimately bounded with practical bounds given specifically by 43, 44s. Right. So, this is what you sort of obtain. Right. Which is what you sort of obtain. So, you, of course, assume these nice approximation properties hold, which is this, you know, for the function approximator with some epsilon n. And we assume that this also holds not everywhere, but in this nice set. Okay. This nice set. And again, you have, I mean, UR, which is a set which is correspondingly defined for the R value. Okay. So, we typically assume that these bounds hold within a certain set. Right. So, then of course, this approximation property holds here. Okay. For this set R in U, R0 in UR. Okay. So, now as usual, you do the Lyapunov derivative. So, this is the nice term. Then you have this Q symmetry term, which is going to go away. Then this first term is coming just from the derivative of our same L. The L is not changed actually. The L is still the same. So, this is the term. So, the first term here, so this term is coming from this. Okay. And then this is coming from your R dot. Okay. So, these terms are coming from R dot. So, in fact, this term and this term is not different from what you had in the previous case. Right. Except for, of course, the fact that you've introduced a V. And so, that sort of also has to show up here. Those terms also show up corresponding to the V here. Right. Because you have this kind of a term. Okay. Great. So, and then then you have this term, which was again same as before. So, so the additional terms if you see are, you had a Sigma hat R transpose and this term already. Right. So, the additional terms here are, so this two terms were already there. So, the additional term is sort of this term. Okay. Let's see. Is this an additional term or this two was already there? Ah, yes. This, this is an additional term because we are using equation number 30. So, we are using this guy. Yeah. So, the, so this guy results in this term and the V. The V term is coming from this. Yeah. So, this guy accounts for the first term here. Yeah. So, everything else is the same. In fact, my mistake, the V is not even stated right until now. Okay. So, once you do that, the authors apply the tuning rule and they do a bunch of simplification. I'm not going into the complete details of this. I would ask you to verify the steps here. Yeah. In order to get to this, I'll ask you to verify the steps. So, this nice negative term still remains. Yeah. And then you have these kind of W minus W tilde V minus V tilde and R transpose W plus V type of term. So, V is still not substituted. Notice. Yeah. So, of course, there is a, you do these kind of nice trace equalities. Yeah. And then you bound these trace equalities. And once you do that, you start working with the norms everywhere, the vector norms and the matrix norms here. And this is where, of course, you introduce the expression for V also. And that expression for V gets substituted here. Right. So, again, I'm not going into the details. I will ask you, I'll request you to sort of verify these details. These are simply just using standard normic inequalities. Yeah. And then this particular equality and inequality right here. And after substitution of V, right, you get this kind of a final expression. Okay. This kind of a final expression. The KV min is, of course, minimum singular value of KV. Right. And the last student quality holds due to 40. So, so basically 40 is this guy. All right. So, a lot of, a lot of the early inequalities that were mentioned have been used here in order to arrive at these bounds at this nice bound here. Right. So, now, if you do a nice completion of squares after defining this kind of a, you know, this kind of a new variable C3. Yeah. So, what is the completion of square? The completion of square is, you know, being done here on this term. So, this term is this guy. And here is what you have here. Right. And you sort of nicely break it into these pieces. And of course, this is where you sort of, you know, you sort of do a standard completion of squares in order to get your residual set down. Okay. And that's essentially what is being done in these steps. All right. That is essentially what is being done in these sets. So, you get two different sets. Right. So, one set is, of course, obtained from here. This guy. Yeah. That is, you know, here. C3 and this. Right. And then you have another set on R. So, one set is on the z tilde, which is a parameter error. And the other set is on the R, which is the state errors. Right. And it's, what you get is that L dot will be negative outside this compact set, which is defined by these two inequalities, 43 and 44. Right. And therefore, you have standard uniform ultimate boundedness for both R and z tilde. And because z tilde contains both V tilde and W tilde, you have what you need it. Right. Right. So, one thing is obvious that any excursion beyond this 43, 44 lead to decreasing Lyapunov function, which means that you will get back into the bound. So, z tilde and R are, of course, restricted within this residual set. Right. So, anyway, I mean, these bounds are, of course, dictated by, you know, number of higher order terms in the Taylor series and the magnitude of the Taylor series bounds, etc. Right. And of course, as you know, that the first terms are nothing but the back propagation algorithm itself. So, this is what is the standard back propagation. And the last terms look like the epsilon modification. So, if you look at these terms, these are sort of like, so these are like the standard back propagation. And these terms look like your epsilon modification because this contains a V hat term. Yeah. And this is basically a W hat term. So, this is like an epsilon modification. Right. And why it's an epsilon modification instead of a sigma modification? Because the additional terms are scaled by the state. Right. Remember, in the sigma modification, the W hat and the V hat terms will be scaled by a constant. Here, they're not scaled by a constant. They're scaled by the state value itself. So, why it's better is because if R becomes close to zero, you don't stop learning the parameter. Because in the constant game case, if R goes close to zero, these terms are dead. These terms, if R goes close to zero, these terms become zero. But this term continues to dominate and push W hat and V hat. We'll continue to dominate and push W hat and V hat towards zero. So, whatever parameter value that was learned is lost. So, this is like an epsilon modification because it is also scaled by normal. All right. So, that's the whole idea. So, this is like an epsilon modification. Right. So, as I mentioned, we will not look into the, you know, passivity properties of the neural network right now. We want to just quickly look at, you know, like, you know, design. Right. So, this is like a standard adaptive control for a, you know, like a two-link robot. This is what planar tooling come. Yeah. So, this is like a standard, you know, set up for a lot of experiments in adaptive control. Right. So, then you have the desired and actual joint angles here. So, you have Q1, Q1 desired and Q2, Q2 desired. Right. And then you also have, you know, mass estimates. Yeah. The estimates, of course, don't converge using. Yeah. Don't look like they converge to true values. Yeah. But whatever. I mean, this is not what you guarantee in adaptive control anyway. Okay. So, note the good behavior which obtains, since there are only two unknown parameters. Yeah. So that means, so you have nice persistence type results. Now, if you add some unmodeled dynamics, right. So, some unmodeled terms, dynamical terms in the simulation, you add and you still implement an adaptive controller, not a neural network-based controller. You start to see significant deterioration in the performance. So, Q1 is like this. You can see, and Q1 desired is this. And similarly, Q2 is this, you know, this guy and the Q2 desired is this dotted line. Yeah. So, you see that things sort of match for some time, but they don't match for a while and so on and so forth. And the estimates, of course, look rather bad. The estimates start to look rather bad. Okay. Great. Then you implement like similarly, you know, I mean, like a back propagation-based weight tuning to this kind of a problem. Right. So, and you basically look at this, the response of a neural network and you see that you get nice better tracking because Q1 converges to Q1 desired even with this unknown, you know, unmodeled dynamics. And similarly, Q2 converges to Q2 desired. The weights, of course, don't seem to, you know, do nice things still. Right. And then, of course, we add this improved weight tuning. And you see that you get, of course, slightly quicker tracking, better tracking performance and, you know, mass convergence also sort of looks similar. Right. So, and then, of course, we see the controller performance with and without the neural networks. So, that, of course, is not matching at all. Yeah, as you can see. So, this is sort of what the, you know, so, so that's what this is sort of highlighted to study the contribution of the neural network. Figure nine shows the response of the controller that is with no neural network and standard results in robotics indicate PD control should be bounded errors if KV is large enough. This is observed. This is how the addition of the neural networks has made a significant improvement in the tracking performance. All right. That's the idea. Right. I mean, so how they sort of illustrated value of this neural network type method is that they started with this nice baseline robotic model. And then they, you know, they added some terms into this, you know, the system matrices like the MVM and GVNF and all that. So they added just one term. Right. So, so, so basically they see that there is a deteriorated performance here. You know, if you don't have a neural network type controller, then so basically is this is like, you know, I mean, you see the performance here using the back propagation is the idealized weight tuning also, where we assume some disturbance terms are exactly zero while they're not. And then all these quantities are still the same which is this, there is an additional term. Right. And, and here you see that you sort of have relatively better results in terms of the tracking performance. A weight convergence anyway is not completely guaranteed even here, unless there is some kind of a persistence, but you do have, you know, very nice tracking performance. And then of course you show improved tracking quicker tracking by using the, you know, like a modified weight tuning method. And that's what is sort of illustrated here. So in conclusion, I really hope that, you know, you sort of got a good feel for how adaptive control is very closely connected to deep learning. And the only difference here in adaptive control is that we look to prove stability and therefore analytically especially, and which may make, you know, designing all sorts of arbitrary tuning laws difficult. But that's essentially what we do in adaptive control also. We design tuning laws and we move stability with that. And in this case, stability would mean some kind of good function, function approximation and therefore good tracking. Yeah. So the other thing is because we were doing this adaptive control-based learning, we also were doing everything online. Yeah. Of course you can use the same laws and do things offline also because we saw that this back propagation is essentially the, you know, based tuning that we did in adaptive control law with adaptive control law is same as what we have for the standard case. Yeah. You know, like your offline learning type of situation. All right. So we are sort of at the, again, our course. And I really hope that this last week did give you a very good idea of how relevant adaptive control is even today in the sense that it has very, very close ties to deep learning and hence to other forms of learning, like reinforcement learning and classification problems which are still connected to some kind of parameter estimation and parameter learning. I really hope all of you enjoyed the course as much as I enjoyed delivering it. And I am of course always open for feedback, comments, applications, cool new results, cool, cool new experiments. Yeah. So I hope you will stay in touch through this course and learn more about the avenues that nonlinear adaptive control efforts us. Thank you. And I'm very glad for all of you, for all of your learning and the fact that all of you listened to what I had to say for these times. Thanks.