 Hello. Welcome to get another session of our NPTEL on nonlinear and adaptive control. I am Shri Khan, supermarker of systems and control IIT. So we are in the final week of our course on nonlinear adaptive control. And I really hope that you have learned sufficient material to be able to design algorithms that can drive autonomous systems such as what you see in the background. Now, as always, we're interested in all sorts of applications. Of course, what you see here are mostly aeromechanical systems in the background, which is due to my own interest in these. But of course, adaptive control designs are not restricted to these. And you can use it on power networks and biological networks and so on and so forth. So I'm always open to hearing more about sort of applications that you folks are envisioning the use of adaptive control. So what we are looking at in this week, in this final week is the connections with learning. And we are now specifically looking at sort of a deep learning problem. But of course, we are, we are sort of different in the typical, different from the typical learning sort of results in the sense that we are not doing the learning offline then implementing it later, but we are doing learning and control simultaneously. We can do this because essentially this is connected through parameter learning, which we already know very well. So that's what we sort of apply. The only difference or which is of course a significant difference is that now we have a very nonlinear regressor parameter form, which is not what we were used to until now. We are always assuming linear parameterization and things like that. But you see here that everything is rather nonlinear. And so we still have to see how we can deal with this kind of nonlinearity. In the previous lecture, of course we started to understand some details about how my robotic arm dynamics looks because this is the application that we are going to be looking at for application for the use of these neural nets. And we saw the basic all electrical dynamics, what each term means, what is the joint on the world space coordinates. And we also constructed this error variable with respect to a desired arm trajectory. And we also, you know, defined this back stepping error type variable, which we have seen before. And we wrote the dynamics in terms of this back stepping error variable. And in this dynamics now we have this function F, which is called a nonlinear robot function. And this is the function, as you can imagine, we will approximate using our G layer neural network. So, but this is where we were I'm going to mark our lecture here. So lecture 12.4. Thank you. So, now, if you look at how we want to design our controller, what we do is put in an approximation or estimation of this, and that's called a factor. So, unlike what you've seen in adaptive control before, we are not approximate, we are not just estimating parameters, but we are now estimating functions. And that's why we have the notion notation F hat. Okay. And then of course, we have a KV times R, which is like a gain matrix, symmetric post-doctoral gain matrix times this back stepping error variable R. And so this is the control. Yeah, so it's called tau zero, because we designed the control sort of assuming that there is no disturbance, right? We cannot do anything to sort of really cancel the disturbance here. So we just design a control assuming there is none. And this is, you know, you sort of try to cancel this guy with its estimate, just sort of getting motivated from the certainty equivalence type idea. And then we leave this term as it is saying, because honestly, this term is going to cancel out on its own. So we don't worry about this term. Great. Now, so as, as we mentioned, F hat is an estimate of F by some means, which is not a dispute. So what happens to the closed loop system? Right. So, there is this notation here. So the closed loop system is not KM, but the VM becomes this guy. Minus KV plus VM times R and plus an F tilde and this tau D. And this is where you have sort of this F tilde, which is the function estimation. So this is just a term Zeta naught, which is combining these two. Nothing special about this. All right. So this is essentially like an error system. So I'm going to highlight this. And so this is sort of the error system. And of course it's driven by the function estimation error. So the F tilde of course shows up here. So, so this is sort of like, I mean, although the tau zero seems to have only one term, which is KV, I mean, KVR. I mean, there's two terms, but one of them is to more or less counter the effect of the nonlinear robot function. And the other term is what is the like the stabilizing term that we usually inject. And this term, although it looks like just one term, it's in fact a proportional derivative term because of the nature of R. So I have a V dot term and an E term. Right. So, so this is important to remember that it has both a proportional and a derivative term. So a PD type controller. Very, again, something that is very standard in aeromechanical applications. Even for space stuff, PD type controllers are known and found to be stabilizing. Yeah. So in the remain, of course, in the remainder of the paper, we mean the authors use this equation 14. Yeah. And we, of course, focus on tuning this neural network in a smart way so that you can get R to go to zero because you want R to go to zero. All right. Great. So why is it? I mean, one, one's very quick aside. Why is it sufficient for R to go to zero? I want to quickly point that out. Whatever R is defined. Okay. So sufficient for R to go to zero. Yeah. Why? Because this means that E dot plus lambda E equal to some phi function. This goes to zero, which means E dot is equal to minus lambda E plus five. Right. And what do I know about fun? Suppose if phi is also bounded, we can prove that phi is bounded, which means R is bounded. If, let's see, I have put it properly. If R is bounded, that is L infinity and R, of course, goes to zero implies phi is bounded and phi goes to zero. Right. Which means that, and what do I know? I know that this is a stable system. Right. Because lambda is positive definite symmetric. So this is a stable system with a bounded forcing. So you already prove this kind of a result when we did this or take a construction a long time ago that if this E dot plus lambda E type thing goes to zero, then he has to go to zero. Right. So this immediately implies that E comma E dot both go to zero as T goes. Yeah. So it's enough for us to show that R goes to zero. So that's what we aim to do. This is essentially like an ortega type construction idea. Right. Even in the ortega construction, we did the same thing. Right. Great. So if you don't remember again, I would ask you to go back and refer to the ortega. All right. So that's what it's mentioned here. Yeah. Right. So E exhibits stable behavior. In fact, E is less than R. I mean, even if you don't have, you know, I mean, of course we are talking about R going to zero in this case, that may not happen. But even if our remains bounded net has some nice bound or just by virtue of this equation that I know R is E dot plus lambda E, then it's it can be shown even without talking about things going to zero because honestly speaking in this article we'll not be able to prove that anything goes to zero percent, but we only prove nice bounded performance because after all, you know, whenever we talk about using neural networks for functions, it's eventually an approximation of the function. So there's always some error. So obviously if there is some error, you cannot expect just like in case of the disturbance, you cannot expect to put things to converge to zero. Yeah. So the best you will do is get some kind of bounded performance. So just with the relationship between E and R, you can actually conclude that the two norm of E is bounded by the two norm of R. And similarly, the two norm of E dot is also bounded by the square of norm R square, norm R square. And so this is important to remember. Okay. Then we also have very standard properties of the robot model. So M is basically positive definite symmetric and therefore it's satisfied this kind of an inequality. Right. So you remember that whenever you have positive definite symmetric matrices, you have this kind of an inequality. Yeah. The second property is that VM is bounded. Yeah. And by some, you know, continuous function multiplied by nine. Right. This is again a property of the centripetal Coriolis term. Okay. And the third property, which is a very important property in the Lyapunov analysis is that the matrix M dot minus twice VM is two symmetric. Right. What does it mean? It means that any quadratic form, which is alpha transpose M dot minus two VM alpha is zero. Yeah. So the, so the quadratic form corresponding to any symmetric matrix is always zero. And so that's what we have in property number three. Of course, we assume that the disturbances are bounded. So, not on this, but yeah, we assume that the disturbances are bounded. This property five, we don't use yet. I mean, we don't talk about it. Yeah. Which is why there is a sign on an R. Yeah. That's why you had the sign on here. Sorry, the Zeta naught here. Right. So if you remember the Zeta naught, that was sort of inserted here. Yeah. The purpose was to talk about passivity, but you remember that we said that we not discuss the passivity aspects of this article in these cities. Yeah. So as of now, we skip it. We need to talk about it later. Right. This is a property. Yeah. So this is not an assumption. These are properties. Yeah. So it's not like we are assuming anything. So of course, this property can also be proved. There's a nice reference that's given for this. All right. Excellent. Excellent. So now we know that we need only good behavior of the R. And we have the dynamics of R given by this equation 14 here. Yeah. Which is also something nice. Now if F tilde was zero and tau D was zero, then this is well known to be a stable asymptotically stable system. Yeah. So if both of these are zero, then you are in very good shape. No problem. Yeah. Now the issues happen when this is non-zero. And of course, there is disturbance. So of course that also if it's non-zero, then, you know, this is at least bounded. So you want some bounded performance. So what you need at the least is that you have some nice bound on this F tilde. So this is the least you want. So you want a good function approximator. So that is why you start talking about the neural network control. Yeah. That is where we start discussing the neural network control. Right. So, so that's what we say that this non-linear robot function should remember. 11. So this guy. Yeah. This whole thing assume that it is given by a neural network as in three. And so it's essentially like a the function effects is replaced by this kind of a weights thing. I mean, you had it here. You had it here. Yeah. Suppose this function effects is approximated in this manner where of course, epsilon is small enough. And then you have these weights and offsets and so on and this activation function. So it's like a approximated by a three layer neural network. Now, why can we do this? Because by this nice theorem 2.1, we have that, you know, you can approximate almost any continuous function in this way. And effects is of course continuous. So, of course, we have this fact that this error is bounded by some constant episode on end. So you always need bounds on the other side. If you don't have bounds on the errors, then you can't get bounded performance. Right. So, of course, there is results would say that these ideal weights are not necessarily unique. Yeah. So, so I mean, that's one cannot expect like, you know, when in standard adaptive control problem, the value of the parameter is sort of fixed and known. For example, if you say the mass of your drone is unknown and you use an adaptive controller, so the mass is a fixed quantity, right, you know what the masses. Well, you don't know what the masses, for example, but you know that it is a fixed quantity. But in this case, that's not necessarily the case because these are not masses and properties of the system, but these are more like weights to some kind of sigmoidal function which is used to approximate actual nonlinear function. So therefore, the choices are not necessarily unique. So, we should put this in. So, of course, this according to theorem 2.1, this mild approximation assumption always waits for continuous function. Right. So, but so anyway, so this is of course, this is of course, the important part here. So, then of course, just for we define a new notation, which is this z equal to 0 0 v, this is just to make our lives easy in terms of notation. Right. Now, before doing any actual neural network type of design, what we want to do is to look at a few different bounding assumptions and facts. And some of them are assumptions, of course, and some of them are facts. That is to say that some of them are guaranteed to hold true and others have to be based on some assumptions. Right. So, the first is an assumption. The facts are easy to prove given the assumptions. And therefore, we start with the assumptions. The first assumption is that the weights are bounded. Okay. This is a very fair assumption. I mean, if the weights are not bounded, then possibly your function X is not bound. Okay. So, having bounded weights is a very reasonable assumption. Yeah. Good. The next is that the desired trajectory is also bounded. Yeah. So, desired trajectory is defined by QD. But here we talk not just about QD, but about all the derivatives or at least two derivatives of QD also. Now, this is, again, no different from the assumption which we made in our previous adaptive control programs. We just said it in words. We said that you have a bounded trajectory with bounded derivatives. Yeah. So, here we are making it a little bit more specific. You have taken QD and it's two derivatives. Right. So, it's, again, a very reasonable assumption. If you want your system to be following trajectories which are too sharp or too jerky and so on. So, therefore, you do assume that you have bounded reference with bounded derivatives. This is, again, something very reasonable. The important thing here is, of course, that this is a known constant. So, you know this value. Similarly, here also you know this constant. So, the important thing is that you know these bounds. In case of trajectory, of course, this is not difficult. But in case of weights, knowing a bound is sort of the kind of thing we assume when we talked about projection in an active control program. Right. So, in this case, we are, you know, posteriori assuming that there is a known bound on this, that we are trying to learn. Then we come to the first fact. It says that for each time T, X is bounded in this way. Yeah. So, X is bounded means it's not like a uniform bound or anything. But we know that X contains of, consists of what? X consists of Q, well, actually I should probably point it out here. Yeah. So, if you look at this, I'm sorry, we define X, yeah, right here. So, the X is defined this way. And so, X contains what? It contains QD and its two derivatives. So, these we already know are bounded by the cap QD, by our assumption. And then the error contains E and E dots. So, at least it contains what? It contains again Q and QD dots. Q and QD and QD dots and Q dots. Right. And this is of course bounded by some R along with some QD. Right. So, this is not difficult to see because that you can actually get this sort of a bound on X. Yeah, like a linear bound with respect to a fine bound with respect to R. Yeah. Right. So, now, you know, we sort of want to talk about using some kind of Taylor series approximations so that we can in fact do a deal with these nonlinear regressor parameter. Yeah. Remember, the linearity in the regressor parameter forms is what helped us design most of our adaptive fin solace. Right. Without that, we would be in quite a suit. Yeah. So, this is what I mean. It is E for extending linear NN to nonlinear NNs. Right. And this requires use of Taylor series based results. Right. So, this is what is important. Right. I mean, eventually when we have a nonlinear regressor parameter for the nonlinear terms in the neural network, we still want to look at some linear versions and we do it by taking Taylor series approximations of these nonlinear terms because without the linearity, it would be impossible to deal with the structure that comes about and different kind of nonlinearities that come about and it would be impossible to, you know, get any kind of stabilization results. So, we of course, as always assume that we had and W had our estimates for the ideal weights. Right. And we of course define the tilde versions for V, W and Z. Right. And of course, we also have the sort of error in the activation function values. This is defined using this notation as sigma applied on V transpose X and sigma applied on V hat transpose X. So, this is again a notation. Right. So, what we want to do is to sort of expand this into expand this with some kind of linear terms. Right. And that's the whole idea. Right. We want to use the Taylor series. Right. So Taylor series gives us some linear terms. Right. This is standard when we linearize nonlinear systems also. Okay. Great. So, so what is the Taylor series expansion of say this V transpose X? We know that we can be written as V tilde plus V cap. We take the V cap as the center sort of and so we have sigma V cap transpose X. Yeah. And then you take the sort of, you know, you sort of take the derivative with respect to your V cap, you know, this V cap transpose X term. So, you take the partial or the derivative with respect to X. Right. And then you have this sigma prime notation. Right. And then you have multiply with the error term, which is this V tilde transpose X. Right. And then you have all the second and sorry, the higher order terms here. Okay. I'm sorry. I think right. Right. We have the higher order terms right here. All right. Great. So of course, this is defined. This is defined as a partial with respect to some sigma prime is defined as derivative with respect to some Z evaluated at Z hat and so on. And O Z square, of course, define higher order terms. And this is all well known. Right. So of course, there are some, I mean, the author's point to some other reference where a different Taylor series was used, but that's okay. Right. So now, of course, we, the authors want to simplify notation. So they call this term as simply sigma hat prime. Why sigma hat prime? Because the sigma prime evaluated with this hack term. So that's the rationale for using this kind of notation. And so what do we have? We have sigma tilde as sigma minus sigma cap. And so sigma, we use, you know, this kind of an expansion here. Right. So what do you have? So sigma cap also contains, sigma cap is essentially this thing. Right. So whenever I subtract, if I subtract these two, I just get this term. Right. That is sigma prime. We had transpose. So let me fix this. We had transpose X. Right. And this V tilde transposes. So you get these two terms right here. Okay. And then you have a second order. Okay. So this is of course simplified. We just call it sigma hat prime. And this is V tilde transposes. So this, this, this term is just to, I mean, I'll just mark it. So this term is just this whole thing. And it's just a short hand. It's nothing other than that. So now there is also the requirement to put some kind of bounds on the higher order terms. And so we, I mean, the authors do do that. So this kind of a bound is sort of used. And again, this is I believe transfers. And so this is like this kind of a bound. It is. So let me see. I'm trying to see if this is any different from what we have different bounds are put on the, okay, okay, okay. So this term is bounded with, again, this sigma tilde, this looks like sigma tilde. Yeah. So essentially this, I mean, honestly speaking, this bound is essentially derived from this equation. Because if I take this on one side, I simply have this subtracting this. And this is sigma tilde. It's essentially sigma tilde. And then you have this guy here. Yeah. So this is what you have. So this, if this guy is to be evaluated, then you take this to the other side. So you have sigma tilde, which is this minus this whole thing, this whole thing. Right. Right. So now in order to get some kind of reasonable bound for this term, you use this expression 24. Right. So you have a fact four, which is that for sigmoid RBF and tan hyperbolic, so all the three activation functions that we looked at, you have this to be bounded in this way. And these are, of course, obtained using some kind of bounds on these guys. Right. So this is somehow using that. So because you see that these bounds also contain the Frobenius normal B tilde. Right. So so these bonds are obtained using this expression, right? Okay. So that's the whole idea. Okay. All right. So of course, they also use the, not just the, yeah, they also use the fact that sigma and its derivative are bounded by constants for the RBF sigmoid and tan hyperbolic. So of course, I mean, the authors mentioned that for nets greater than three, this kind of bounding and these kind of ideas and are not too difficult to extend either. And then it can be done. All right. Excellent. Excellent. Great. So what did we look at in this session is that we had already seen the robot dynamics that we saw a little bit more of, you know, sort of how the control structure looks in today's session. We also saw how the nonlinear robot function looks. Right. And we understood why are going to zero sufficient for our purposes. And finally, the, once we understood that this F tilde that is the function approximation error is what we want to drive as close to zero as possible. We understood that this is where the neural network comes in. Right. So this neural network is used to approximate this function using this theorem that we looked at earlier. We know that, you know, we sort of know that this function approximation using neural networks will help us to get close to value of F, therefore F tilde can be made to have like an epsilon n bound. Right. And that's what we looked at at how this neural network approximation will work. How we use the Taylor series to sort of linearize the terms in the neural network and bound the nonlinear terms. And this is what we'll use in the subsequent analysis to define these parameter estimation and parameter learning. And these parameters, of course, the weights of the neural network and we will hopefully be able to show that you have nice stable performance the way you require. Right. So we'll continue with our discussion in the subsequent session. And I hope to see you again soon. Thank you.