 Hello everyone, welcome to get another session of our NPTEL on nonlinear and adaptive control. I am Shrikan Sukumar from Systems and Control IIT Bombay. So we are in this very last week of this course on nonlinear adaptive control and we are talking about some exciting things and I hope you found the course in general as a rather interesting course. So right now we are going into slightly more advanced and research topics and we have been discussing about learning. So we started the week by trying to create a chronology of how things evolve in adaptive control and parallely in stochastic and discrete systems in the form of self-tuning regulators and also another parallel in the form of learning classification kind of problems. So I do hope that you are relatively convinced that whatever algorithms that you've learned through the curriculum will help you drive autonomous systems such as what you see in the background. So in this paper that we were looking at, we were looking at a very specific learning setting which is now a very popular setting and that is the one on deep learning. So deep learning is essentially learning algorithms that make use of deep neural networks that is multi-layered neural networks and that's the sort of application that we are looking at here. One of the sort of differences from what you typically are used to in learning algorithms is that here we have everything to be online that is you're doing learning also in real time. So we of course established a parallel of deep learning or neural networks with parameter learning. So essentially that's what it is because you have weights and then you're trying to learn some weights of the different layers on the neural network. So we did establish that current and the difference here lies in that we are trying to do the learning online that is real time learning and control simultaneously. So we talked a little bit about where the literature is in the beginning and we also sort of tried to understand that standard learning rules may not be a great job and so this article talks about some modifications to the learning rules. Now we also then of course discussed a little bit of the background. Let's see, I want to see where I've marked these, it does not look like I marked the lecture. Just a second, I think the first lecture of course started from this paper here. So I think that's fine, that's marked in this paper. So let's see where we're going to find it, just give me a second, not here either, there's the STR. I just want to find where I marked lecture number 12.2, yes that's here in the discrete and deterministic systems. And I believe we started, I'm going to mark it here as lecture and let's see, now this is fine. So this is, yeah, this is all of lecture 12.2 I believe, yeah, yeah, I believe this is all of lecture 12.2, yes, yes, this is fine, all right, so we do have all the lectures marked, my apologies. So then we started looking a little bit at the background of what is the problem set up, pretty standard in most articles that you read. Here we talk about the type of norms on matrices and signals and vectors, then we do the most important definition which is that of how the neural network looks which is with one input and one output layer, and then one hidden layer in between. So this is how you have the three layer neural network structure, so you have the inputs going in and then they're scaled by some weights and offset, then you get a summation over them and that's the second layer where you have this activation function, right, and then you have sort of the connection with the third layer which is via this weight w i j and the theta w i offsets, yeah, so the aim is of course for the neural network to identify all these quantities, yeah, and if you do then you have a very clear relation between functional relationship between x and y, which is the input and the output, yeah, don't think of these inputs and outputs as a control, etc., this is a more general setting, all right. So we also saw that there are different possible choices of these activation functions, they can be signeoidal, hyperbolic, tangent or radial basis functions, right, and with some, you know, sort of flexibility in terms of how we define x and v and w and so on, we can account for both the weights and the offsets in this sort of structure, which is this non-linear regression parameter. Why do I say it's non-linear? It's because you have this, you know, w and also a v here, so this is what makes it non-linear because both w and v here are unknowns, right, so usually you would probably just have something like this in a regression parameter, okay, but here you have first two different unknowns and then you also have this activation function, which is also potentially non-linear. So then of course, we talked about what it means for, you know, something, some function to be in the functional range of a neural network and that was defined using this kind of an expression that if you have the neural network, it gives you something like this and then if you have a small epsilon offset from the true value of the function, then it's somehow in the functional range, right, and there are some nice results for specific cases of sigma, right, we were talking about these activation functions and for some specific cases of sigma, you do have results which say that the functional range of the neural network is dense in the set of continuous functions. We'll try to interpret this result of that. First of all, there is the term squashing function, so what is a squashing function? So basically it's defined here, a squashing function is a bounded, measurable, non-decreasing function from the real numbers on 201, all right, so basically this is this kind of function includes the step function, the RAM function, sigmoid function, etc, right, so the squashing functions, there are other relatively large class of functions, right, so for these kind of activation functions in your neural network, what this theorem claims is that the functional range of the neural network is dense in continuous functions. So what does it mean? This statement, yeah, for those who have not seen analysis courses, essentially means that almost in continuous function can be approximated via this neural network, okay, so this is a very specific neural network, by the way, it's exactly a three-layer neural network, so one input, one output and one hidden layer, right, so with this specific neural network you have a result, it says that you can approximate almost any continuous function quite well with this neural network, right, so this is what it means, so of course the dense is in terms of the supremum norm, so this is the whole idea of how you define dense, in fact, what the author state here is that the last-layer threshold values, that is the offsets, were not even required, yeah, so the last-layer thresholds were not needed for this result, you don't even need the last-layer thresholds for you to have a good approximation, right, so of course how to select these sigma and then choosing this into, in a particular, for a particular set S are of course topics of research, so then there is this section on stability and civility, which I'm going to skip for now, for now I'm going to skip it, yeah, if I do see the need I will get back to it, yeah, we have not discussed passivity, which is a very standard notion for nonlinear control and if you have a system which is passive, designing controllers becomes sufficiently easy, this is usually covered in a more detailed nonlinear control case, yeah, which we don't aspire to be, we are more of a nonlinear adaptive control course, all right, so we will skip this for now and we'll see if we need it later on, we will talk about it when passing, yeah, so then in this next section the authors are starting to look at, oh by the way, I was supposed to start to mark the lectures, so I'm going to mark it here, lecture 12.3, all right, so the third lecture of our last week, so in this section C the authors start to look at the robotic arm dynamics, yeah, why because this is the application that the authors are focusing on, I mean if you focus on another application, you can write the dynamics for that in a particular form, so this is the standard form for robot dynamics, it's called the Euler Lagrange form, so the Euler EL equations, very standard to call them EL equations, here this is a structure which holds for any n-link robotic manipulator, right, so it is generally enough, like if you have an n-link robotic manipulator you can use this and this is written, these equations are written in what is called joint space coordinates, yeah, so if you have any, I mean I usually make this picture, I will try to make this picture here for you folks, so if you have any sort of axis, say, right, and then you have one joint here, right, then you have another joint here and then you have another axis here, sorry another sort of link here, joint, so this is, so this is, say, joint one, okay, and this is say joint one, this is link one, joint two, link two, and then of course you can mark it, right, so now if you look at some coordinate system, say this is a zero zero, this point, then this coordinate in the world frame is usually has some coordinates in say two-dimensional frame x y, it's a planar manipulator, if it's not a planar manipulator then it will have all three coordinates, but let's keep things simple and say it's a planar manipulator, and so these are the coordinates in the world frame, but then there is another way to specify these coordinates, so this is, let's see, let's see, if you look at this angle, this is say q one, and you look at this angle, this is say q two, yeah, so there are two ways to look at robot dynamics or two ways to write the robot dynamics model, one is in the joint frame, these q ones and q twos, and another is in the world frame which is with this x and y, you see that both have equal number in this case, in this particular case the number of joint coordinates and number of world coordinates are the same, but that is not necessarily true, right, because I could have 20 joints in a two-dimensional frame, so I have a lot of redundancy, yeah, then of course there is many more joint coordinates than there are world coordinates, so, and of course there is a forward and inverse kinematics for those who have seen robotics, there is forward kinematics to move from the joint coordinates to the world coordinates, so that's again something standard, joint world in the forward kinematics, and the world the joint in inverse kinematics, yeah, so you have these kinematics which means that it's just a relationship between the joint coordinates and the world coordinates and vice versa, so that is what is your kinematics equation, so, so now what these authors are doing there in this particular problem is they're writing the everything in the joint coordinates, so that's why you have these q, yeah, these are joint coordinates or they're also in Euler Lagrange dynamical notions or generalized coordinates, why are they generalized coordinates, because they could represent angle or also linear motion, because you could have joints which are just lengthening or shortening, right, so you could have linear actuators also or you could have angular actuators also in a robot, right, so this q could represent both length and angle, right, so that's why q is also called generalized coordinates, yeah, but most important to remember is that it is in the joint frame, yeah, it is not in the world frame, so then you have this m which is your inertia matrix, so, so m is the inertia matrix, it is a function of your q which is the joint coordinates, then you have vm which is the Coriolis and centripetal matrix which is a function of both your q and q dot, then you have gq which is usually the gravity matrix and gq is the gravity vector or the gravity sorry not the matrix but the gravity vector and finally f which depends on q dot is the friction, right, then you have these terms tau and tau d, tau d is basically say any kind of external disturbance, right, so any kind of external disturbance is sort of put into the tau d, you know we've already seen how disturbance gets into adaptive systems, so or any non-linear system for that matter and we usually assume that these are bounded even though we may not know what the values are at each instant in time and then you have tau which is the actual torque that I can exert, right, so when you talk about this tau for example if you look at this manipulator how would you apply this tau, you would assume there is a you know there is usually a motor, yeah m1 and there is another motor m2, right, so you would have some motor here on this joint and another motor here on this joint and this tau is essentially obtained by rotating the motor, it could be whatever I mean you could have a servo drive or you could have a you know I mean DC motor whatever I mean you could have different kinds of motors but the point is that's how you get the torque tau, all right, great, so once you understand how this robot works and how the model looks like of course we are not giving any structure for this m notice, if you have a specific robotic model you will have to actually write the dynamics either your Newtonian formulation or with the Lagrangian formulation or Hamiltonian formulation but the point is you will have to write the dynamics and then once you put it in this structure and you will be able to put it in this structure that you will get the expressions for m, v, m, g, f, so on, yeah, excellent, excellent, so suppose how is the problem formulated, I mean any control problem is formulated with some kind of a desired reference trajectory and this is we are already used to this kind of a notion, all right, so this is how any control problem is formulated, so here you see that your desired trajectory is some Qd of t, yeah, and this helps us define an error which is the true value of the joint, sorry which is they have defined it in non-opposite way, the desired value minus the actual value of the joint variable, all right, so if you have only angles then the desired value of the angle minus the actual value of the angle, now it is also well known in robotics to use this kind of a great tracking error, right, in order to do the analysis for the system, this is actually for you folk we should essentially like the like a backstipping variable, right, like a backstipping variable, why because if you look at this dynamics, yeah, and if I did write it as you know, this can be written as a double integrator sort of dynamic side, so if I take Q1 as Q and Q2 as Q dot, right, then what would be my dynamics, my dynamics would be Q1 that is Q2 and Q2 dot or M Q2 dot is equal to minus Vm Q dot minus G Q minus F Q dot minus tau d 9 plus tau, okay, so this is what would be my state space equations and if you see if I think of some kind of a backstipping type of algorithm to control them because this is essentially a nonlinear double integrator, right, some kind of a nonlinear double integrator, so we want to use backstepping to design then I would choose Q2S some minus lambda Q1, right, Q2 desired would be some minus lambda Q1 and so R is like the backstepping error, right, so I mean of course I'm writing everything in the error if QD is zero, if QD was not there, then E is just Q and forget the sign, the sign is irrelevant, if QD was not there then E is just Q, right, so R becomes Q dot plus lambda Q, right, so that's what we said we will choose Q2 as minus lambda Q1 and the backstepping error would be, right, so let me Q2 desired would be something like minus lambda Q1, if I were just looking at the stabilization problem that is around to sign Q1 and Q2 to zero and the backstepping error would be what Q2 plus lambda Q1, yeah, and you see this is and lambda is some positive definite matrix, this is exactly what you have here, sorry, like a Q2 plus lambda Q1, yeah, if there was no desired, if there is desired it's the same, right, I mean if I write it in terms of the desired variable I will get something very similar, right, because if I wrote this as E1 dot, E1 and E2 it will be E1 dot equal to E2 and ME2 dot will be some equation on the right hand side, okay, I'll still get something very similar, it doesn't matter if it's tracking or stabilization, but this is like the backstepping, this is like a backstepping error, very easy, yeah, it's very standard to use this in robotics, why, it's obvious, right, because it's a just a nonlinear double integrator and if you want to get some kind of strictly app-on-off function or get some good app-on-off candidate, this is the what makes sense, yeah, so of course lambda is some positive definite symmetric matrix, right, I mean you can of course have it as a scalar, I usually use select it as some diagonal matrix, right, you can you just have a scalar or diagonal matrix, you can also have a full matrix just positive definite symmetric, okay, it's just a design parameter, okay, now if I take the derivative of R and I plug in from my equations here, right, it's not difficult to see that I will get this kind of an equation, it's not difficult to see that I'll get this kind of an equation, right, because it should be evident to you that M R dot is equal to M E double plus M lambda E dot, right, and M E double dot, so this is just M times E double dot, sorry, this is M E double dot is M Q d double dot, M Q d double dot minus M Q double dot, so minus M Q double dot is this guy, minus M Q double dot is essentially V N Q dot, V Q V N Q dot plus G Q plus F Q dot plus tau d minus tau plus M lambda E dot, all right, okay, so this is this big thing that you did, all right, so what you do is you can write this, you know, what you will do is you will write this whole thing, you want to write this whole thing as this guy, and you want to write this whole thing as this guy, all right, and how do you do that? You simply, you know, you add and subtract minus V M R, right, and yeah, you just add and subtract minus V M R, then you combine it with this guy, right, so what do you have? You will have, you know, this so this F is essentially to encapsulate all the remaining terms, so you have these two terms here, right, then you have from here and from here, you have these guys, then you have this term, this term here, then you have this F Q dot coming from here, okay, and of course we have retained the, you know, the tau d and tau have been retained as it is, right, so I just need to check the sign, I think there is some sign issue here or do I miss the sign thing, somehow the sign seems off, right, if I take M R dot, M E double dot minus M lambda, sorry, M E double dot plus M lambda E dot, which is fine, and so from M E double dot, I have M Q D double dot minus M Q double dot minus M Q double dot, I see, I see, so I think I messed up the sign, so this has to be, all of this is with a plus sign, all of this actually has a plus sign, so now it's okay, this is fine, all of this has a plus sign on it, so this, this is all fine, so then I have tau D minus tau, all right, so this F is what encapsulates this kind of a nonlinear robot function, so why do we call it a nonlinear robot function is because this is what sort of encapsulates the dynamics of the robot, of course there is also VM, but we are not going to worry about that term, this is what encapsulates what the dynamics looks like, right, so and this is this FX, where of course we have chosen X to include all these quantities, E E dot Q D Q D dot Q D double dot, basically everything that shows up here, you have Q D double dot dot Q D dot E and so on, so you have all the variables appearing here and so it's a function of all these variables X and so you have, that's how you sort of put it, yeah of course it's also a function of Q which is not written here, but then that is already inside E and so that's fine, okay, so great, so this is the sort of simple, relatively simpler structure that we'll work through, we'll try to drive R to 0, we look at what that means in the subsequent section, right, so we'll look at what that means in the subsequent section, so essentially what we have done today is we have sort of continuing our discussion on learning-based or adaptive control-based learning and we are using it for a planar robotic manipulator and so today we spend some time understanding the dynamics of the manipulator and in the subsequent session, we hope to look at the control design part of the manipulator, so I hope you're enjoying our sessions and I hope to see you again soon, peace.