 So in this series of mine just on deep learning for people who are interested in deep learning Perhaps more specifically for medical personnel health care personnel people involved in health care want to solve problems in Health care using deep learning just making these first these couple of videos just to Get behind some of the mathematics now I have a short video just on linear algebra, which is very important in deep learning and this one is just going to be on derivatives derivatives Here we go Now if it sounds like the world is coming to an end our right outside my office here They're building a new neuroscience center and of the last couple of months. That's already well. It's driving me insane There we go. So derivatives are very important. We're going to have this Concept in deep learning called back propagation and that really depends on derivatives I want you to relax though. You don't actually have to do these derivatives You're gonna write a line of code and the computer is going to do the derivatives for you You should have some basic concept and if you haven't done calculus in a long time This is it now I do have two very large playlists on YouTube one on linear algebra and one on Multivariable calculus and I'll link them below if you're really interested you can have a look at those 100 100 more than 100 videos more than a hundred videos really if you want to get Really deep into into the topic. I've got those playlists for you Let's do a few short minutes just on derivatives Now, let's imagine an equation. I have y equals x squared and that's it. Oh, let's make it plus four X squared plus four. So if we have this it's going to be something like this And that's going to be four and we have this nice X squared plus four remember behind the scenes There's another x here, but it's to the power zero anything to the power zero is just one and four times one It's just four and remember how? Derivatives work must have had seen derivatives at school if I have x to the power n This is just simple and if I take dy dx Or dy dx is one way to write it why prime you might have seen I bring that in forward and I take one away from the exponent So if this was why is x squared y prime is going to be bring that in which is now to forward That's x and I'd subtract one from it. That means one. That's 2x And the same this is exactly what we have having y prime is going to be two times x And if I bring the zero forward that's zero times four zero zero times anything is zero So if we have a constant the derivative of a constant is zero it is gone Okay, so if I have something like y equals 3x squared plus 2x plus five I have that dy dx is going to be equal bring that forward two times three six Subtract one from that. So there's just one left bring the one forward two times one. It's just two and There's a one day one minus one is zero So there's zero left at the top anything to the power zero is just one so two times one is just one So it's two x six x plus two is that derivative Now, let me just clean the board Now that's pretty simple if I have a single variable, but what if I have two variables We usually write that as z and let's have z equals six x squared plus four x y plus three y squared Now I have two and how am I going to take a derivative of this if I look This will be in three dimensions and we usually have this as the x y plane and that's orthogonal to each other So you can think that's in the corner of the room and here we have z So we're going to have that this thing. I can't draw that but imagine it is just going to be this Three-dimensional thing in space is not going to look like that believe me But just imagine now what we need to do in the algebra is we're going to the system has to learn certain values And it has to learn those values and the way that it learns it is it writes a Multivariable equation like this now some neural networks. We're going to write and they're going to have more than a million Parameters more than a million of these x y z ABC there's any 26 but imagine we carry on and there's over a million So it's going to be this equation of over a million and it's also going to eventually you could if we had access to so many Dimensions imagine a million dimensional space, but it will be some convoluted shape in this multi-dimensional space And what we really want is the minimum We're in this now. This is very easy at two dimensions Where is this whole multi-dimensional thing at its minimum? And then when it's at that minimum, that's the point we after all the values of x y z all those millions of parameters if I had a value for each of them and So that that whole equation which is some graph in multi-dimensional space is at its lowest point at its minimum That is the value that a deep learning network is is after if it gets that it has the best values to answer the question We're trying to answer through deep learning So how do we get a derivative? How do we get to the lowest point? Well, if we just have One dimensional space. Let's have that we can start at any little point The derivative remember is the tangent point the point of a straight line that just touches there in three dimensional space It will be a flat plane imagine a piece of paper that just that is flat And it just touches wherever the space just in one little point, but that'll be a plane Three and four dimensional space. It'll be a hyper plane and multi-million dimensional space It'll be a very convoluted hyper plane, but it touches our graph in one point and I can use that slope To get to a point Yeah, I can There's something I can do to get to this point here and at this point the slope of That tangent line is going to be zero and the plane That lies here that touches just there is going to have this slope of zero So in one dimension as I said here in this dimensional space It's easy I have a slope and we use the slope to update to get to a new point Which will have its slope which will have a new point will has its slope And we can to change the slope change the slope change the slope until we get to a slope Very close to zero and then we know we're okay. We know that we are okay now This is easy to see but imagine a multi-dimensional space which we can't Find them in our brains You don't know where it is it's like you are in the space and you start walking blindly Now imagine blindfolding yourself and you in a place where there's a bit of a valley And you have to get to the bottom of the valley You know you can wander around until you go lower lower lower until you get to the bottom of the value of the valley And that's exactly what we're going to do. We're going to be some way and then we're going to start taking steps now Remember when we looked just well in my video if you're watching that just on the simple linear algebra for deep learning You know if I want to get to this point and I see this point as a vector What I could do is walk that way That way but I can move this one here. So that's there so to get to this point I might as well walk this way and that way so that will be so many units of x I'll walk in the x direction and so many units in the y direction if I make that walk I end up in that same place as if I went straight there and that's what we're going to do here We're going to walk in the x direction Which points downwards for us and then we're going to turn 90 degrees and walk in the y direction Along the slope that goes down and if we do those two things combined We've gone further down and the more dimensions of this you'll just have to do this in more one dimension But what we do is when I walk along the x-axis, I keep y constant I don't change on the y-axis and when I change the y-axis I state one plus in one spot on the x-axis. I see those two as constants And that's exactly what we do when we do derivatives And we write partial derivatives like this and I'm just going to take the derivative with respect to x I'm going to see y as a constant. I'm not changing on my y-axis I'm just staying straight on the x-axis. That means all the y's here are actually constants So if I do this derivative that x I can do that's going to be 12x by bringing the 2 forward Now that is a constant There's a one there I bring the one forward one times four is four The x disappears because it becomes a zero and the y is still there because it is a constant That whole thing is a constant and the derivative of a constant is zero It's gone because remember there's an x zero here if I bring the zero forward that's nothing And that is my derivative of z with respect just to x And if I do the partial derivative of z with respect to y I see x as a constant That means here's a y zero here. So that's going to disappear There's a one there. So that's going to become four x And that's going to become six y And that's going to become six y And that is those are partial derivatives and all you do with partial derivatives You keep all the others constant. So if there was also W's and v's and a's and b's and whatever in there you keep them all constant You only look at the derivative with respect to this you keep or you keep the other one constant And eventually you'll get this multidimensional pointing things And you combine all of them Do the one and the other one as you walk there and there you walk along this way then along that way And in multidimensional space if you have basis vectors that are all orthogonal Everything is going to lead you in a different direction But if you combine all of those things you at a lower point And now we've got a new point On this thing we can take a new slope Do all the same thing again and eventually we'll be lower And so we will go until we get lower and lower and lower if we just repeat We repeatedly do this And that is why we have these things called derivatives. So together with linear algebra derivatives Are very important now if this was all agreed to don't worry about it If you're interested in it watch my over 100 videos just on Multivariable calculus, it's quite interesting multivariable calculus But really you don't need that for we where we are going As if you just have some mild slight concept of what is happening here That's all that's required. We don't have to take any derivatives when we design We just have to understand that that is what the algorithm is doing It is trying to find a path That lowers the slope so that we can get to this bottom point And that in any dimensional space is what we are after because that is going to be The best values that are learned by the deep learning network to give us the most accurate solutions