 Okay, so last time we were talking about implicit differentiation, things like that. I want to move on from that. We're a little behind on the schedule. I want to try and catch up a little bit. I guess before I start that I should remind you we need a midterm on November 11th. Don't believe me there. 13, thank you. This is Wednesday. I'll be here. I won't be here. Eventually I guess I can ask would people prefer, so on November 11th, I will have another A-named person cover the class for me. Then basically just review stuff and go over problems, yeah. Yes, his name is Arden. His last name is Luke. So on the 11th, Arden will cover for me. But if people would prefer actually not to have a review or have a midterm on the 11th, nobody prefers that, right? No. Okay, if we won't do that, I won't even mention it anymore. Get I ever said it. Okay, and this will cover chapters 5 and 6 and maybe a little of 7. So 5 and 6, we should finish chapter 6 either today or Monday. I'll start chapter 7, which is non-integration. So properly there might be one question on integration. Depends on how things go. Okay, one of the things that one uses, six variable calculus for a lot is if you have some function, lines with fx, and you want to find the minimum, I want to maximize. This is one thing that is a major use of single variable calculus. And we have a similar situation in multivariables. This really only makes any sense to talk about the largest or the smallest value in the single variable. And this comes down to, we look at, set it to zero, find the critical points, determine which critical point is the biggest, if it's on a small interval, if it's on a compact region, we might have to check the ends as well. The same thing applies in more than one variable. Of course, in this case, we really only have the situation where f takes several variables to one variable, where we might look for the smallest or the largest value here. If we have more than one variable, we can't talk about largest, because, I mean, if I'm going from R3 to R2, I don't know which way is largest. I don't know where the largest is in the x direction of the y. In the x direction of the y direction of some combination. So it really only makes sense to talk about maximizing something where we have biggest and smallest built in. So that's, right, so I'm going to take some space into some, take some block there into some other block there. And right here, when we're no minimum, we're going to set ourselves a compact region, fading even if it's not the same. Okay, so some function absolute, replace it for minimum, at some point x0, f, well, what does this mean? It means that if function at f of x0, which is a number, so it's bigger or maybe equal to f of fusing, I want to restrict f to some smaller set. And then we also say that we have a local, a local or a relative maximum. Well, here, it means that for everything near x0, it's the biggest problem, but maybe far away there's something bigger. It needs to be either some kind of, we don't say the word at maximum. And if we don't want to distinguish between max and min, so, you know, if you're like a member of the KKK you have very extreme values, it means extreme. Yeah? If you have a function of, we're about to line up, we're going to find, you know, do you consider all the points of the best views? You can. So, yeah, you could. So, right, I mean, if I'm, you know, in a one variable case, something like that, is this whole thing, is everything in this situation is the maximum. So, all of these are maximum. Okay, so we have that situation. And here it's kind of important to realize we want this equal because you can sort of blow this up to more dimensions where you might have, like, a tent. And these are all at the same height. And, again, this can happen quite easily. Maybe even make a ground, so it can be smooth, so it can take derivatives. But, again, I want to think of all of those, that entire line is a line of maximum, which, you know, these kind of functions are not very nice. Whereas these functions are quite complicated if you just take, I don't know, z equals x square watt that I have. That's right. So that has, you know, an entire line of minimum. And that's a perfectly nice function. Okay. So it's the same as the theorem in the single variable case says that, I'm still going to call this, so I have some function f on some set, which includes its end points, and it doesn't go off to infinity. It achieves its maximum and minimum, so x naught and y naught. Maybe many. On a compact set, so in the single variable case, we're talking about a closed interval. A continuous function has a maximum minimum. Right? In one variable, we're just talking about a closed interval from a to b, and this is a graph of f, and its minimum value happens to occur at the end, up here is its maximum, and here are some other useful points. The same kind of thing happens for functions of more than one variable. But in functions of more than one variable, we have sort of more complicated boundary behaviors. So rather than, so let's, let's take an easy example. So minus four x, x minus x squared, minus one, minus, I don't know if that's square, right? Yes? Minus y, times y squared minus one, if you think of that function. What did the maxima and the minima look like? So I drew this on my computer earlier, so we have here the region minus two to two here. The absolute maxima seem to be here and here, and the absolute minima seem to be here. A little bit hard to see, otherwise that's okay. Exactly what it looks like in here. So let's zoom in a little bit. So this is just the graph of that function. If I zoom in a little bit on a slightly smaller region, you can see that, you know, in that little flat spot on the top, there's a hole in the middle where it bumps down. It looks sort of like a chair. Not a very comfortable chair. Maybe it looks like a pipe. I don't know. There's sort of a hole in the middle and a place where you can set my hands on. A little splish splish. Yeah. It's kind of a little thick here. It's thick. So I guess I didn't say the theorem. So here, if I'm trying to match, which is x between plus and minus two and y between plus and minus two, I might be distracted by this minima here, which is very hard to see, that minima in the center and these two little bumps here and all of the maxima and the minima on this guy occur on the boundary, on some smaller region like this. Well, that still happens, but if I go in an even smaller region, then maybe my maxima, you know, if I cut it off here, then my maxima could be there and my minima could be in there. So depending over what region, you're looking at the maxima and the minima could be different things. You do exactly the same process that you do in single variable calculus when you're looking for maxima and minima. You identify the critical points and then you also check the boundaries, check the edges, the endpoints. It's just that in several variables, the endpoints are a much more complicated thing. Right? Here, the endpoint is a bunch of lines, so let me take one more example here. So I'll come back to this one in a bit. Let's take another example, say g of x, y is a nice simple function. Let me draw a picture again. It's a nice simple function where I put the 2. I put the 2 on the y. x squared plus 2y squared. I'm keeping these functions relatively simple because I'm going to have to take derivatives and I don't want to look hard. I want to consider what I want to maximize g squared plus y squared less than or equal to 1. So the graph of g here, second, g is x squared plus y squared so it looks like a paraboloid except this 2 stretches it out. So I get a bowl-like shape where the bowl is longer in the x-direction, but it's okay. It's longer in one direction than the other and I want to maximize it over the part in the unit circle which is going to be slightly more complicated. So rather than me drawing it wrong, let me just put this same picture up there. So here, of course, my bowl shape because the computer likes to draw my pictures over a square. So it goes up more here if I go out to the square. Here is the unit circle. This color should be terrible. Should I turn the light off? Is it better if I turn the light off? So that's much better, right? So here's the circle. If I lift the circle up to the surface I get this higher here and lowest here and lowest here. So the problem I want to solve is to find, so here the absolute minimum is right here at zero. Now you can do a check and then the maximum seem to be here and here. The boundary of the region that I care about is a circle. It's still a little more complicated. Okay? Which one is right? First on the inside part, so suppose I have on some region U in Rn, so I have state in Rn suppose I have some extreme value. So then, that's because f is a function from many variables to one so we have a gradient. So the gradient of f would be zero. And here it's a zero vector because the gradient is a vector. We have the zero vector there. Yeah? So gradient of f would be zero. Okay, so intuitively we could just say since the gradient of max increase if you're at a max you can't go up anymore so there's no where for it to point or if you're at a minimum everywhere points up and so there's no one where that can point up so in both cases it can't point up because either it wants to point lots of ways or there's no way for it to point. Okay? Visually it has to be a horizontal plane. Yeah? Yeah. I mean you have a slope so another way you can think of it is that there's a tangent plane here of course this plane could be a hyper plane but you have a tangent plane there and all of the slopes have to be zero. This symbol and you can subset. So u sits inside of r and it gets all of r u. So it's the analog of this but this is a signal point and this is all the set. Sorry for just using set notation randomly too bad I get used to it. I'm sorry? It's like a C but it's not a C. It's like a C. Yeah it's like a U. It's like a sans-serif U but yeah. It's actually I mean it comes from a C for container. And then the underline needs maybe it could be the whole thing. Some people write this and sometimes you might see that which means it's contained but it's not everything. And you probably will never see this one which means it's compact which means they won't use that. I don't need that. Yes. Can I follow up by saying it's an interior point of view? Yes. Extreme point for you for pointing that out. I need that absolutely interior to you. I can show you where it is in my notes. It's right here. I'm on the wrong page. But anyway it's right here. So we have some interior point and it's important that it be interior or you could be you could be an open set where every point is interior and it's more okay. Anyway. So we have an interior point then we have to then the maximum has to occur. So we have the gradient zero. Another way, let me not if you want but another way you could prove this a little more formally is just, as Nick said all of the directions have to be slope zero. In other words if I write f let me just leave that off. So the functions fix one value and the functions from R to R I'm going to fix this is f restricted then these have to have a rational. That means that the partial has to be zero there. Then all the partials are zero because all the partials are zero that means the gradient is zero. You can prove it a little formally, more formally but hopefully it's really x squared minus y squared zero and that the gradient would be zero notice that it says if it's an extreme point then the gradient is zero. The gradient is zero doesn't mean it's an extreme point exactly in the same situation the exact same thing happens in one variable. Right? In one variable here is a critical point here is a critical point here is a critical point all of these have zero slope zero slope doesn't mean that you're in extreme value but extreme value means you have zero slope. Yeah? So isn't it not really true I said that gradient points to the the direction of the highest increase because at that point in the middle would it be zero? If there is a gradient if there's a non-zero gradient it points in the direction of maximal increase zero then the gradient is zero right? So again right here we have to look at a higher derivative to see that it goes up so in this case the first derivative is zero the second derivative is also zero Anyway, in this case eventually there will be a non-zero derivative which tells us which way is up but we have to work a little harder work a little harder right? So exactly the same situation can happen here as you mentioned we could have we could have a saddle which goes down in one direction and up in the other direction and at this point which in fact I guess the screen is still up there right here for example right at this point it goes up in this direction and down in this direction max or local min what the gradient is just like in one variable where we can have critical points where or not local masses are mixed okay so that gives us a way and of course I just erased the example I was going to do that's nice it gives us a way to identify relative extrema so minus so that's my function which if I multiply this out x and here I get I'm just going to leave before there x cubed minus minus three y squared and so this will be zero this is I can take more out so let's put the minus back in three y squared one so I'll get critical point that was fine doesn't matter okay so I get x is plus or minus one over root two right because the square has to be a half in order to kill it with the one so I get either x is zero or x is minus root two one over root two or x is plus one over root two and so this means that I have so my critical points so this will be zero root three zero minus one over root three yeah get all the combinations one over root two one over root three one over root two minus one over root three one over root two negative minus one over root two one over root two okay somewhere I screwed up the size it's alright minus one over root three so I get these six critical points which are exactly the six points then if you look carefully you can see here I get one two three four five six right I get the three saddles the two local maxes and the one local mansion yeah one over root two plus one over root three so I get those six possibilities maybe it's easier to see from the top no because you can't see anything with the lights on okay anyway I get a saddle here a saddle here a minimum there a local max here a local max there and that's a saddle saddle means it goes up in one way and down in another it means that a saddle is what you put on a horse when you want to ride it the horse goes up in front of you and up a little bit in back of you your legs go down to the side so it's just like a saddle you can sit right here your legs going down it means that in some directions if I slice it in certain directions in some directions that will increase and in other directions that will decrease so it's not a local max because for a local max it needs to decrease in all directions and it's not a local min because for a min it needs to increase in all directions and here this in between kind of point is a maximum in some ways and a minimum in other ways I think in economics or something they call it a mini max I think right it's a weird word but anyway we also have a point well it depends on which part of the pants here it is you're out of much alright see that I feel the best so I'm going to come back to this example so I'm going to come back to this example yep, nope so let me come back to this example do we need this picture up here? no because you can't see it anyway it was on the screen it's on my screen the thing that I want to get the picture it's pretty obvious that this is the easy to see let's use the same letter it's a good idea so it's great to see that we get a critical point might have to work a little more to see that it's actually a minimum but it's a minimum from the picture and then we want to see these other points here so what we can do ok, check the edges what does that mean? how would we do that? I mean I can put it on my computer which is a picture but that sounds cool so parameterize the curves I point it that way I need to calibrate them yeah so let's look at this x squared plus y squared equals one minimize g of x y subject to that so this curve x squared plus y squared equals one I could I suppose solve for y blah blah blah but I could also just think of this as some function of parameterize let's sorry? yeah I want to find the extremum just along that curve so now I'm just looking at this circle I'm not drawing it right it looks like a 7 finally I'm just looking at this circle so if we write this curve x squared plus y squared equals one as cosine t sine t that'll parameterize the circle x squared plus y squared let's call it gamma cosine t sine t so this is just the circle in the x y plane and then I plug this in to the function g here it is to get a function from r back to r so g of gamma of t is just going to be well this is g of cosine t sine t which is just sine squared t plus two sine squared t yeah isn't it just one plus sine squared? no I mean my function g is this maybe this is my function no the one minus cosine squared t equals sine squared yeah I could do that too so whatever so maybe it'll simplify I don't care I don't care this is good enough for me and so now I can just compute g prime of t and see where it's maximum right so that will be at the maximum so you can write it as one little friend or so or we could just use the chain rule right we could use the chain rule here to see that so let's do it by the chain rule by the chain rule e dt of g composed of gamma is going to be the gradient of g which is 2x 4y but it's at t so it's really 2 cosine t or sine it's the same I'm going to get exactly the same answer but I'm doing it the other way and I'm going to dot that with the derivative vector of my parameterization so this will be the the derivative so that will give me negative sine t plus cosine t which gives me the same answer 2 cosine t sine t plus 4 cosine t sine t so this is zero with cosine is zero or the sine is zero so in other words I have critical points as we knew already from the picture the cosine or the sine is zero so I have t equals zero t equals pi over 2 t equals pi or t equals 3 pi over 2 which are these four four points right here my function at all of the points see where it's biggest and smallest that gives me my maximum exactly the same gain that you go through in single variable calculus where you find the critical points on the inside of the interval evaluate the function at the end points and on all the critical points your max is at the biggest one your min is at the small why didn't you choose that one anyone, any parameterization of the circle is fine but I wanted to parameterize the whole circle think about this edge curve somehow I have to it's lowest and its highest in the other case where I was restricting to a box so suppose I wanted to do this question instead of over this circle maybe I wanted the same question let's change the question where I wanted to get here let's change the question again let's find the answer to this just by looking but in harder questions let's find the maximum or the minimum where instead of just looking over the circle let's say over the box x between minus one and one y between minus one and one then I have four sub functions to look at so here over this then I find the critical point the origin g of the origin is the origin a problem there and then I need to find the extreme of the four edges edge function g of one y which is between minus one and one g minus one y which is the same and g of x and plus or minus one just do them together which is x squared sorry this is x squared plus two these guys are the same and I'm plugging in one so that's one but all of these are parabolas and the maximum at the corners so that means I just care about the corners because these are all parabolas if I had a more complicated function then I'd have to look at more complicated stuff right so I have three potential extremals for these here here and here so I have to check those points does anybody want me to actually check all of those points you just plug in which is biggest that's your max which is smallest that's your min so here the problem is a little more complicated here for each of these boundary curves I find well I get over the box I get the corners and I get the middles and I get that middle and then I have to check it six nine points I have to see where the function is the biggest and all of those nine points to find the maximum and where it's the smallest to find the minimum and so this is my minimum and these guys are all the same value again you just look at the picture that's fine but you know if we have a function of 12 variables and we're playing this game it might be more complicated yeah so if you run from r3 to r1 to xyz to check the edges you would just fix all the variables except for one are we over a square r3 so do you want if you had g of xyz would you to check the edges you would just fix all the variables of one no you'd have to look at the basics right so let me not do it as opposed to my function h of xyz and is let's just do the simple case that guy and let's say x, y and z are all between minus 1 and 1 so I get a minimum at the origin that's easy to check and then what do I have to say well here x, y and z are all less than one that means at the value of the function on the unit q so that means I'm going to have to look at six different slices here this face I want to minimize the function so let's see let's put something so this is x equals 1 and y and z are between plus and minus 1 so that means I'm going to look at the function 1 plus y squared plus z squared with y and z between minus 1 and 1 so now I've transformed this three-dimensional problem into a two-dimensional problem if this function were more complicated then I would have to work hard to determine maybe whether on this square it was biggest and here I mean here it's easy it's smallest it's easy the maximum is at the corners, we're done but if it was more complicated I would now have this problem of maximizing or minimizing the function over this face and over this face and over that face and over the top and over the box so it could get annoying but you have to consider all of those I mean in applied mathematics often you have just linear functions that you want to maximize over complicated regions and that's hard enough that they're linear functions so there are all these theorems that they occur at the edges and at the verbal vertices and along the boundaries and so on but it's hard enough just to maximize or minimize a collection of linear functions over some convex region so this is can be a real common event okay so I guess I wrote that theorem down let me not do the proof because I sort of said the proof right so the theorem that I'm skipping now is when I wrote down it says function which has a local extreme value at a point inside then the grade is zero so I did that already and I did that example just now there's another way to do this same kind of problem which is a little easier a lot easier a little easier which is called Lagrange multipliers boundary then things get easier they get it's more straightable so that's one choice another choice is this method of Lagrange multipliers an example or two so I have some and some surface some box or some blobby thing right so I'm thinking of I have like this I have a Q the Q is S and I want to maximize so then and suppose that I can write by some function some function g of x, y, z I have some function g and S is the level set edge on boundary here so here notice g goes from r into rm so that means that g, sorry I skipped ahead g looks like g1 g2 up to gm and as I have m's functions which may be in the jargon or called constraints like the extrema of f on the edge that I care about are going to occur at the critical points of f, lambda 1 g1 lambda 2 g2, gm not to be subject to obviously that they satisfy g does that make any sense? no, okay so that means I don't know I don't know I saw that the gradient of f plus some number of g1 plus lambda 2 gradient of g2 plus however many I need lambda n m gradient of gm that's the zero vector also this point let's do this in an example let me do it here so what does this really say? let's take that example that we already did before in this example I want to maximize and minimize subject to the condition in one condition I live on this circle this is the thing I want to maximize to use Lagrange multipliers I would just look for my critical point on the inside so I find this guy and now I want to worry I want to find these guys here here is I'm going to look at let's call this too many f's and g's I'm going to call them f and f times the gradient of g figure out where that's zero notice I have an extra variable now I also need also and two I also need that point so here I have this is two this is something in three variables but I'll find two of them because it's a gradient there's really two equations and three unknowns and there's the third equation so that'll give me my solutions this is sort of not a great example to illustrate it but let me do it anyway so in this case my Lagrange function that I want is the gradient of f and lambda this is two x two lambda first component my second component is four one I'm only taking the gradient so I don't take the gradient because it's really two y times two plus lambda equals zero plus y squared this tells me that either so for that that's what I need and here lambda is negative two or y is zero obviously I can't have both lambda equal to negative one and negative two at the same time so if x is zero and y is zero plus zero squared equals one that doesn't happen much I can rule that one x is zero or y is zero but not both I don't really care what lambda is it was just an extra thing laying around x is zero or y is zero not both here's my circle I'm either here, here, here, or here so I just check those four points and I'm done do you think it's an all or is it just an extra thing that you throw in I mean sometimes you solve for lambda and I'll do an example finding lambda actually this is one we already knew the answer the advantage of your first exposure I think where you know the answer you know it's right that's a good thing here it's all the way over there I'll do a more complicated example why would this so using this method does that give you critical points or actual issues it gives you the critical points of the restrictive function right so it's similar to this kind of method because they didn't have to parameterize the boundary so this saved me having to find parameterize the boundary because sometimes it's hard to parameterize the boundary so here I just have some conditions and I don't really care for some nice representation of that condition especially if I have like two or three conditions then it's a real pain in the neck to try and express that to find the intersection of all of those conditions if you have two or three then it's really multiple yeah I'll do an example of that in just a second so I want to um yeah why does this work yeah okay you can tell you're thinking about the slope at the minimum has to be equal to the contour line going around the things and then I think yeah it's because so let me draw the one variable case I'm not even going to try and draw the entire variable case so here's my constraint my level curve right so F looks something like this these are the level curves of it and G is some other level curve so in this case, bring it this way if I add some tangent vector to G which comes through this point well they're going to match up at a maximum the gradient vector of G will be here and they're pointing in opposite directions so if I take some unit vector here U the dot product here the dot product of these two things are zero so because the gradients are normal the gradient is normal to the level curves of G so I have to be both on the level curve of G and the biggest piece of X they have to be perpendicular sorry for this purchase and in higher dimensions these things are I have here are the values of G and here are some other values of G but again we have let me skip that there's probably a special picture ok so let me do another example here let's do one with two conditions because I have to make sure I'm holding it ok good so let's something simple say I want to maximize X squared no X plus Y plus Z so I want to maximize the sum just the sum of X, Y and Z and I want this to be so subject to so this is along the intersection let's say something both conditions X squared plus Y squared same thing zero and also Z so here I want to maximize the sum of the three coordinates where X and Y live on a circle and Z is exactly two so then I can just set this up so by Lagrange multipliers so I guess I could parameterize this still but let's not by Lagrange multipliers then what I want to look at I want to take the gradient lambda times this guy and now I need another z minus two also let's call this G2 solutions to this, this is five variables but then also subject to those two constraints so when I take the when I take the gradient here I'm going to get one doing with respect to X plus two lambda X and that's gone so that has to be zero and then with respect to Y I get one plus two two lambda Y is zero and then with respect to Z I get mu is zero oh oh one plus mu so I get one from here plus mu so this tells me that over two lambda Y is negative one over two lambda and mu is negative one I don't know anything about that but it's two just from my constraint wait a minute I don't need this so if I if I I lost a minus sign why did I have a minus sign and now I go two lambda X minus Y so if I subtract this minus this this tells me that two lambda X minus Y is zero right and mu doesn't matter so this tells me that either lambda is zero minus one which falls out of this and Z is two so we're kind of done and so now I lost my place so now I just can plug this if X equals minus Y lost my branch yeah if X equals Y then I have two X where so X is plus or minus root two and X equals Y can be sort of involved in the game so that we can actually use the constraint so we didn't have to lambda mu and we couldn't write so I guess it's so that this won't necessarily be right on the edge so