 Now, when we are looking for solutions, now that we know at least one solution exists, the way we look for solutions is the following. We can talk of many different types of solutions. The first kind of solution is what is called a local minimum. A local minimum, so X star in S is said to be a local minimum. There exists a radius R greater than 0 such that f of X star is less than equal to f of X for all X in a ball of radius R around X star and in a ball of radius R around X star. So, the way, what is going on here is the following. Let us look at, let us plot, take an example. So, suppose my domain is like this, it is this interval here and maybe also and this interval here. So, this is my domain, the feasible region is this, this blue line here and suppose my function looks something like this. Now, what is the local minimum here? So, S is this and this and what I have plotted here is my function F. So, you can see in this picture, actually there are two local minimum. One is this point here and the other is this point here. This point is not a local minimum. Why is that local minimum? It is infeasible. So, at the very beginning, local minimum the way I have defined it is an X star in S. So, it has to at least be feasible. So, this point is an infeasible point. So, it cannot qualify to be a local minimum. This point is feasible and moreover, if I see that look, zoom in and see into a small region around it, the way the function looks is that it has its least value at that point. Likewise, this point is also a local minimum around this, if I search, the function has its least value over there. That does not mean that the function cannot have a lower value elsewhere. That is a completely separate matter. So, let me give you another example. Just a similar sort of region. Let us take S as this and this and suppose I consider a function like this. This is my function. This is my set S. So, now can someone tell me where, what is my local minimum? So, this rightmost point that you have is, this is the local minimum. Let me draw the function more completely. So, function keeps decreasing for that. So, why is this rightmost point a local minimum? So, if you look at this, if you look at the function on the entire region, entire ambient space, which is on R, then of course the function decreases for that. But when we are talking of local minimum, we are comparing only with points that are in S, not points that are outside S. There may be points outside S that may be better than points inside S. That is not of relevance to us. So, if you look at this point, this point is such that, now if I take a small neighbourhood around this, a small ball around this and intersect that ball with S, what I am left with is only this part, only this part. And in this part, it is the least, least part. This point gives you, the right end point gives you the least possible value. Any other local minimum? What is the other local minimum? So, if I take the right end point of the left segment, again around this I can draw a ball like this. The ball intersects only this part and in this part, this is the least. It is not at all relevant to the optimization problem that the function has further decreased here and so on. All of that is not relevant because after all we are comparing only with points in S. Is this clear? The notion, the notion of solution that I mentioned earlier in the first class is what can be better class, this is more, a better name for it is what is called a global minimum. Global minimum, so it is an X star in S is said to be a global minimum, f of X star is less than equal to f of X for all X in S. It is true. So, what the question here is that, so you are referring I think to the right end point of the first segment of the left segment. Yes, so if I take a radius R that is very large, then what is going to happen is say for example, if I take a radius large enough to include all of this suppose, a very large segment, a large radius, then that also you will include also this point. And so consequently, this will not be the minimum in such a large ball and this is usually the case. If you make this region larger, you will start discovering points, you will have a, you will be comparing with a larger set of points. So, consequently or it is possible that you will find a even better point than the one you started. But a local minimum, the definition is there should be some neighborhood, however small, some neighborhood of positive radius in which it is the best. So, for me to show that this is a local minimum, all I have to do is demonstrate for you one neighborhood. It did not work for all neighborhood. So, a global minimum is defined in this way. So, now come back to our two examples, can you point out what the global, can you point out in these two cases what is the global minimum? So, let us start with this one, where is the global minimum here? So, it is unfortunately my drawing is not that good. So, let us assume this one is lower than this one. What is the global minimum? Yeah, it is this point here. So, this one here, this is the global minimum of the function. Is the global minimum attained in the set? Yes, global minimum is attained in the set. In fact, this is you can see the interval, these are two closed intervals, their union that is a compact that is a closed and bounded set and the way I have drawn the function, the function is continuous. So, the global minimum is attained in this. Now, global does not mean what is the minimum of the function without the feasible region means the minimum of the function which over both feasible and infeasible point that is not what global means. For that there is a different name that is what is called an unconstrained minimum. So, I will just unconstrained minimum is simply some x star in Rn that is this is a completely separate concept. So, the unconstrained minimum need not even be a feasible point. The unconstrained minimum may not even be finite because you are now looking for the function, how the function for the function even outside the feasible point. It may even be that the function is not defined on the full region. So, all sorts of things can happen once you search outside the constraint but assuming it is finite and finite this is what it is referred to as. This is the definition. So, let us come back to the thing that I was mentioning. So, you have a global I was talking of global minimum. So, what is the global minimum in the first case you pointed out it is this particular thing here. What is the global minimum in the second the right end point the right end point of the second segment here that is your global that is your global minimum. What is the unconstrained minimum in each of these cases in the first case in the this particular point here this point here this is the unconstrained minimum and what is the unconstrained minimum here it is not attained the the the the unconstrained infimum the value of the function is 0 if it keeps going this way it will be 0 it is but it may not be it is not necessarily attained. So, if you see Weister's theorem what it is guaranteeing you is the existence of a global minimum or it is telling you when a global minimum is attained. Now, even when Weister's theorem is not satisfied you may have that the local a local minimum can be attained and that that is very much possible. So, even though you may not a global minimum is not attained you may be able to find local minimum that are now let me cover strict local minimum. So, X star in S is said to be a strict local minimum if so X star in S is said to be a strict local minimum if there exists an r greater than 0 such that f of X star is strictly less than f of X for all X in this ball of radius r around X star and lying in S except of for X star itself. So, if you compare with every point X that lies both in the ball and in S except for the point case where X is equal to X star itself then you will find that f of X star has a value strictly less than f of X in that case you say that X star is a strict local minimum. So, again let us go through this example in both of these in example in both these cases that we have drawn let us look at the two local minima that we had are this let us check are they strict. So, if this one is it strict this local minimum this local minimum is strict it is the way I have drawn it it is strict you can find it is you can find the ball small enough. This one also there is a curve here if you look closely enough this one is also strict what about this one this particular one this is also strict likewise this is also strict. So, these are all strict local minimum. So, what does a non strict local minimum look like? So, what must happen is that something like this. So, all these points they have the same function value take a you take this point you take this point you take this point around them you can you choose a neighborhood there will always be another point there is always another point that has the same that has the same value. This is one way in which this is one way in which you can you have a minimum that is local but not strict. So, there is a very closely related notion I will write that now which is called an isolated minimum isolated or precisely local minimum isolated local minimum x star in s x star in s is said to be an isolated local minimum if there exists an r greater than 0 such that x star is the only local minimum in B of x star, r intersection. So, an isolated local minimum is one where in which you can find a small radius around that point such that in that radius there is no other local minimum. So, now can someone tell me how is this related to being a strict minimum a strict local minimum. So, clearly a local mean an isolated local minimum to begin with has to be a local minimum does it have to be strict or is strict look the same as I implies isolated or isolated implies strict. So, the answer is isolated actually implies strict and the reason to see this is the following. So, when is the minimum isolated it is isolated when in its neighborhood there is no other local minimum. Now, suppose if it was not a strict local minimum if you have a local isolated local minimum that is not a strict local minimum then in that case what would happen? So, let us do this carefully. So, x star so suppose the answer is I am good to argue that isolated every isolated local minimum is a strict local minimum this is what we are arguing. So, why is this the case? So, let x star be an isolated local minimum and suppose x star is not a strict local minimum suppose x star is not if x star is not strict then what does that mean go back to the definition if it is not strict that means that there must be then it is just a routine local minimum and what does that mean that means that so it was a it is a local minimum means that there exists an r greater than 0 such that f of x star is less than equal to f of x for all x in this neighborhood. Now, if it and it is if it is strict then there exists an r greater than 0 such that f of x star is strictly less than f of x for all x except for x star and in this neighborhood right. So, if it is not strict then it must be that in this in this inequality here equality must be attained for some point and this must be true for every radius r no matter how small you make the radius you should continue to get a point which is all in that in that ball and in that lies in that ball and in such that there is equality here right. So, what does this mean for all this means that for all r greater than 0 there exists an x in in this which is not equal to x star such that f of x star is equal to it. So, now let us let us take some some radius r also in that radius r let us see what is going on here is your point x star in this and here is my so here is my so suppose so I am just zoomed into point x star and this is the so this ball does not look like a ball. So, this is my point x star this is my radius r around it and so suppose here is my set s and so for every radius r I can find a point x such that f of x star is equal to f of x a point x in this ball right. So, here is suppose a point x that point x now this point x has the value of f at this point x is equal to the value of f at x star now what can I do f of x star is certainly as lower less than value less than equal to all of these everything in this radius. So, what does this mean this means that so everyone is following me here see remember x star is a local minimum. So, there is certainly a small enough radius in which it is the it is the look it is the it is it is it is its function value is less than the function value of all other point. Now in every radius around x x star also there is another point whose function value is whose function value matches with that of f at x star. So, which means that if I take if I take a small enough radius then in that radius x star is both a local minimum and there is another point which matches whose value is the same as that of same as that of x star. So, what does this mean now around this other point now I can draw another small ball. So, what what can I that that that small ball is going to be that small ball is going to be inside this this particular ball right. So, all these points their value is going to be is already less than equal is greater than equal to f of x star but f of x star is the same as f of x. So, f of x is going to be less than equal to the value of all of these small all of these small which would make x a local minimum right. So, if x star is not isolated this these new points that you keep finding they themselves become local minimum for small enough radius. So, what this means is this would imply that x for small enough r. So, for all small enough r all small enough r x is x is x is a local minimum. What this means is that x star is not isolated. So, x star cannot be isolated. So, if it is isolated if it is isolated and not strict we get to a contradiction that it is not isolated to begin with. So, it has to therefore be it has to therefore be strict okay. So, every isolated minimum local minimum is also a strict local minimum. The converse is not true. So, every isolated local minimum is a strict local minimum that is what we just argued. So, every strict local minimum need not isolated. Here is an example of a function like this that demonstrates this. So, I encourage you to go and plot this when you go back to your go back home you can plot this at in MATLAB or something. So, to consider this function f of x equal x to the 4 cosine of 1 by x plus 2 x to the 4 and let us take this in say minus 1 to 1 except 0 at 0 you define f of 0 as 0. So, what you will find when you plot this is that this function has many, many undulation as you go around 0 as you go closer and closer to 0. But of course, the least possible value is at 0 but you have many, many, many, many undulations as you go closer to 0. So, what will happen is that 0 is a strict local minimum but in its neighborhood are many, many other local minimum. So, in every neighborhood there is another local minimum. So, it goes like this at 0 but then something like this. So, there are many oscillations all happening at higher and higher frequency as you go here. So, this is at 0 and then in every neighborhood I have zoomed into this but if you go for no matter how much you zoom in you will keep finding another local minimum. So, it will be this local minimum, this local minimum, this local minimum etcetera.