 Now let us make a few other observations here. I just said that an optimal solution is this lambda i star equals 1 and all the other lambdas equal to 0. Are there other optimal solutions? Are there other optimal solutions? Yeah precisely. So if there are there could be other optimal solutions say for example here I said well x i star is the one that is giving you the least here least of these right x i star was the one that is giving you the least. There could have potentially been another one right say for example this could also there could also be another one say let me write it here suppose c transpose x i star is equal to also c transpose x i hat another index which another extreme point which happens to have the same objective value as c i star then both I could have taken i hat here and I would have I could have continued with the same argument I would have got that one as a solution. In fact in that case I could have done even more instead of putting lambda i hat equal to 1 and everything else is 0. What I could have done is this I could have constructed a solution like this lambda i equal to 0 for i not equal to i star and i not equal to i hat. So if it is not i hat or i star then it is 0 and the remaining two the other two that means i star and i hat are chosen to be if greater than equal to 0 and sum to 1. All that matters is that they sum to 1 because then in that case if you look at this term here all the what is this going to reduce to in that case if you look at the objective the objective is going to reduce when you look at this summation lambda i c transpose x i this because all the other lambdas are 0 what is left is just lambda i star c transpose x i star plus lambda i hat c transpose x i hat but then c transpose x i star is equal to c transpose x i hat there they are both both extreme points have the same value. So then what you are left with is just lambda i star plus lambda i hat times c transpose x i star and so long as this sums to 1 I have I still get the same value. So what this means is if I not only is x i star as an optimal solution x i hat is also an optimal solution and the segment joining them the segment joining them which is lambda i star x i star plus lambda i hat x i hat where lambda is sum to 1 that entire segment joining them is also an optimal take any point on that segment that is an optimal solution. So that is seen in this figure that I had drawn here you have this is like your point x i star this is your point x i hat and this is the entire segment joining these two points. So good question so is it possible so the question he is asking me let me draw a new open new a diagram here which is very nice question. So the question he is asking me is suppose let us take this sort of polygon here is suppose your x i star i star and the other point that I so the way I illustrated this was I said like let us take this as x i hat right x i hat here and x i star here and these two are both optimal and the entire segment joining them was optimal but what the question that has been asked is what if you did not have this as x i hat but rather say this as x i hat say it was you know what he said was diagonally across say suppose this is x i hat in that case this would the segment joining them which is passing through this would that be optimal. So then where is the fallacy you are saying it cannot be optimal then where is the fallacy the point here is that you you it will never happen that such such to these two points are both optimal because the very fact that these two points are optimal means that they are on the same contour and then your contour can be moved this way or this way to get a better answer better value right that can that sort of thing cannot happen. So eventually you it has to be that the extreme points are what is there on the same face of the polyhedron okay the extreme points that are all optimal are on the same field in which case the entire phase becomes optimal yes yes so we are exploiting to the hilt here that the problem is linear okay in the sense that the optimum the objective is linear the constraints are linear the shape of the polyhedron all of that is being exploited completely okay if I have if I had some other non-linear objective then a lot of these arguments will not go through as elegantly as they go through here and of course if you did not have a polyhedron then you do not have Minkowski while theorem to begin with so that is that is another problem okay. So you this says that so let me come back to what I was saying so this just ensures tells us that there is at least one extreme point that is a solution does not mean that there are non-extreme points cannot be solutions okay there can be that depends on the depends on the objective okay. Now let me ask you can there be a ray of solutions so of course there can be a segment when you have two extreme points that are that are a solution the entire segment joining them with the solution but can you have an entire ray means that a point and a direction along which you keep going and you keep yeah and you keep getting all those points are solutions is that possible yes so it needs a polyhedron to begin with for you to have a ray of solutions you need to have a ray of feasible points right so the polyhedron has to be unbounded okay. So it is interesting to ask this question only if you have an unbounded polyhedron which means that you are in the regime where all these D's are these D's the DI's are not 0 okay there is at least one non-zero DI here okay a non-trivial extreme way okay then in but in that case but does that ensure that your you have a ray of solutions it can be and then what must be the case for that situation right so remember we said that we argued that C transpose DI's have to be greater than equal to 0 right and therefore we said we should put all the coefficient mu corresponding to it as 0 all the muses were set as 0 but what if one of the C transpose DI's was exactly equal to 0 in that case the mu would be material it would not appear in the objective at all right. So you look at the objective sorry it is here in that case the mu the mu would drop out because C transpose DI is equal to 0 right so if you have any if C transpose DI is equal to 0 for some I then what can I take the mu corresponding to that that I as I can take it as anything right so then mu the mu I greater than equal to 0 is optimal any mu greater than equal to 0 is okay right which means then I can start with any solution. So if X star is a solution then I know that X star plus mu DI belongs to P this is feasible it belongs to P because after all DI is an extreme rate right so I can keep going along DI form X star and remain in P the X star plus mu DI is in P and what is the objective at X star X star plus mu DI well the objective is C transpose X star plus mu DI is equal to C transpose X star plus mu times C transpose DI and what is C transpose DI C transpose DI is 0 right so this means this is equal to C transpose X star. So what this means is if you if X star is a solution and DI is a direction D such is a direction such that C transpose DI is equal to 0 then X star plus mu DI is also a solution for any greater than equal to 0 right so I can keep adding this mu DI to my point X star and I will keep generating a solution so it is not just X star that is a solution the entire way starting from X star going all the way along the direction all the way till infinity along the direction DI that entire thing is a solution okay so this way you can get a way of solutions also right so but remember it requires a fine coincidence your C transpose DI has to be exactly 0 right so they have so your objective function the direction in which it it is decreasing has to be exactly orthogonal to the direction in which the set received okay so you get but you get a way of solutions but remember still that the optimal value is still finite because the optimal value is not changing because of this orthogonality between C and DI the optimal value just does not change as you go along that way that is what this right okay so what does this taught us what what this is taught as is see Minkowski Weyl theorem is super powerful if you it basically has reduced optimization over linear optimization linear programming to just a matter of enumerating C transpose Xi is over the extreme point right but so but the catch in all of this is that we do not know what the extreme points are to find to given the X less than equal to B type of representation of your polyhedron your half space representation from there finding the corresponding X1 till Xk and find and the D1 till Dm that is where the challenges right that is what so the all algorithm design would have to in some way or the other deal with this issue okay trying to get to this okay at least if it is aiming to find an extreme point solution just somehow has to get to this okay so this this is the qualitative theory of linear programming that is since we have a few minutes let us just do a few examples okay before I do an example let me I want to also mention one more point yeah this is this is important go go back to the statement of the theorem again so we we saw how we made use of of this point that the optimal value of the LP is finite we made use of this quite that the optimal value of the LP is finite that is how we got that the mule should be 0 right where did we use that the that the polyhedron must have at least one extreme point yes we we use that in invoking Minkowski Weyl theorem right in Minkowski Weyl theorem also requires that that this set has at least one extreme point that is when you can write it in this sort of form right so the Minkowski Weyl theorem we used it in Minkowski Weyl theorem now can you give me an example of a polyhedron that does not have an extreme point yeah so so if I take any hyperplane for example that is a polyhedron that has no extreme point right so in that case we cannot say that the extreme point is a solution okay. Now what the so so come back also now to what I had done in the previous lecture if we said we said we start with a problem like this which is simply x belongs to some polyhedron which means it is written in this sort of form x minimize c transpose x subject to ax less than equal to b but then we converted it to this sort of problem we said minimizing c transpose x subject to ax equal to b and x greater than equal to b all right we converted it into this form remember so the again emphasis was on form these a's are not necessarily equal the b's are not necessarily the same the c's are also not necessary in fact the x is also possibly changed just that it means that there is a problem you can write this in this form and recover back the solution of your original all right okay now focusing on this form again now a polyhedron like this when I write it as ax less than equal to b this sort of polyhedron need not have an extreme point right so this could be for example just a hyperplane two opposing inequalities differ that will define a hyperplane that has no extreme point what about this sort of polyhedron what about the polyhedron on the right so we were let me be more specific here we said that a is full row rank and we said that b can be taken as greater than equal to 0 so this this sort of polyhedron now is a polyhedron which expressed in this form ax equal to b and x greater than equal to 0 for all the constraint all the all the variables right this sort of what can you say about this is a much more specific polyhedron right it is not a generic polyhedron like the one on the left the generic polyhedron may not have an extreme point but what about this specific one so it turns out this sort of this specific polyhedron actually has an extreme always has at least one extreme point yeah see I will I will I will give you a pictorial view of this right now we can prove it in the next class see the way this sort of polyhedron looks ax equal to bx greater than equal to 0 the way this sort of polyhedron looks is your from the two rays to n possible orthans of the space Rn the x greater than equal to 0 is one of the orthans right so the signs of x will distribute the entire space into two rays to n possible orthans x greater than equal to 0 just tops out brings out one of them this is suppose that orthans so this is just talking of this is just expressing x greater than equal to 0 what is ax what is ax equal to b ax equal to b is the intersection of several hyper planes in this space right so it is actually a hyper plane okay okay a lower dimensional hyper plane it is intersection of several hyper planes so it is as you keep taking intersections you get lower and lower dimensional sort of affine sets so it is some affine set ax equal to b it is an affine set that is being that is being intersected with the non-negative orthant this orthant here right so an affine set remember is like just a shifted subspace so it is so I will I will draw it like this this is my my ax equal to b and ax equal to b intersected with x greater than equal to 0 is this region it is basically just this right just that segment that part there here because we are in R2 it looks like a segment but in general it will be some it will be some say some poly drill set this sort of set can also be unbounded okay it is possible for example for example it can be that this ax equal to b is aligned like this with the axis in which case it will be unbounded in this direction it can be unbounded it is not a bounded set but one thing is for sure it cannot escape eventually it cannot it cannot happen that it does not hit any of these axes there will be it will either hit this axis or this axis one of those n different axes is going to be hit so it will intersect one of and when it intersects one of those axes okay one or more of those axes right okay this is this is all in happening in higher dimensions so remember when in several of its coordinate when it intersects several of the coordinates of x will end up being 0 and the others will not be 0 when you when you get to that sort of point that sort of point will end up being an extreme point of the set so the kind of polyhedral that do not have extreme points are those that are sort of you know say hyper planes etc which which you cannot which you are not able to sort of confine so here the x greater than equal to 0 forces you effectively to end up having an extreme one so I am giving you an intuitive picture here but basically what you what what is the claim the thing to notice is a set like this ax equal to b and x greater than equal to 0 this sort of polyhedron always has an extreme point so what this does is this actually helps us in designing algorithms for linear programs implicitly a set like this an optimization problem like this it did not if it did not even have an extreme point so we could not even say where okay that we did not even have a target saying okay we are looking for an extreme point to solve the problem now we this once we reduce it to this standard form not only is it standard and and easy to program and so on so it in addition it is guaranteeing us that there is an extreme point so the search for a solution also becomes easy so you get so this sort of set must have an extreme point is this clear so yeah so this is so the extreme points of this sort of set are what are called basic feasible solutions I will I will talk about this basic feasible solutions so since you have about 10 minutes I can just define you for you what this is so if I take the set the matrix A okay and I can now what I can do is I can write it in the following form to remember A is full row rank so if A has m rows and n columns then I can write it in the form in this sort of form where the vectors B the vectors B are such that they are columns of A that are linearly independent and I can how many can I put in how many such columns can I add to be m of them right because that is the rank of A so I so I will take B to have m columns okay so B will now have m rows and m columns and all the remaining I n minus m I am putting in n okay n minus m columns so I can I can choose such a such m columns from my from this from the n columns that I have from for A then that will give me a way of writing A in this sort of form as B and n and then if I solve using this what I will do is to solve for x what I what I will do is I will so I am going to multiply this by x let me write this as x B and x n so this x B is of length m it is what multiplies with B x n multiplies with n right so what I will do is I will put I I will solve for x B and put x n as 0 so putting x n equal to 0 and solving for x solving for x B what does this give me this give me the equation B x B equal to small b right which is and because B is now m cross n and it has linearly independent columns right so B is invertible so x B will be equal to B inverse B right so a point like this x B comma n equal to B inverse B comma 0 this sort of point is what is called a basic solution a basic solution is a point is a point like this so what you need to do is you create you look for columns of A that are linearly independent you will m of them you should be able to find m of them so take m of these linearly independent columns solve for the x's corresponding to those columns and put all the remaining x's as 0 so put the remaining x's as 0 and solve for the x's that are corresponding to this column that will give you a vector like this B inverse B comma 0 now this sort of vector may not satisfy x greater than equal to 0 right so if B inverse B is also greater than equal to 0 then we call it a basic feasible solution so what I will argue next time is that actually basic feasible solutions are the same as extreme points of this sort of set so A x equal to B and x greater than equal to 0 solving for extreme points is effectively solving for finding points in x by putting some of the coefficients equal to 0 and solving for the rest right and that is like what I was saying you have these you have this affine set it intersects it hits one of the axis by when you say it hits the axis it means that some of the coordinates become 0 you need the other ones need to be found and that then gives you the that the other ones need to be found through the through the equations that you have so that is how we get to extreme points we will do this in more detail next time