 Welcome everyone, so today I will talk about an first to begin with I will talk about an important property called an important condition you can say called complementary slackness. And this is a condition that comes up because we whenever we consider optimization problems which have inequality constraints, so I will first illustrate this today in the context of linear programs and then from there then we will take off to a more general problems and hopefully you will see either today or in the next lecture complementary slackness coming up as a condition in more general optimization problems. So, let us look at this linear program, so this is called let us say we are maximizing c transpose x subject to Ax less than equal to b and x greater than equal to 0. Now, this is not in standard form, but and moreover it is also a maximization problem, but you can convert it to standard form and then from the standard form derive its dual and if after you simplify the dual you will find that the dual takes this form it is I mean it is a minimization. So, here the maximization is over x here minimization I will just use the variable lambda here minimization is over lambda of the objective B transpose lambda subject to the constraints that A transpose lambda is greater than equal to c and lambda is greater than equal to 0. So, now let us repeat the observations that we had made about about linear programs and standard duals before as well. So, if my matrix A is in R m cross n, C is in R n and B is in R m. So, in that case the number of inequality constraints that we have here these inequality constraints apart from the constraint x greater than equal to 0 the number of inequality constraints that we have here this is equal to this is equal to m which also happens to be the number of variables in this in the dual problem. The dual problem has m variables because lambda is a m length variable is an m length vector lambda is in R m. So, the number of inequality constraints that we have in the primal problem is equal to the number of variables of the dual problem. And if you look at the number of inequality constraints in the dual problem that is equal to the number of variables in the in the primal problem. And I had mentioned to you before that there is a close correspondence between these. So, what one can think of is that for every constraint in the primal you actually have a variable in the dual and vice versa. So, every constraint in the primal LP has so this is I have not written here this is primal and this one is this is the dual. Every constraint in the primal has a corresponding variable in the dual vice versa. Vice versa means that every constraint in the primal has a corresponding variable in the dual. So, what I will do today is through this condition of complementary slackness I will actually make this more precise and I will tell you how these which variable is actually corresponding to which constraint and so on. So, let us define a few quantities. So, let us define omega p as all x such that a x is less than equal to b and x is greater than equal to 0. So, omega p is simply the feasible region of the primal and omega d is all lambda such that a transpose lambda is greater than equal to c and lambda is greater than equal to 0. Now, because these are duals to each other these two problems the we have you can very easily verify that we already have weak duality, weak duality between them. What does this mean? If I take any x in that is feasible for the primal and look at look at c transpose x and that has its value is less than equal to. So, what have I done here? What I have taken I have this is true for all x in omega p and all lambda in omega d. So, what have I done to get to this relation? I have c transpose x. So, and I know that c is less than equal to a transpose lambda or c in other words c transpose is less than equal to lambda transpose a. So, I have multiplying both sides by the vector x and x being greater than equal to 0 because x belongs to omega p x is greater than equal to 0 then that ensures that the inequality is not flipped. So, in short I am taking the dual I am taking the dual constraint and then multiplying both sides by x that gives me lambda transpose a x is greater than equal to c transpose x. But then lambda transpose a x is actually the same as it can be it can be further bounded because a x is less than equal to b. So, that can be further bounded and then you get that it is bounded by b transpose lambda. And again lambda being greater than equal to 0 let me make sure that my inequality is preserved. So, this is like we did in the standard form you can get you have weak duality here as well. Now, the important thing here is the complementary slackness condition and so let me write the theorem now. So, x star in omega p is optimal for the primal LP if and only if there exists a lambda star in dual in omega d omega d such that now such that let me write it like this. So, suppose on the side let me introduce a bit of notation. So, let the matrix A that is represented as matrix of numbers a ij where i runs from 1 to m and j runs from 1 to n. So, i corresponds to rows and j corresponds to columns. So, x star in omega p is optimal for the primal LP if and only if there exists a lambda star in omega d that means it is a variable that is feasible for the dual LP such that you have the following two conditions hold. If I look at a ij x star j and I sum this from j equals 1 to n this quantity is less than Bi and this implies the condition that this is the condition that this inequality holds means that summation a ij x star j from j equal to 1 to n is less than Bi this should this implies lambda star i equals 0 and the other way round as well the other condition is that if I do again do a ij and write it this way. So, they consider the summation lambda star i a ij i running from 1 to m that is less than less than cj implies x star j equals 0. So, when so what this means is so if the first inequality if this inequality here the way you should understand this is it says that if this inequality here is strict. So, then you must have that lambda star i corresponding to that inequality is 0. So, for every every const so this what is this inequality this inequality is the ith constraint of the primal LP this is the ith constraint of the primal LP. So, if the ith constraint of the primal LP holds strictly I mean it does not hold with equality then the lambda i corresponding to that must be 0. Likewise if you look at the jth constraint in the dual LP if that holds strictly sorry this there is a there is a slight mistake here in the direction of the inequality. So, sorry there is a mistake in the direction of the inequality I just corrected it. So, if likewise if this inequality holds strictly the jth inequality in the dual if that holds strictly then the corresponding primal variable xj must equals 0. So, what you should then the way to think about primal and dual variables is that what you have so your your lambdas are actually the the the lambda the variable lambda i which multiplies with bi in it in the objective is the one that corresponds to the ith constraint here. For the ith constraint here you have a variable lambda i and likewise the variable cj xj that multiplies with cj in the objective of the primal is the variable that corresponds to is the variable that corresponds to the jth constraint in the dual. So, so what you have here are actually there are from i equal to 1 till all the way till m there are m of these constraints and then for each of these constraints you have a you have a dual variable which is lambda 1 till lambda m the and likewise in the in the dual you have constraints now going from j equal to 1 all the way till j equal to n and corresponding to the each of those variables you have dual variables of the dual which are actually the primal variables. So, those are then x1 till xn and these variables when you have when you are at optimality they end up satisfying these constraints these conditions this is equivalent to being optimal that means whenever whenever the ith constraint in the in the primal LP holds with strict inequality the corresponding dual variable must be 0 and whenever the jth constraint in the dual LP holds with strict inequality the corresponding the corresponding dual variable of the dual which means the primal variable x star j must be equal to 0. So, this so we will this is what is called complementary slackness. So, what this is referring to is that whenever there is whenever there is the word complementary slackness only says this that whenever there is slack in one of these constraints then the there is can be no slack for the for the dual variable corresponding to that means the dual variable must be at its least value. Likewise if there is a slack in this constraint then this one must be at its least value. So, it will it will turn out that moreover it will turn out that lambda star is so that is a that is a good point. So, here this statement only says that x star is optimal for the primal LP if and only if there exists a lambda star like this it will turn out that lambda star itself is actually optimal for the dual LP. So, in fact, x star and lambda star end up being optimal for their respective problems through these conditions.