 So, now let me actually write out a convex optimization problem. So, consider this convex optimization problem minimize fx subject to gi of x less than equal to 0 and Ax equals b. So, I going from 1 to m Ax equals b. So, A now is a matrix in R p cross n. So, it has p rows and we will assume that A is full row rank. So, in short rank of A is equal to p. Now we will assume. So, suppose f and all these gi's are convex and suppose p star is greater than minus infinity and attained. So, that means the optimal value of the primal is finite and it is attained by some point x that is feasible. Now, first little observation that is quite easy to make is that and I will leave that as a claim. It is very easy to prove that if f, gi is I going from 1 to m. If these are convex then g the set g is convex. So, to if you want to see this, this is not hard to show. So, how does one show this? Maybe we can quickly take a look here. So, what we need to argue is that if u1 v1 t1 and u2 v2 t2 are in g then any convex combination of there of is also in t. So, maybe I will just show you the proof of this. Let me do a proof of this. So, let suppose u1 v1 t1 and u2 v2 t2 suppose they both belong. So, they all belong to g then alpha in 0 comma 1 and define say u bar v bar t bar as alpha u1 v1 t1 plus 1 minus alpha u2 v2 t2. We need to show that u bar v bar. So, we need to show that u bar v bar t bar belongs to g. So, how would you show that? Well, you look at u bar. So, what does it mean to show this? We need to show that there exists that is we need to show there exists an x such that fx is less than equal to t bar g i of x less than equal to u i bar and a x minus b is equal to a x minus b is equal to v bar. So, what have I done here? See notice that this is my this was my optimization problem in the original form. So, to bring it in the in the form as before I have basically written this as a x minus b equal to 0. So, I had my when I took points u1 v1 u1 v1 and u2 v2 t2 that 0 got replaced by v1 and v2 respectively and now and now for this particular point I am looking for v bar. So, since u1 v1 t1 and u2 v2 t2 lie in line g. So, clearly we have that there exists x1 and because of superscript x1 and x2 such that f of x1 is less than equal to t1, g i of x1 is less than equal to ui1 and a x1 minus b equals v1 and similarly fx2 is less than equal to t2, g i of x2 is less than equal to ui2 and a x2 minus b is equal to v2. So, now remember t bar is what? t bar is alpha t1 plus 1 minus alpha t2. Similarly, u bar is alpha u1 plus 1 minus alpha u2 v bar is alpha v1 plus 1 minus alpha v2. So, what I will do is I need to show that there exists. Let us call this x bar. Let there exist an x bar such that all this. So, what I will do is take x bar as take x bar as alpha, take x bar as alpha x1 plus 1 minus alpha x2. In that case, what would I get? Well, since f is convex, I would get that f of x bar is less than equal to alpha f of x1 plus 1 minus alpha f of x2 and that itself is less than equal to alpha t1 plus 1 minus alpha t2 and that is what? That is simply t bar. Similarly, g i of x bar is going to be less than equal to alpha g i of x1 plus 1 minus alpha g i of x2 and that is less than equal to alpha ui1 plus 1 minus alpha ui2 which is nothing but alpha u bar. So, which is nothing but which is nothing but u bar and A x bar minus B is actually equal to A alpha x1 plus 1 minus alpha x2 minus B and that is equal to now what I will do is I will write this I will split this B as alpha B plus 1 minus alpha B and that would give me A alpha times A x1 minus B plus 1 minus alpha times A x2 minus B and this is equal to as you can check A x1 minus B was equal to V1 A x2 minus B is equal to V2. So, it will be alpha V1 plus 1 minus alpha V2 and that is nothing but V bar. So, in other words, we needed to show that this particular thing is true and that means that required us to show that there exists such an x bar that satisfies all of these and we were able to show that by taking x bar to be alpha x1 plus 1 1 minus alpha x2 where x1 and x2 are actually the points that the x1 and x2 that give you that because of which u1 ui v1 t1 and u2 v2 t2 are in G. So, what is this done this is given us that by doing this what have we shown we have shown basically this claim here. So, just to make sure everyone is on track make this is the claim that we have shown that if F and these G's are convex then then these F's and G's are convex then this set G is also convex. So, now let me move forward to the duality problem. Now, let me define this another set let me define this set T as capital T as follows it is 0, 0, S such that S is less than P star. So, what is this well this is nothing but an open segment along the T axis it is a 0, 0, S such that S is less than P star. So, it is an it is a segment that goes from minus infinity. So, if you or rather let me go back to the figure that drawn here well it is a segment that starts from minus infinity and goes all the way up till P star but does not quite touch P star. So, it is open at P star. So, it is this segment. Now, this segment is obviously a so this is obviously a convex set. Now, the thing that that is important for us is that T intersection G claim is that T intersection G is empty. Now, this may seem very natural because of the kind of figure I have drawn here no wonder it has to be you would think that yes T intersection G is non-empty but it needs a bit of proof. So, for example that G could very well do this it could go down here and then intersect here again and so on. So, we have to be a little careful and rigorous about this particular argument. So, let us do that. So, T intersection G is empty. So, suppose proof again suppose u v T belongs to G sorry belongs to suppose u v T belongs to T then it means that or rather let us put it this way let us start with suppose u v T belongs to T intersection G. Now, if it belongs to G if it belongs to T then it means that u equals 0 v equals 0 u equals 0 v equals 0 and T is less than P star. But it since it also belongs to G it implies that there exists an x such that f x is less than is less than equal to T. So, there exists an x such that f x is less than equal to T Gi of x is less than equal to 0 and Ax minus B is equal to 0. But then T itself is less than P star from here. So, if T is less than P star it means that there exists an x which is feasible. So, by this feasible for P such that the value of f of x is less than the optimal value of P. So, what does this mean? It basically means that if T intersection G is not empty then you should be able to find an x whose optimal value is actually less than less than the stated value P star and that is absurd because P star is the least possible value. So, consequently this is not possible. So, this is a contradiction. So, what does this mean then? This means that T intersection G is empty. Now, T itself because it is the kind of segment that I mentioned that for minus infinity it is an open segment for minus infinity to be that. So, the set T is a convex set. We just said that we actually just showed that G itself is also a convex set. So, T is a convex set G is also a convex set. So, T is a convex set G we showed is a convex set. So, now what we have are these two convex sets whose intersection by our claim here is empty. So, we have got these two convex sets that do not intersect which means what? So, then what does that mean? That means that we are in the regime of now the separating hyperplane theorem. What is this saying? So, which means that there exists a separating hyperplane. So, what does this mean? So, there exists a separating hyperplane or a hyperplane separating for T and G. So, there is a hyperplane that separates T and G. So, that means there exists let us call this. I am going to use this notation lambda, tilde, theta, tilde, say let us call this and let us call this mu, mu tilde, say such that. So, there exists this lambda tilde, lambda tilde, theta tilde, mu tilde such that this hyperplane remember from the separating hyperplane theorem says that this hyperplane should have a non-zero slope. So, that means such that this is not equal to 0. And so, there is a hyperplane with a non-zero slope such that now if I look at the the set G and look at the set T, they lie on either side of the hyperplane. So, if I look at the least value of this hyperplane over the set G. So, you look at infimum over u v T in G of lambda tilde, theta tilde, mu tilde transpose u v T this has to be greater than equal to the largest value over u v T of the same function lambda tilde, theta tilde, mu tilde transpose u v T as u v T ranges now over T. So, this is the separating hyperplane theorem. So, what is this effectively saying that entire set G lies on one side of it and the entire set T lies on the other side of it and there is a weak inequality here because they could both the infimum and the supremum can end up being equal. So, now let us evaluate these things carefully. So, u v T belongs to capital T if means what that it has to be that u and v are 0. So, this particular quantity on the right hand side is actually just supremum over T less than p star. So, u v T belongs to capital T means that u is equal to 0, v is equal to 0 and T is less than p star. So, it is this T less than p star of mu tilde into T. Now, I have not yet we are not yet in a position to evaluate this. So, but let us leave it at this particular state. So, now let us look at the one on the left then what is the one on the left. So, why aren't we in a position to evaluate this? Let us understand this for a moment. So, why aren't we in a position to evaluate mu tilde into T the sorry the supremum of mu tilde into T over for T less than p star. The reason is because we do not know the sign of mu tilde. See, if the sign of mu tilde is if mu tilde if mu tilde was were positive, if mu tilde were positive you would evaluate this quantity you would maximize it by putting T equal to the supremum would be at T equal to p star. If mu tilde is negative then the problem becomes very different. So, let us just hold our horses here before we analyze before we go ahead and analyze. So, let us first look at what is on the left. What is there on the left? Well, this whole thing I can spread out and write this a little bit. Let us write this out in a little more explicit form. So, let us write this as lambda tilde transpose u plus theta tilde transpose v plus mu tilde times T as u v T belongs to G and so I should put this as a greater than equal to. Now, if you look at look let us observe this particular equation here. So, on this quantity here this u v T belongs to G which means that you can can be so go back to the definition if you go back to the definition see what did I what would we say is how does G look G is unbounded in the u direction. So, whenever there is a u in G a larger u can also be fit in because of the way the set has been defined. These inequalities will continue to hold. Similarly, if you have a T in G then you can always fit in a larger T. So, what this means is G is always unbounded in the u and T along the u and T directions going towards plus infinity. So, what this means is that if u v T belongs to G if u v T belongs to G here then u and T can be made arbitrarily large. So, these their values can be made arbitrarily large. So, now for if their values can be made arbitrarily large what does this mean? If their values can be made arbitrarily large what is the what is the only conclusion that can be drawn from here? Well, the only conclusion that we can have is so if the if these so let us think of it this way. So, if lambda or if either lambda or mu were negative lambda tilde or mu tilde were negative. So, if lambda tilde or mu tilde were negative and the fact that you can drive u and T all the way to plus infinity would mean that this the infimum here would be would actually be the infimum here would actually be minus infinity. So, if lambda tilde and mu tilde were negative then the infimum here would be minus infinity but then the infimum is lower bounded by some particular finite value here remember. So, the infimum is lower bounded by some particular finite value which is which is the supremum of this linear function over T. So, since this is a supremum the supremum cannot be minus infinity because it is you can always take any real value and evaluate it to something finite. So, so the fact that this is this is lower bounded by some finite value means that you cannot possibly have lambda tilde and mu tilde as negative either of lambda tilde and mu tilde is negative. So, if lambda tilde less than 0 or mu tilde less than 0 then LHS equals minus infinity which is absurd since RHS is finite since RHS is finite. So, this cannot be minus infinity. So, which means that both lambda tilde and mu tilde are greater than equal to 0. I write this separately lambda tilde greater than equal to 0 and mu tilde also greater than equal to 0. So, now if mu tilde is greater than equal to 0 then that tells us also something about what we what we were talking about here. Well, if mu tilde is greater than equal to 0 then the supremum here is to be equal to mu tilde times P star. So, that has simplified our expression a bit let me change my color and so you can keep track. So, the left hand side has now become lambda tilde transpose U plus theta tilde transpose V plus mu tilde times T greater than equal to mu tilde into P star and the infimum here is over U, V and T belonging to G.