 x 1 is 20 x 2 is we want to use that whatever the theorems we have studied it use this theorem to find out the dual solution directly from the primal solution and we have seen using this equation equation a and b we have ultimately we have solved and got these values that y star is equal to 1 y 2 star last class we have written 2 solution with it will come 1 so our solution is y star is 1 y 2 star is 1 and y 3 star is 0 so if you find out the objective function value in dual problem that will come if you see the expression for that one that fd 80 x y 1 this is 80 x y 1 star plus then your 100 x y 2 star 100 x y 2 star plus then you say 40 x y 3 star 40 x y 3 star then y 3 star value is 0 so now y 1 star value is 80 y 1 star value is 1 and y 2 star value is your 1 so this value will objective function dual problem it will come 180 similarly if you see the primal problem objective function values that of the primal function objective function is zp is equal to zp is equal to you see 3 x 1 3 x 1 plus 2 x 2 agree this is our objective function of the primal problem and we have given x 1 star you see here the value of extension is 20 solution of primal problem is maximization of the objective function at what point will get the maximum value of the function x 1 is 20 x 2 is 60 if you put it here 30 x 20 plus 2 x 60 so its value is 180 so objective function value in dual problem that maximization problem value is same as the dual problem minimization minimum value of the objective function value is 180 both are same at optimal conditions optimal point in what is called primal domain and as well as in what is called in dual domain or dual domain so today we will discuss about the solution of what is called quadratic programming problem using the simplex method agree so let us take in one example and through example we will explain that how we will solve the quadratic programming problem using the simplex method so let us consider that our what is called function is our function is minimize f of x that is we have to minimize is equal to x 1 minus 3 whole square plus x 2 minus 3 whole square and if you expand this one we will get it here x 1 square minus 6 x 1 plus 9 from this factor and from this factor x 2 square minus 6 x 2 plus 9 and if you simply this one x 1 square minus 6 x 1 minus 6 x 2 plus 18 so you can easily write in quadratic form like this way you see this one I can write it half to 0 0 there is no cross product term x 1 x 2 so I will write it this is this way and this before that one will be that x transpose and this multiplied by x then your next is your plus minus 6 minus 6 into x plus 18 which you can write it half x transpose this if you consider as a h matrix h which is in symmetric in general it is a symmetric matrix if you convert into quadratic form into x plus this if you consider as a c transpose that c transpose x again plus what is the remaining term is left in this expression 18 plus this I am denoted by d so this so this one the objective function which is expressed in quadratic form that one and a function is in quadratic form and this function is a objective function is a quadratic and convex if it is convex the what is called Hessian matrix of this function should be positive semi definite so which if you take the what is called Hessian matrix of this one second derivative of this function then it will come only capital H this term will come to 0 so now this is the problem which is we have converted into a standard what is called quadratic form which is a convex function in your problem so subject to inequality constant and what is called equality constant x 1 plus x 2 is less than equal to 4 so all inequality constants if here only we have one inequality constant in general if you have a number of inequality constant is m and number of what is called equality constant is pre then we can write number of inequality constant in matrix and vector form let us call this we have represented into a transpose into x is less than equal to our b that this the inequality terms right hand side what you get it that is b so where where our inner case x is what there are two decision variables x 1 and x 2 so this is our equation inequality so if you have a inequality constant of m such constants are there I can always write in terms of a transpose x is less than equal to b now if you have a equality constant are there let us call you have a equality constant x 1 minus 3 x 2 is equal to 1 so that we represent this one equality constant by n transpose x is equal to e and that inequality constant if you consider cross p number of equality constant if you consider our inequality constant is m cross 1 so we can write it here what is a transpose in your case in your case a transpose if you say see this one is nothing but a 1 1 this is our a transpose and what is our n transpose here n transpose in this case your will be 1 minus 3 agree so our e is equal to in our for our example is one and in our case b is equal to b is equal to 4 so so we have identified what is h what is our c transpose agree what is d what is a transpose what is b what is n transpose what is e all these things so our in general our problem we will write it like this way minimize a function f of x which is a half x transpose h x plus c transpose x plus d d is the constant subject to a transpose all inequality x transpose x is less than equal to b and b is the dimension of vector is m there are m such inequality constants are there which is combined together and equality constant n transpose x is equal to e that we consider m p cross 1 so now this problem can be solved by using simplex method first what is the necessary condition that this function should have a minimum value at what point this function has a minimum value this function means quadratic problem that function or quadratic which is a convex in nature so what is the necessary so one can easily find it by using KKT condition so let us write it first that lagrange function and you if you recollect we have discussed in details the lagrange function how to generate this lagrange function so lagrange function l is equal to our objective function what is our objective function x x transpose h x plus c transpose x plus d this is the objective function then that inequality constants then equality constants so we have a equality constant let us call we have a equality constant lambda transpose the lagrange multiplier I am just following what we have discussed earlier the KKT condition to find out the optimum value of an objective function so our equality constant is n transpose x minus e so this quantity is zero at what point we will get the optimum value or feasible solution of this one agree it must satisfy this zero so lambda transpose of this plus then you have a mu transpose lagrange multiplier is a vector mu transpose that multiplied by a transpose x plus s minus b so a transpose x minus b is less than zero so we have to add some what is called variable agree and that variable is called the variable what we have considered that variable should be some positive value so that this equal to zero so this variable is called the slack variables agree so this because that means you have to you have to this is the surplus variable that means we have to add this one x transpose x minus b is less than equal to zero so you have to add some value of this one so this is the your call what is called slack variables of this one and when that this will be greater than equal to zero that means this is a positive quantity you have to subtract something so that is called surplus variable so in your case it is a slack variables are there this and this value is greater than equal to zero but this lagrange multiplier associate with the equality constraints that sign is unsigned this value may be positive or negative for all types of inequalities x transpose x minus b less than equal to zero we have proved it that mu should be greater than equal to zero lagrange multiplier would be greater than equal to zero in reverse type of inequality for reverse type of equality means a transpose x minus b is greater than equal to zero that type of things then our mu should be less than equal to zero that means non-positive numbers in that situation but our case is ax x transpose a minus b is less than equal to zero so it is a non-negative numbers so keeping let us call this equation is equation number one okay so now now we can easily write the necessary condition KKT necessary conditions to have the minimum value of the function necessary conditions so what is the necessary condition of KKT del l lagrange function with respect to x you have to find out now see this our objective function is that is this one now I am differentiating this thing with respect to x so x transpose hx if you differentiate with respect to x that will be coming twice h second difference if you differentiate this one with respect to x then it will come twice hx so twice x h and yet is a half is there so it will be a coming h of x then next time you see c transpose x if you differentiate this one then it will come c only that is we have discussed earlier if you recollect this things then similarly if you this x term is involved first of this bracket first term okay so lambda transpose n transpose x this is no x variable lambda transpose e no x is involved so that term will be zero so here this will be a I can write this one if you see this one I can write n lambda whole transpose x this quantity I can write it this form now differentiate with respect to x that will be will be n into lambda only so that will be n into lambda similarly this x term involved in the first second and third there is no s term so if you partial differentiation of that term with respect to x we can write it this one this mu transpose a transpose x we can write it equivalently into this form a mu whole transpose x form then you differentiate with respect to x if you differentiate with respect to x you will get a mu is equal to zero now see this one and how many equation you will get it there are n equation is like this way and we have used this property when you are differentiating this with respect to we have we have used this property x transpose p x when differentiate with respect to x we will get it twice p x then we have a c transpose x when differentiating with respect to x the result is c that is we can verify by simply principles of what is called differentiation of a scalar quantity with respect to vector the way we did it so this is the first we got it next equation necessary condition of differentiating this with respect to mu assigned to zero now mu term is involved in only in the this term so if you differentiate with respect to mu that will come what will will get it for this one mu is a vector of that one you will get it and this whole quantity is a scalar quantity that will come a transpose using this property you can write a transpose x a transpose x plus s minus b because you are differentiate with respect to mu a transpose x plus s plus b is equal to zero this is this term is constant or you can say this one because this is a scalar quantity you take transpose both sides it will come whole transpose that mu now differentiate with respect to mu using that property it will get that one so and note here and note that our s is equal to we define s how many inequality constraints are there we have a m equality constraint that each component of this we defined like this way if you recollect our earlier case this one and how many this equation will be this is m cross one equation next stage we have to differentiate this with respect to lambda the lagrange multiplier with respect to lambda so lambda is associate with the this term again so since it is a scalar quantity take the transpose then you differentiate with respect to lambda partial differentiate with so that will come it will be n transpose x minus e and how many equation will be there equality constraints are there there are p equality constraints so we have all together n plus m plus p equation so this equation we have in addition to that we have a switching functions that we have seen earlier than del l your differentiate with respect to del s agree so this if you differentiate with respect to s this is a s square this s each m component of this one is s square and differentiate with respect to s so this will get it twice mu i see this one this this so each component will be mu one s one square plus mu two s two square plus mu three s three square so i am differentiate with respect to i so there will be twice mu i s i is equal to zero and i is equal to is equal to one two dot dot this is associate with the inequality constraints how many equality constraints are there m so this i can write further i can write it this mu i s i whole square both side multiplied by s i is equal to zero i is equal to one two dot dot m agree so if you give the equation number this is equation number two this is equation number three this is equation number four and this is equation number five and note that lambda that lambda is free in sign no restriction on its sign no restriction in sign in the earlier we told it beginning of the statement so you have a five equations are there in addition to that if you want to solve by solve this by using the simplest method that we have a lambda also one of the variable in the expression so this i can write it that next is note lambda is is free in sign so we can write it lambda is equal to lambda what is the lambda associate with equality constraints how many what is the dimension of this one p cross one so lambda i can write it y minus z each is p cross one and each component each component is greater than equal to zero this is also greater than equal to zero positive the difference of this one may be positive negative and zero this so this is the new variables we have been doing in place of lambda i will write it y which is greater than equal to zero minus z greater than equal to zero so we can reformulate the problem now if you just see we can write it this equation equation number two three four two three four i can write it now see this one equation number two what we can write it h so what is the vectors are involved x unknown lambda is unknown mu is unknown i will rearrange first x then mu then lambda and you have a s also this s also you have an s what is the dimension of s you have a m cross one yes so i will just rearrange in this way first x then mu and then s s and mu in the same expression s then y is a lambda lambda is a two parts if you see the lambda we have a two parts y and z accordingly i will write it so first expression h into x so i am i am just writing it here so i am arranging this one x and what is the how many decision variables of x are there n cross one side by side i am writing the what is called the dimension also so that it is easy for our to track this equation so next i will write mu what is the mu dimension m cross one mu is m cross one then i will write it s s is a vector whose dimension also m cross one so i will just partition then i will write it our lambda lambda has a two parts y and z so why i am writing this dimension m cross one then another is your z p cross one so this is our all unknown variables how many unknowns variables are there n m plus m n plus twice m plus twice p this is the unknown variables are there so from equation number two what we can write it say this is the constant this is we know from the description of the problems so i will take that is in the right side right of the equation so hx h multiplied by x then you see next variable is mu mu multiplied by a multiplied by mu mu is a vector column vector so a multiplied by mu a multiplied by mu so this i have written it then lambda n multiplied by lambda lambda is y minus z so there is no s here so i will write this value of s is here zero and that is lambda is n into y minus z so y is involved with n z also will involve with minus n so i will write it n n then partition minus n so let us write this dimension of this one and what is the how many equations we will get it from the first equation number two how many equations there are n equations are there so we have here to here n equation and this is n a into mu a dimension is n and mu dimension is m this is also same as s dimension same as mu so this is also will be m n into p n into p this dimension n dimension is n and this is a p and this is also p so we know this block so this is the corresponding to i can write this corresponding to equation number two is equal to what is the right hand side right hand side is equal to c if you take that is a minus c so minus c and what is the dimension of this say what is the dimension of this one it is a n row and one column agree now write it it is a second equation a transpose x so x corresponding to a transpose then you see there is a s and this is a constant term from the description of the problem you take it right hand side b so s is next component is zero then it is a s i because s into i i can write it always i into s this i can write i into s and other terms is zero zero this and this term is equivalent to b and b is the number of inequalities right hand side of the b is the right hand side of the inequalities that dimension is your m so this dimension is also will be m now what is the this is equation two corresponding to equation number three we have represented now c equation number four c equation number four that this is the e n transpose equality constant n transpose x is equal to e it is a constant term it take it right hand side so then if we have a only n transpose x so first block will be this first block will be n transpose and other elements are zero zero zero and if you take the right hand side of this one it is a e and how many equality equations are there p so this also will be pay so we know the whole description of that one that that means if you consider the whole matrix is b the b dimension of this one is how many rows are there n into m plus m p rows if you see this one the rows of that one is n plus m plus p so this dimension is n sorry n plus m plus p this one into number of columns n twice m plus twice p so this is the matrix the whole matrix which way you say which way got it obtained from KKT necessary conditions agree KKT necessary conditions and that dimension of this one let us call this I am capital X and that capital X dimension n first n components of x then m components of mu s components of s then y component y p components z p components so this x as a dimension n plus twice m plus twice p agree and that much and this is equal to I consider this one is your d and this d dimension is if you see how many rows are the n plus m plus p so this dimension is n plus m plus p into one into one that number of rows is one agree number of rows is one so now see this one now if I write it this equation now I can write it h h into x one h into x one plus a into mu n into that is y minus n is equal to y or you can say this one directly what is this equation stands for put the value of h put the value of c put the value of n and put the value of a in this expression then you will get it this let us call I am writing it first necessary condition h into x plus n into lambda lambda is y minus z that one agree plus a mu is equal to minus c agree so h is what if you recollect this problem that our h matrix is that one our h is if you write it h 2200 the diagonal matrix that is our 2 0 0 2 our x is what x 1 x 2 this is from equation necessary condition from 2 then n is what in our case n if you see this our n is where is the inequality constraints equality constraints this is our n n is n transpose is one minus three so this will be a n is n will be one minus three so n will be you are writing n is one minus three then it is y that our y is there is a only one that is this is the lambda that there is a only one what is called equality constraint so that dimension is in our case is one for our particular example is one so I can write y then minus sign one minus three z is dimension is one cross one because we have a only one equality constraints and a a matrix is you see a transpose is one one so it will be a column vector one one and mu we have a only one inequality constraints this equal to c value is c what is the value of c here that minus six minus six if you take the c transpose is minus six minus six if you take the transpose of that one it will be a what is called minus six column wise is equal to minus six and that preceded with minus six so it will be a six six so write more details two x one plus y minus z plus mu is equal to six one equation agree the next equation twice x two minus three y plus three z plus mu is equal to six so this is this two equation we got it from the equation number two of kkt conditions agree or directly if you write it this one directly after that notation you are familiarize you directly from this equation if you write it you will get two set of equations that is n is equal to n is equal to two we have a two equations we will get it here next m is equal to one number of inequality constraints is one we will get one equation from this one what is this one if you say a transpose x plus s so a transpose x a transpose is what a transpose our a transpose is one you see the our a transpose a transpose is one one so if you write it a transpose from this from third equation from third equation kkt from equation three is your you can write it a transpose x a transpose is y one a transpose is your one one into x x as a two components x one and x two agree so next is your s s is a there is a only one inequality constraint so this will be your i into s is equal to say is equal to b that is equal to this is equal to b or second equation of this one you see a transpose x plus s is equal to b so b value is what from the problem of the b value is our four agree we have a this equation this equation this equation then another equation will get from four kkt condition from four and four condition is n transpose x is equal to n transpose x is equal to e so what is n transpose n is what n transpose is what one see this problem one three into n transpose x n transpose x is x one x two is equal to our e value is what from the statement of the problem you just see the statement of the problem our e value is given to one e value is one so this is equal to one so if you make it this one x one minus three x two is equal to one so our kkt condition all are if it is a quadratic at a programming problem that our kkt condition is coming a linear form all equation is linear in more general is b into x is equal to d form agree this and where we have seen also it is a linear now instead of using y z mu s we can just continue thus variables in terms of x one so you have already have x one x two so if you consider next variable is if you see the word the way we make it argument of this one next variable will be mu mu how many components are having mu m components so I will consider up to x one x two dot dot x n next is x n plus one is equal to mu one next is x n plus two is equal to mu two in this way last element of mu will be x n plus m similarly this variables how many variables are n m variables are there so the first element of the s I will represent as a x n plus m plus one similarly second element of this one I will define as a new variable x n plus m plus two in this way and similarly we can continue this way so if you redefine if you define this one now then what we can write it and if you define this one what we can write it you see x and we have a another constraints are there switching constraints mu I or mu j s j is equal to zero that is what we got it mu j s j is equal to zero that what we got it and j is equal to one two dot dot m so now what we can write it from this equation what is the position of mu see the position of mu if I express all the variables in terms of x then first element of mu will be x n plus one second element of mu x n plus two so if it is so I can write it x n plus j multiplied by see this one elements of this one the elements of this one is s one square s two square all this thing what is the first element of s in terms of x x n plus m plus j means first element is one x n plus m plus two will be the second element of s so if we in similar manner I can write it x n plus m plus j is equal to zero for j is equal to one two dot dot m so this I can write it this one what we did it we assign now I am writing it we assign that x n plus I is equal to mu I and I is equal to one two dot dot m agree that x n plus m plus I is equal to s I square and I is equal to one two dot dot m and what we assign it this one y z y n z x n plus m plus m twice m so I can write it x n plus twice m plus I is equal to y I I is equal to because we have a how many in general p equality constants agree so next we can write it that x n plus twice m for z plus p plus I is equal to z I is equal to I is equal to one two dot dot p so these are the variables we have defined in place of mu s y and z agree so our problem now boils down to solve a what is called linear equation that this linear equation we have to solve it so recall this equation recall this equation b into x is equal to d agree and this dimension of this I have just mentioned you earlier also this dimension is n plus m plus p this multiplied by that n plus twice m plus twice p this into one so this dimension is this into twice n plus this this dimension is n plus m plus p this is a p plus twice n and what is the dimension of that one n plus twice m plus twice p into one see what I have written it here explained it there same thing I am writing it in this case because it is a column vector that's one will be there agree so what is our d d dimension is n cross m cross sorry plus this cross one and this one we have to make bx is in such a way all right hand side of this one is positive quantity agree so now how to solve this one using that simplex method so first you say first you say that that our quadratic programming problem using KKD necessary condition we can convert bx is will be from linear set of equation now question is this linear set of equation we have to solve by what is called by simplex method we have introduced new variable in place of mu as y lambda is replaced by y and z that we have replaced by new variables x agree and then question is how to solve that one next question so this problem to solve to solve the to solve the problem using simplex method so we do not have any objective function here because only from the KKD necessary condition we got set of linear equation and we have to make in such a way that it is a standard LP problem form agree but still now it is not a standard LP problem I can make right hand side of the equation d always positive by some manipulations agree so our b into x plus p is equal to g so I have introduced some artificial variables which is a vector that the dimension of this one is same as dimension of d so this dimension is n cross n plus m plus p cross 1 agree so this dimension you know this dimension you know this dimension you know and this is this p is a vector which is vector is artificial variable agree whose elements are greater than equal to 0 artificial variable of dimension this p is a vector whose elements are the artificial variables and each element is greater than equal to 0 this so you can write it the p is nothing but our p 1 p 2 dot dot p n plus m plus p this one and this dimension of this one this dimension of this vector is that one n plus n plus m plus p cross 1 so now we have just introduced some artificial variable so that this is equal to the ultimate ultimate artificial variables value must be 0 this one so when we have introduced the artificial variable we know the sum of the artificial variables is our objective function so w is nothing but our objective p 1 plus p 2 plus dot dot p n plus m plus p element this is our objective function that objective function is called artificial artificial objective function then how will you get the function value in terms of x parameters that p 1 I can always write the first element of p 1 I can always write in terms of x and d elements so p 1 express in terms of take all take b x in the right hand side and element wise p 1 p 2 wise you equate and then you add p 1 p 2 all this things so ultimately this will get a function of all decision variables agree and this decision variable not only decision it is a decision variable then lagrangian multipliers mu lambda associated with this one n s with this one so you know how to solve this one let us call that one what is called prequation or one observation one must note that we know this we know mu into s s square is this equal to 0 agree so both this product of this one both product cannot be non negative that means if one is non basic variable other must be both the both the elements cannot be what is called basic variables that is if basic variable value is what is called non zero so this it is a non zero both the things then it cannot be product cannot be zero so one can see the product of these two things for j is equal to one on this both variables this one and this one cannot be a what is called both basic variables either one of these will must be a non basic variables in general so note that x n plus j and x n plus m plus j are not simultaneously the basic variables agree so the problem given in this one I can easily solve by using the simplex method if you just want to solve this one you can solve it in simplex method so in our case how many variables are there if you see in our case this one how many variables are there into that mu s mu s y z n twice m twice p n is 2 m is 1 2 2 m 2 plus 2 4 plus 2 6 so we have a six variables are there and six variables are there so if you just put the table form I can easily write it now x 1 x 2 x 3 x 4 x 5 x 6 agree this x 6 then we have a how many we have a what is called vector of artificial variable there are four artificial variables are there because we have a four equations are there set of linear equation artificial variable is four x 7 x 8 x 9 x 10 then b then then your ratio you write ratio the standard wire and our basic variables are what basic variables in this direction there are basic variables is x 7 x 8 x 9 x 10 and then our artificial artificial artificial objective function you can fill it up according to the our problem if you fill it up this one that I am just writing it this one to from the equations from the KKTA equations what you get it and we have written this equation if you see this equation this equation if you write it in terms of mu z mu in terms of x if you write it you will get it 2 0 1 0 1 minus 1 1 then 0 0 0 agree next is 0 2 1 0 minus 3 plus 3 0 1 0 0 and this b is 6 this is also 6 then next is 1 1 0 1 then 0 0 0 0 0 1 0 and last equation 1 minus 3 0 0 0 0 0 0 0 1 and that right hand side is 4 this right hand side is 1 and the cost function if you see minus 4 0 minus 2 minus 1 2 minus 2 0 0 0 0 0 0 and that is w so next class I will complete not complete I will explain how you have to proceed and get the solution of linear quadratic regulator programming problem using the simplex method that is basically from KKT condition we have converted into a linear set of equation for quadratic programming problem only so here I just stop this one