 the last class we have discussed the problem formulation of quadratic optimization problem using the simplex method if you recall we have this is the quadratic function our problem is to minimize f of x which is in quadratic form and we have a inequality constraint and equality constraints but this inequality constraint and inequality constraint they are affine functions so we have formulated this thing into standard lagrangian function inequality constraint we have converted into a equality constraint then it is what is called lagrangian function is formed then mu lambda is your lagrange multiplier which is associated with the equality constraint and that equality constraint that lambda associated with the equality constraint and the lambda value is not restricted that its value can be positive and negative on zero whereas this factor is associated with the inequality constraints and this lagrange multiplier corresponding to inequality constraint we consider as a mu that is greater than equal to zero when this type of inequality constraint we have that is we have discussed earlier in details that why the mu value should be greater than equal to zero so if you now write the what is called KKT necessary condition we have shown that these are the equation we will get it equation number two three four and five so this equation we can write into matrix and vector form which will be equal to into this form that first is x variable then we have considered mu lagrangian multiplier associated with the inequality constraint then lagrangian multiplier equal to what is called that next is variable we have considered the s associated with the that is your select variables then we have considered lambda this tooth are lambda lambda is since unsigned we have split up into a two positive quantities so this equation equation number two three four and five if you write in matrix form equation number two three four if you write in equation number matrix and vector form it will come into this structure be this whole matrix I am getting b into x x is the all variables lagrangian multipliers and the your decision variables and right hand side is the d we have defined is d so we have to solve this one now this problem is simply a what is called algebraic equation bx is equal to d so this problem we are we cannot solve directly what we consider introduce we introduce what is called a artificial variable along with this one bx is d we have consider artificial variable b p and p how many this is a vector of dimension n cross m plus p into one so there are m plus n plus m plus p variables are there so this artificial variable all elements all elements of each artificial variables which is a p is a vector we add together and that w is called artificial cost function so look at this expression so our problem is minimize this artificial objective function subject to the constraints what is the constraints we have we have constraints these equations which is converted into a standard LP problem is a standard LP problem our problem is minimize this artificial objective subject to this objective function now we know this b matrix if you now I write in tabular form to solve in discipline simplest method you can see carefully that how I am form this matrices so the equation number 2 the equation number 2 we have taken a simple example we have seen where h and c a b e and all these things are see all these things are described in the examples so I am just writing the equation number 2 this equation number 2 this one which in turn it will come this and this equation so now this is our one equation number 2 this is we got from equation number 3 and this is we got from equation number 4 with each equation there is a n artificial variable with this equation p 1 is there with this equation p 2 is there with this equation p 3 is there with this equation p 4 is there if you write in tabular form now it will be like this see x 1 x 2 our the 2 original decision variable then x 3 is corresponding to our mu then x 4 is corresponding to our artificial variables then x 5 is corresponding to x 5 is corresponding to y and x 6 corresponding to z and y n z commonly they form is a lagrange multiplier associated with the equality constraints so remaining x 7 x 8 x 9 x 10 are the what is called artificial variable introduced in the system equation that is b x plus p this one so this s is a our select variables agree I mentioned earlier artificial is a select variable x 4 so now how is from this table you see 2 x 1 coefficient of x 1 is 2 I have written 2 then you see there is no coefficient of x 2 that x 2 is 0 then this is we have denoted is x 7 what is called that y is we have denoted in x 5 so x 5 coefficient is 1 so x 5 coefficient is 1 then z is x 6 we have defined x is that coefficient is minus 1 I have written minus 1 then mu mu is the mu that variable is defined at x 3 that coefficient is 1 see x 3 is 1 and remaining coefficient is 0 except the what is called artificial but there is artificial variable p 1 you can see from this expression that p 1 so we have written this at today x 7 is 1 and it is equal to 6 that is the first equation we have written and similarly second equation you can write it which you will get it this equation and that artificial variable is x 8 coefficient is x 8 is 1 and from equation 3 you see x 1 x 1 plus x 2 plus x s is defined by a variable x 4 so it will be x 1 coefficient 1 x 2 coefficient 1 x 4 coefficient 1 see x 1 coefficient 1 x 2 coefficient 1 x 4 coefficient 1 and this is is a third equation then p 3 will come and p 3 coefficient is p 3 corresponding to x 9 is 1 and this equal to 4 that 4 similarly last equation one can write it is nothing but x 1 minus 3 x 2 plus a artificial variable that will p 4 which is we define a new variable x 10 so it will be you see this 1 then minus 3 and then 10th position is equal to 1 so this now how you get that artificial variables that is the artificial variable now what is the artificial variable cost function that is we have written sum of p 1 p 2 up to p n in our case up to p 4 so if you just p 1 p 2 p 3 p 4 and what is p 1 p 1 from this equation p 1 is nothing but a 6 minus if you take that side right inside 2 x 1 minus x 3 minus x 5 plus x 6 agree is equal to x 7 and x 7 is nothing but a p 1 similarly p 2 is equal to x 8 is equal to p 2 I can write is 6 minus 2 x 2 minus x 3 plus 3 x 5 minus 3 x 6 if you take that side is equal to p 2 so if you similarly I can write p 3 and p 4 if you add all these things I will get this one so the artificial objective function I can write it now 4 x 1 minus 4 then 0 there is no x 2 x 3 minus 2 x 4 coefficient minus 1 x 5 coefficient 2 x 6 coefficient minus 2 is equal to that is we have denoted w so w we have written so now we have to follow our standard simplex technique to minimize this objective artificial objective function w equal to 0 that is how it is so next step is what you find out the pivot column this you from the you just look the what is the maximum negative coefficient associate in the artificial objective function here so this column is the pivot column and what is the pivot row you divide this coefficient along this column the 2 divided by 6 divided by 2 is this is 3 you can write it this is 3 then this is 0 ignore this is 4 4 divided by 1 that 1 divided by 1 this 1 so this is the minimum ratio you got it so this is the word pivot element and this is the word pivot row so this way we have identified pivot row then after that you know that x 1 will be entering as a basic variable and x 10 will live as a non basic variables and then procedure whatever we have discussed earlier repeat for few iterations then you will get the final solution of this problem that means the initially problem is given if you see the initially problem is given a quadratic optimization problems to solve that by using what is called simplex method that is this quadratic objective function is quadratic and what is our that our inequality constant inequality constant and affine function this so this is called quadratic optimization problem that can be solved by using what is called simplex method before that you have to convert what is called KKT necessary condition by converting into lagrangian functions so this is we have discussed last class after this so please solve this problem complete this problem you will see after few iteration the solution is given x after 4 iteration see what is the value of x 1 is coming this is 13 by 4 x 2 is coming 3 by 4 then x 3 is coming 3 by 4 that way so our basic importance variables are x 1 and x 2 so this solution of you will get optimum value of the function at x 1 when it is 13 by 4 and x 2 3 by 4 so today we will start that is the solution of what is called optimization problem using interior point method so our next topic is interior point method interior point method for solving optimization problems so interior point method can be used for solving the linear programming problem first it can be interior point method can be used for linear standard linear programming problem linear standard programming problems and also we can solve the convex optimization problems convex optimization problems that includes that includes inequality and equality constant that includes equality and inequality constant so you through interior point method we can solve two class of problems one is linear problems linear standard LP problem another is converted to optimization problems along with the inequality and equality constant standard LP problem may be equality and inequality constant also there in relation to this so let us see that what is our standard LP problem standard LP problem recall our standard LP problem is minimization f of x whose dimension and decision variable is we written c transpose of x and subject to the standard LP problem I am writing if you can recollect this thing that what we have discussed earlier a x is equal to b b dimension is n m cross n this agree and b is greater than equal to 0 and x is greater than equal to 0 and we define now feasible we have to find out the solution of the solution of this problem such that the function below is minimum not only minimum is must satisfy this equation that means that point at which the function below will be minimum that point must be inside the feasible region then what is feasible region the feasible region as we define like this way define the feasible region f f is the feasible let us call this is the feasible region that f we are denoted this is the feasible region feasible region f is denoted by x any point in this feasible x it must satisfy all equality condition of standard LP problem standard LP problem that is a x is equal to b it must satisfy for x greater than equal to 0 so this if for any below of x in this region if it is satisfy this one then f is called that feasible region of the corresponding problem so simply definition is like this way x whose dimension n cross n is called an interior point of f f is what feasible region x is called interior point of f this means if any point in the feasible any point inside the feasible region is there if consider that point is the interior point of f f if x superscript this belongs to f and not only this every component of x must be greater than equal to 0 and also x j superscript 0 means any point in the initial point is must be greater than equal to 0 for j is equal to 1 2 dot n so this is the definition of interior point so if you have a feasible region if the point is belongs to this in the feasible region and it satisfy that the all values of x is greater than equal to 0 then this point is called interior point of the feasible region so let us see that how one can solve the this type of problems whether it is a linear programming problem or it is a convex optimization problem what is the basic steps are there to solve this problem so our algorithmic steps algorithmic steps are like this way so what is the algorithmic steps first you find out the interior point of the feasible region of the problem then you move from this point in a such a direction so that the function value the objective function value decreases agree that objective function value is decreases that means we are in the interior point from there we move in such a direction so that function value is decreases and then test it whether it this point has reached to the optimal value of the function or not if it is if does not reach to the optimum value of the function then move to another point so that the function value is decreases from the previous value of the function and in this way we can reach to the optimum value of the function starting from the initial starting point should be inside the what is called interior point so our algorithm steps first first step is find an initial find an initial feasible point solution agree to begin the iteration process is first step and next step step to generate a new point next is generate a new feasible interior point that will give lower objective function value that means from the initial point you move in such a direction so that function value is decreased step 3 test it for optimality test the new point optimality if it is not optimal if it is not optimal move to the if it is not optimal repeat step 2 until optimality is reached modality is reached so this is the 3 basic steps where we will adopt for interior point methods so let us see that how to find out the what is called the feasible direction so next is definition a direction d whose dimension is same as the feasible decision variables a direction d whose dimension n cross n is a descent direction descent direction if moving along the direction if moving along the direction decreases the function values function value of the objective function that we have discussed in details when we have discussed the how to find out the optimum value of the function by using steepest descent method if you recollect our earlier discussion so let us see this one suppose we are in the feasible region interior point of the feasible region and we are here let us call first that is x superscript 0 we are here and we move in such a direction so that the function value objective function value is decreases from the previous value of the function so let us call this is our direction that if you move in this direction then this function value will decrease so parallel to this one we are here now parallel to this one so this is this is this one parallel to this one I moved in this way from initial from this state I moved like this way so this is our we can say it is a lambda into d where lambda is greater than equal to 0 greater than equal to 0 if it is 0 then this is this point so now what is our new vector of x this is our new vector and that is our x so we moved from initial vector which is in the feasible region in such a direction so that the new vector new decision variable with the help of new decision variable the function value should decrease that is our basic so this is our x one let us call for two dimensional case two decision variable case that can be extended for n dimensional decision variable agree note now see this one our objective function value is what objective function value objective function fx is equal to c transpose of x and let d is the our decision direction that d is the decision direction and in this direction if you move from this initial function if you move in this direction agree and we are if you are here then our vector is x is here so our we can write it c transpose x must be equal to c transpose of our initial vector initial decision values of this vector x then we are moving in the right direction to minimize the functions so that is the direction what is called decision direction and from this one we parallel to this one we drawn this this one now we can write it c transpose x is what this vector plus this vector so x of superscript x of zero plus lambda into d and d is a vector of dimension n cross one is equal to less than c transpose x of zero and then c transpose x of zero plus lambda is a scalar quantity we can take it out and we can write c d is equal to c transpose x zero so this this itself since lambda is since lambda is greater than zero greater than equal to zero so we can write that c transpose d is equal to zero that also we have discussed in earlier in decision decision method now what's your conclusion which direction will move if you can move in this direction and this direction how you decide if you decide d is equal to minus c then from equation one that means this condition indicates this condition indicates that our function value will decrease if you move from x to x zero to in this direction agree and this will be function value will decrease and this condition much satisfied and when this condition much satisfied with this one sorry this is greater than equal to less than equal to when this condition will be satisfied if you one of the choice of d is if you consider d is equal to minus c then from equation one we can write it from one we can write it c transpose into d is minus c is equal to let us see what that is minus c transpose c this is nothing but a norm of that c vector and square and this quantity if you see this quantity always greater than zero greater than equal to zero okay so this indicates this quantity since it is minus sign is there minus sign this is preceded to minus sign that whole quantity is less than zero this quantity so this you can write it less than equal to zero so this is the condition that one of the choice of decision direction is will be with minus sign of the cost function coefficients so now we have and if this is the direction and this axis belongs to this feasible region if you move in this direction and that final vector is x if it is belongs to what is then feasible region this new point this is the new point it must satisfy our equality constraint if you see this must satisfy our equality constraint agree is equal to b let us see what is this one x is what x zero plus lambda d is equal to b and this x zero plus lambda into a d is equal to b and this quantity since x zero is belongs to the feasible region means interior point of the feasible region this quantity is b so therefore this condition is a into d is equal to zero since lambda is greater than equal to zero lambda since moving from this point to some point then lambda quantity is positive you can omit zero also if you like so this so this is the condition so so a and d must be zero this is the conditions we got it so remarks what is the remarks we can write it for this one if f f d d is the decision direction again is a feasible is a feasible direction it must satisfy it must satisfy a d is equal to zero if d is a feasible direction that it must satisfy feasible direction means what feasible descent direction you can write it feasible descent direction means that the function value will decrease if you move in this directions and it must satisfy the our equality constraints also agree so satisfy this one and not only this the x which is equal to x zero plus lambda d must be greater than zero to satisfy the non negativity of the decision variables this is a non negativity non negativity restrictions on the decision variables in short if d is the descent direction then it must satisfy a into d is equal to zero this is the what is called equality constraint must satisfy let us take one simple point and see the effect of interior point then how to choose the interior point of this one example this is we have given till now some basic definition of this one let us call minimize f of x is equal to minus x one plus x two agree which you can write c transpose minus one one x one x two which is nothing but c transpose x x dimension two cross one subject to x one plus x two is greater than less than equal to five then x two is less than equal to four and x one and x two is greater than equal to zero this is our problems of this one now let us see if we represent this one in graphically what it looks like this is easy to represent first we have a coordinate axis x one and this is x two and this since x one is greater than equal to zero then this indicates the upper half of this from this line since x two x one is greater than equal to zero this means this part of this one will be our feasible region and another constraint is that x two is less than four that x two suppose this is four this is two two four so that should be less than equal to this this now we have a x one plus x two is less than equal to five so when x one is zero x two is equal to five let us call this is the five some five point five and similarly when x two is zero x one is five so let us call this is two four two four this is five so you draw this sorry you draw this line agree and this region below this region is the feasible region so our effective feasible region is this portion this from here let us take we take there are two points are there inside the feasible region interior point we take two points let us call one is let us call we take the one point five and one point five this coordinate is let us call one point five and one point five sorry it is here somewhere not here this point this point this is this is one one point five let us call this coordinate is one point five and this coordinate is one point five here then another coordinate you consider that three one let us call this is a two this is a three and one this is two our case the one will be somewhere here so this coordinate is three one so this point we will call the point b and this point we will call the point a this point agree so there are two points are there and we have considered inside the interior region the interior point now we can say that point a is more centered point a is more centered than the point b we can say point a is more centered than the point b and point b coordinates are three one and point a coordinates are one point five and one point five more centered in the means this point a point is equidistance from the coordinate axis this is one is one point five from here also one point five equidistance and this here of centered you can say this point b is more of centered compared to a so now clearly if you see our objective function is what if you see the our objective function is this one this function we have to minimize agree when fx this equal to zero money object function value is zero we can write x one is x two is equal to x one that means our objective function expression is that one with a forty five slope is forty five and this is the our objective function value fx which is given minus x one plus x two is equal to zero this one and it is passing through the origin now you see this objective function line if you move in this side the function value is decreasing decreasing decreasing decreasing and at most you can go at this point when it is crossing this one at this way it is a function value is minimum agree this is graphically one can see that one so if you move in this direction the function value is increasing and from this one if function value is decreasing now if this point a is the our you can say starting point of our iteration of internal point point this then which direction we have to move it just now we have proved it that choice of d must be equal to choice of d money what is called direction vector will be minus c and what is c just say c is our case is minus one and one is three so with minus sign keeping with multiplied by with minus sign one whatever the vector you will get in that direction you move it let us call c is what that c the what is called our objective function this is the c transpose is minus one plus one so take that c c is minus one plus one this agree so our c is minus one in the directions and plus one it is a two plus one somewhere here so this so this is the c vector agree so what the vector will be which one if you see our D vector our D vector is opposite to this tent so this is our D agree so which direction from this point which direction parallel to this direction you have to move it parallel to this direction from this point a have to move parallel to this direction and this direction is in the in this direction so this if you move in this direction the function value is decrease and physically also or graphically also you can see if you just move in this direction you will get optimum value of the function at this point let us call this point is c you will get this and here if from a point you have to move in this direction let us call if you are it at b point you have to move from this point parallel to the d in the directions so this point and difference between this point and this point if you move in this direction it will hit the wall at some iterations so you can say this one that is if both the cases function value is decreasing no doubt so we can make a remarks like this way it is clear that affects function value decreases more decreases substantially substantially by moving along the steepest along the steepest descent direction direction from point a other and the other end point b the off center point point b is the off center point will take us towards the wall again of the feasible region the feasible region before much improvement can be made before much improvement can be made so now question is how to centered because our initial guess may be anywhere in the feasible region now question is how to centered the off center point in a feasible region the simplest way is do some transformation on the original decision variables and that will make that will transfer the original point into a what is called transform point which will be the center point center point means from the axis from the axis is the distance says will be same so next our point is how to how to center and interior point suppose in this case suppose we are at this point now this point how to make it centered so you have to do the initial variable x 1 x 2 some transformation which will convert into new variable in new variable will get the center point from the x some from the center point from the both axis is the point will be in the transform axis the point will be equidistant from the coordinate axis transform systems so let us call we take in one example and explain what is the meaning of this one so let us call we consider the our point b point which is off centered the point b is off centered point suppose this week take it so if you take it now we do transformation like this way after that transformation you will get new coordinate axis and that point should be equidistant from the coordinate transform coordinate axis so we define new coordinate axis y 1 is equal to the old value of x 1 divided by 3 that then the coordinate y 2 is a x 2 divided by 1 now you see whatever the this coordinate axis I got it the x 1 of 0 x 1 of 0 divided by 3 x 2 of 0 divided by 1 this what will be so it is better to write x superscript 0 0 than 0 this is the transformation we made initial point which turns since it is the off centered point that point is not a equidistant from the axis 1 is 3 x axis is distance from y axis distance is 3 from the x axis distance is 1 so it is off centered point so we have made it this one which equivalently we can write into matrix form y 0 y 1 of 0 y 2 of 0 this equal to write it that one is what we get 1 by 3 1 by 3 that is 1 0 0 multiplied by x 1 0 x 2 0 again now you see if you substitute this is the transformation so what is this is coming if you do the x 1 of 0 value is what 3 the x 2 of 0 value is 1 then if you this into this this value is 1 and this is 1 so now see this one the transform coordinate axis the point y 0 point y 0 point is equidistant from the transform coordinate axis 1 1 is so now see the our in transform system what is our what is called our optimization problem we will see now substituting the values of x and x 0 agree in terms of y 0 by substituting minimize the function which will be in terms of y is equal to minus x 1 plus x 2 x 1 is what just say x 1 is 3 x 1 is 3 y 1 3 y 1 agree in general now if I write it in general this is the 3 y 1 then x 2 value is y 2 so this which you can write it this is minus the 3 1 this is minus 3 1 so I can write minus 3 1 y 1 y 2 and corresponding to this this value this point x 0 that is it will be 3 y 1 0 y 2 0 agree in general this is the transformation we made it this one then subject to our is x 1 plus x 2 is less than equal to 5 which we can write it x 1 is what see this one relationship between this x 1 is 3 x 2 in general that relation is what y 1 is equal to x 1 by 3 and that corresponding point we can write it this one agree so this is that one and we can write 3 y 1 plus y 2 is less than equal to 5 so now it is converted this is in transform coordinate x is our problem in minimization f of y what is f of minus 3 y 1 plus y 2 and subject to this condition another constraints is there what is called x 2 less than equal to 4 which equal to nothing but a y 2 is less than equal to 4 and then y 1 and y 2 is greater than equal to 0 so if you transform coordinate x is we can say minimize f of y what is objective minus 3 y 1 plus y 2 subject to constraint inequality constraint 3 y 1 plus y 2 is less than equal to 5 another equality constraint is y 2 is equal to less than equal to 4 and y 1 y 2 are greater than 0 since y it is greater than 0 since x 1 is x 1 and x 2 greater than equal to 0 x 1 x 2 greater than equal to 0 that is why y 1 y 2 is greater than so now you see if you just read this optimization problem in what is called graphical graphically if you plot the optimization problem this looks like this and you will see that point in coordinate new coordinate system that point is in the center point so I am now plotting that one you see this constraint this is the one constraint this is another constraint 2 and this constraint so our new variables in transform variables are y 1 y 1 y 2 and y 1 greater than equal to 0 that means it indicates it is this side and y 2 is greater than equal to this indicate this is that side and we have another constraint in transform coordinate axis y 2 is less than equal to 4 suppose this is 4 1 2 3 4 suppose this is 4 this is 2 so this is one thing is that one and our inequality constraint you see this one 3 y 1 plus y 2 is equal to 5 let us call y 1 is 0 y 2 is 5 y 2 is 5 here and y 2 is 0 then y 1 is 5 by 3 that mean 1.66 so this is 1 1 2 then it is a 4 this is 1 1.6 may be here so our that constraint is that one so this so our feasible region is that this shaded blue shaded portion this is the problem and our what is our initial interior point 1 1 so this is 1 and that is 1 so this is our initial point in transform journey initial point that is 1 1 which is a centered point this is a centered point so in general now we will show how to convert if it is if the point is of center point how to convert into a center point in transform coordinate axis that we will discuss next class in details this one how to convert into that portion so we will stop it here now.