 So, we have discussed that how to solve the linear programming problems and quadratic programming problems by using the exterior sorry interior point method. And we have taken in one example to illustrate this technique you could not complete this problem. So, we recap this problem once again. So, we have given this is the function we have given which is the linear function subject to some linear constraints. This is the constant this is a constant and as well as x 1 and x 2 are greater than equal to 0. So, if you want to solve this problem by using the interior point method first step is convert this linear LP problem into a standard LP problem. What is the standard LP problem is there? So, inequality sign what is there you convert into a equality sign by introducing the new variable x 3 which is artificial variable and x 4 another artificial variable in the second equation. So, our problem is now to solve this linear equation agree that linear equation subject to equality constraints. And the right hand side of this equality constraints are all we have to make it all positive. And the all variables x 1 x 2 and the artificial variable x 3 and x 4 are greater than equal to 0. So, first step is convert to a standard LP problem. Next step we know what is how the way we have defined f x in the symbol c transpose x and equality sign this constant we have given a into x. So, according to this problem our a is this one b is the right hand side of the equality constraint right hand side of the equality 5 4 then c is the coefficient associate with the objective function. So, our problem is now to solve using a interior point method. So, we take a initial guess of let us call 3 comma 1 x 1 is 3 x 2 is 2, but that is not the if you see we have a additional 2 points are there x 3 and x 4. And accordingly if you take x 0 is the initial condition according to this 3 x 1 3 x 2 1 then x 3 I have considered 1 x 4 is 4 this is the initial condition. You have to choose the initial condition in such a way. So, that that point is interior region or the in the feasible region of this problem. So, how to check whether it is a in the feasible region or not. So, we have to check the a into x must be equal to b if this condition is satisfied then this initial guess is inside the feasible region or interior point of the region. So, a is this one multiplied by this vector you will get 5 and 4 and this implies that this is inside the region that means the feasible region. So, once you take this initial guess in the initial point interior point initial interior point then you see this point is not equidistant from the axis agree. So, what you have to do we have to change into a new variable by scaling the variables. So, new variable changes are made by using that one x 0 x superscript 0 means initial condition y superscript 0 the new variables and we have a new variables are 4 also and d is the scaling factor of this one or you can say y 0 is d inverse from this one y 0 is d inverse x 0. So, you scale the initial condition of x initial interior point in such a way in new coordinate variables or new decision variables this y 1 y 2 y 3 y 4 are equidistant from the axis. So, this is the choice of scaling this and the d choice with the whatever the initial point we have taken initial interior point and this d will be with that element in the diagonal form and if you multiply d inverse x 0 you will see this will be 1 this indicate that x y 1 of 0 y 2 of 0 y 3 of 0 y 4 of 0 all are equidistant from the axis that new variables. So, if you look at this our objective function with the interior point in original interior problem interior point then this value is minus 2 at this moment. So, our job is to now in transform coordinate axis to move and also you find out that in transform coordinate axis coordinate axis what is the objective function value and if you scaled that means transform means scaled LP problem f y is nothing but a c transpose x and x is what d into y. So, this nothing but a p transpose y and p is d c and subject to conditions our original problem is a into x is equal to b. We know x as a relation with the new variables with d into y. So, if you put x variable d into y that will be coming b into y a into d is equal to b we have considered that product. So, if you consider that in scaled LP problem mean optimum value of not optimum the value of the function at that initial interior point new initial interior point that value of the function will be equal to once again it is a 2 if you see this one that is 2 we have seen p transpose y is 2 and our initial if you see a d is what is nothing but a b I can easily compute b because I know a I know d. So, I can easily compute b. So, this is our b now once you know b that means I am now in new decision variables that means y from that decision variables when a interior point and that the y is y 0 of superscript 0 is the initial interior point in what is the new decision variable coordinate axis. So, if you now our job is I have to move in such a direction. So, that function value is decrease. So, that is our main goal to this one and that point that point what is y of 0 must be interior point of this problem interior point that means must be in the feasible region of the new decision variables. So, we know what is the direction of this one that we have derived the decision the direction of the vector is nothing but a minus t into p and t is the this this matrix is t and this t matrix is a projection matrix that we have discussed earlier multiplied by p and p you know this what is this p is nothing but a d c transpose d c transpose p transpose. So, p is nothing but a d into c. So, I can calculate p is nothing but a d into c this p equal to. So, once you know the b matrix b is what a into d that just now we have told it b is equal to a into d I know b I know p I know this is identity matrix with proper dimension I can compute the direction which direction I have to move it. So, this is the direction which direction I have to and this direction if you move it that will give you the what is called descent direction. That means function value in the new coordinate axis agree or new decision variables after scaling the L P problems that function value will decrease. In other words the original value problem if you go back to the original problem original value original optimization problem that function value will also decrease from its initial values. So, now we can compute this one if you see this one we can compute that one that is our this quantity by using the numerical figures of using the numerical figures of b and p. So, our first we have to compute t. So, t is equal to t is equal to I minus b transpose b into b transpose whole inverse b this is our t. So, this t will be equal to b value you know b b is what just now you compute it 1 3 1 1 0 then 0 1 0 3. So, put the value of b here matrix b here then you will get I am not computing in details and one can see this one because I is a symmetric matrix minus another symmetric matrix results is symmetric matrix. So, see this one b b transpose is a symmetric inverse of this one is a symmetric matrix then I am multiplied by both side b transpose b that is also symmetric matrix. So, results is coming this is a symmetric matrix. So, I am writing the results after putting the value of b in this expression I am writing this values are there 1 7 0 3 minus 0.477 minus 0.2752 then 0 8 26 then this since this matrix is symmetric this is same as that one. So, star means the corresponding elements of symmetry is same. So, now I am next is your 8.257 minus 0 8 26 then your minus 272752. So, now 1 3 3 1 elements are same since it is a symmetric matrix that this element will come here. So, I am not writing just star means the corresponding symmetric elements are same. So, next is then your 3 2 you see 3 2 position is same as 2 3 position that means this position will this numerical value will come here. So, it is this one then 3 3 position and your 3 3 position is 0.9083 then this values is 0.0275 agree. So, now fourth row is 1 4 this element is same as 4 1 because it is a symmetric matrix star then 2 4 4 2 elements is same as 2 4 that means this element is here this then fourth element is same as 3 4 element this element is same as this element. So, this now 4 4 element is your 0.0917. So, this matrix dimension is 4 by 4. So, our position matrix T is known to us. So, once you know to us I can immediately find out the direction whose direction in y coordinate axis I have to move. So, that the function value will reduce from the earlier value of the function value if you move in that direction. So, at this moment we know T. So, once you know the T immediately you can find out the what is our what is called our direction D is equal to our minus T into P. So, if you calculate that our direction of this one T you know. So, we are now at what is called initial interior point in transform axis. So, the initial point is jth and then from there which direction you have to move that is minus T into P and that not only that this is a feasible direction it also gives the what is called the descent direction. That means function value will also decrease both feasible and function value also decrease in this direction if you move. So, this value is T value is known this is known P value is your just I we have calculated P value is this 1 0 0 P is equal to D into C. If you see this the P or P is equal to nothing but a D into C D into C is 0 P D into C is equal to P this value I am writing. So, this if you calculate that one if you calculate this one it will come 0.7706 minus 1.5688 minus 1.740 3 1 then 0.5229. So, we know the descent direction agree. So, if you see the T you got it the T you got it and P you have calculated P how will you calculate P where P is just once again I am calculating P is equal to D into C that we have calculated and D you already know what is the D value you have considered. See this D value of consider this is the D value D value is that one that 3 1 1 1 4 by 4 matrix D into C is the P values that you have you have already calculated. So, I know the descent direction then what you have to do next our that if you see this our we are now y 0 initial interior point. So, we have to move in D direction agree we have to move in the direction and what up to what that means our y of superscript 1 is equal to y of 0 plus lambda into D 0 superscript D 0. So, in this direction in this direction we have to move it and which in turn the function value will decrease and this point also in the feasible region that we have proved it earlier. So, let us see that one how to decide the lambda max in the direction of this is the direction this is the direction lambda max if you move this one in this direction this is in this direction you have to move it agree. So, now I have to find out so D 0 you see D 0 is the is not only feasible, but also a descent direction means this if you move in the D 0 the point you will get it y of 0 it is the interior point agree as well as the function value also decrease for that one you have to choose lambda in such a way. So, that y of y of 1 will be in the inside the interior point. So, that operation or this things we are now doing and which is we already explained earlier. So, now you see new feasible new feasible point new feasible point y of 1 is computed as note in scaled LP problem where y is greater than equal to 0. So, now we will see how new is computed. So, if you see this our y of y of 1 is equal to y of 0 plus lambda y of 0 plus lambda then D of 0 agree. So, this lambda max of lambda is computed minimum of D j is equal to 0. Less than 0 D j means elements of D j means elements of D means how many elements of D is there if it is any elements are there D 1, D 2 dot dot d n the elements which are less than 0 that you compute like this way lambda max y j superscript 0 minus D j superscript 0 minus D j superscript 0. So, for this problem you see what we are getting it here mean D j superscript D j 0 D j superscript 0 you can write it if you write it superscript 0 do not write it. Then how many y 1 y 1 of superscript 0 minus D 1 of superscript 0 another point is y 2 superscript 0 minus D 2 superscript 0 then third one is y 3 superscript 0 then minus D 3 superscript 0 then minus D 3 superscript 0 and last one is y 4 that y has a 4 coordinates are there. When you convert the standard LP problem into a scaled LP problem by transformations then this equal to you D 4 superscript 0 minus. So, you compute this one minimum of lambda j superscript 0 less than 0 for less than 0 now you see D we have computed here. So, this is if you consider this is our D 1 superscript 0 this is our D 2 superscript 0 this is our D 3 superscript 0 this is our D 4 superscript 0. So, we will consider only the less than 0 D j superscript less than 0. So, this D 1 superscript 0 value is positive. So, this quantity positive the ratio is because y 1 0 is equal to 1 this is positive procedure with a minus sign. So, this is negative this is negative of that one. Now, this is dash means do not care when D j is positive you do not care this one D j D 1 0 is positive you do not care this indicates dash indicates do not care because we are interested about D j less than 0 and that explanation we have given you earlier that if it is now look at this one that if lambda e is a your if you lambda e is your negative quantity or positive quantity that we have to analyze this first. Let us we compute first that that one then this is equal to 1 divided by D 2 that D 2 value is minus minus and preceded with minus it will be plus plus 1.5688 then D 3 value you see minus minus and preceded with a D 3 is preceded with a minus. So, it will be a D 3 0 is 1 that will be a 0.7431 and last one is y 4 0 is 1 y 4 0 is 1 D 4 value is positive. So, results will be a negative one. So, it is do not care do not care. So, out of this which one is minimum you have to take into account as a lambda max. Now, see this one out of this is the maximum or a minimum because 1 by this quantity is minimum then small value than this one. So, our value is 0.637 lambda max value will choose 0.63 values and if you choose this value exactly we have seen earlier and explain that if you choose lambda max is this one then there is a you will get one of the element of this y of 1 is 0. It indicates that you are on the boundary of this region to avoid that one to avoid that one what we select this one whatever the lambda max you got it that you multiplied by a less than 1. Generally, you multiply this 0.9 it ensure that all the elements of new point of y 1 will lie inside the region I mean inside the region means it is in the interior point not only the boundary of the region this ensured. So, keeping this thing in mind then we can write set lambda is equal to 0.637 into 0.9 which will come 0.5733. So, this is the lambda will select for our problems once you select this one then what is our new interior point from y 0 to y 1 what is the new interior point. So, y 1 is equal to y of 0 y of 0 then plus lambda into d of 0 y of 0 we have scaled 1 1 all 1 1 element then plus lambda just now you got it 0.5733 0.5733 so this is 0.5733 into d 0 d 0 you got it 0.7760 minus 1.5688 minus 0.7431 0.5229. So, this is so if you compute this one then finally you will get 1.442 then you will get it 0.101 then you will get it 5.739 then 1.300. So, this is the new value of that that one. So, this is the new value of that that one. So, now see what is the function value previously we have seen if you look at this previous expression y of 0 that value you got it minus 2 if you see this one. Then if you compute this one this value you will get the value of that one is this value if you put it here this value is y what is the value of y of 1 that value you will get it point minus point 4 225. You see from this point to this point you moved it in proper direction y of 1 this is 1 sorry we have written y superscript 0 this is y superscript 1 and function value is reduced from minus 2 to minus 4.225 agree, but once again I have to go back to our original problems. So, we know how the new variables and original variables are they are related. So, now you can write it this one you can write it using you can write it using the scaling matrix matrix D. Then if you recollect what is this how they are a related x of 1 is equal to D of 1 either this one with bracket you can write it with bracket if you write it is the iteration indicates first iteration we have a. So, this is the variable decision variable in original problems that means x 1 x 2 x 3 x 4 we can compute once I know D and our D is if you recollect D is this 0 0 0 0 0 1 0 0 0 0 1 0 then 0 0 0 3 and this y 1 you know already what is the y 1 value put this value here. Then you will get finally 4.326 this is known this values is known 4.36 0.0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.5739 then 3.90 this is. So, it is a in the original problems we have with D operations that means with the matrix of scaling matrix D we back to our original problem the decision variable x of 1. So, previously x of 1 if x of 0 was if you see x of 0 was your that is 3 1 1 3 it was now it has moved to this point. So, let us see this function value in original variables function value this value is minus x 1 if you see this minus x 1 plus x 2 what is x 1 now value minus x 1 of superscript 1 plus x 2 of superscript 1 this superscript indicates the iteration. So, that value we got x 1 of this is our x 1 of superscript 1 this is x 2 superscript 1 this is x 3 superscript 1 and this is x 4 superscript 1 this superscript 1 this superscript if you like you can put it in bracket means it is a iteration it indicates the iteration first iteration after first iteration from x of superscript 0 to x 1 we moved. So, you put this value this is minus 4.326 plus 0.101. So, that value is 4.225 and that value previously if you look the expression x of 0 previously that value was 2. So, we have reduced to minus the function value is decreased from minus 2 to minus 4.225 that means with the choice of the descent direction we able to reduce the function value from minus 2 to this and you have to repeat this process because we have not reached the optimal value of this one. So, if you repeat this process once again agree. So, I just note that that the objective function value what I have just told you just writing this one is reduced from reduced to minus 4.225 from minus 2 after 1 iteration. So, next is repeat this process then what you have to repeat this one we know our now we will consider this x of superscript 1. If you consider this x of 1 bracket this value you know it what is the value you got it just now you have calculated these values. So, I am writing these values are there 4.326 4.101 then 0.5739 0.3.9. So, this is our you can treat at this is our initial guess. So, these are not equidistance from the what is called and this is obviously this is an interior point. If you want to check it you can check it a x of 1 because you have solved this way only interior point this equal to it will be a x superscript of 1 this will be equal to you that you see this one. So, our basic equation if you see x superscript 1 and this x you will do the x 1 and x 2 terms or if you write it this you will get it x superscript 1 the values are 5 and 4. I just show you that will be a 5 and 4 see this one this is our after converting the standard by x this and this if you put the value of x 1 x 2 x 3 here you should get it 5 because it is the interior point if you get it 5 and this equation if you get it 4 it indicates you are inside the feasible region. So, you can one can check that this will be a 5 4 check. So, now what will be this is our initial guess so what you have to do they are not equidistance from the x and z what you have to do we have to scale same procedure. So, we have to do y of that is new value of y 2 is x 2 we got it y 1 then you just got y 2 is again d into your is x of y 1 sorry x of this equal to this x of y 1 is equal to that y of this values. So, using this one choice of d is same as the initial values of this one diagonal elements will be that one. So, next step is repeat the procedure. So, repeat the procedure to obtain the optimum value of the function. So, you can you do another iteration you will see the minimum value of the function minimum value of the function f of x is equal to minus 5 for x is equal to 5 0 0 4 transpose and physically also physically also you can see this that you will get the value of the function is minus 5 from the statement of the problem. Let us see the statement of the problem what is this if you see this is the our objective function this is our objective function in our original problem and this is our the constraint on the problem. So, if you draw this equations and objective function in a graph paper you will see that our problem is like this way. So, when x 1 is greater than equal to 0 x 2 is greater than 0 x 1 greater than 0 this is our x 1 this is our x 2 x 1 greater than 0 means it indicates this region and when x 2 is greater than 0 this indicates that that region. That means we are in the first quadrant of this coordinate axis. So, our first equation x 2 is equal to you see the first iteration x 1 plus x 2 is less than equal to 5 that we can represent by let us call this is 5 and this is we call this is 5. So, that is the equation and this is our region and that is your x 1 plus x 2 is equal to 5 and less than this one this is our region. The next equation is x 2 is equal to less than 4 that means the next 2 is that region is the this is the x 2 is equal to 4. So, this constraint truth all constraints we have represented here again this is origin 0 0 then what is the objective function f of x is equal to minus x 1 plus x 2. So, this is we can write this is our objective function value is 0 when it is passing through the origin this is the equation of a straight line. So, this is our objective function that x 1 minus x 1 plus x 2 is equal to 0. Now, we can see visualize this one our feasible region is if you see look carefully our feasible region is that red shaded portion the red shaded portion or the boundaries of this one is that one. Now, this is the objective function f x whose values is 0 when it is passing through the origin. So, if you move parallel to this one now you see it is y is equal to m x plus c and c is minus that means objective function value is decreasing and you can move parallel with this one this line parallel with this line of course, otherwise the slope will change and objective function description will be changed. So, you can move parallel to this one up to that one. So, at this point this is the feasible point of this one. So, at this point what is the function value of that. So, this point is our x 1 is equal to 5 x 2 is equal to 0. So, our function value at this point f of x at x 1 is equal to 5 and x 2 is equal to 0 and that will give you the maximum what is the minimum value of the function. If you put this one minus x 1 plus x 2 is equal to minus 5 that graphically also one can see this is the problem. So, one can draw a conclusion like this way in a linear programming problem if you use the interior point method that point initial guess you must have to take inside the feasible region and when you do the iteration slowly it is approaching to the one of the vertices of the boundaries it will give you the optimum value of this function. In this case this vertices will give you the optimum value of the function. So, at this moment we know how to solve the linear programming problem as well as the what is called linear problem problem how to solve using the what is called the interior point method. So, next topics what will cover it this one that is the basic concept of what is called first will say how to solve the non-linear programming problem using the exterior penalty function again then you will see how to solve the same problem non-linear problem using interior point method. So, first is solution of so that portion whatever leftover was there last class we have completed the solution of the problem solution of non-linear programming problem using exterior penalty function method. So, in linear what is the interior point method we have seen if it is a we have solved to like to solve the problems of linear programming problems using interior point method you have to take initial condition inside the feasible region means interior point then do the scaling go in the proper direction. So, the in the scaled LP problem function value is decreased from the initial values and go back to your what is called the original problems and if you repeat this process ultimately you will see the your optimal point will be one of the vertices of the LP problems agree. So, that so this is our next problem that non-linear programming problem how to solve the exterior penalty function method the constraint let us call what is the basic statement of the problem let us consider non-linear problem of the following type. So, if you have a non-linear problem and constraints also is either linear or non-linear first you convert that reformulate the problem into a unconstrained what is called minimization problems UMP unconstrained minimization problems that we have already discussed earlier, but only the point is your initial guess must be the outside the feasible region. And then once you convert into a unconstrained optimization problem that problem you can solve it by using what is called our standard technique what we have discussed earlier either what is called steepest descent method or you can solve the Newton Raphson method or you can do the what is called conjugate gradient method the advantage of each method is discussed at length earlier. So, once you have a non-linear programming method you want to solve this by what is called exterior penalty function method what you have to do convert this problem into a what is standard what is called unconstrained optimization problem then you can solve it either analytically or numerical methods by applying the Newton Raphson method steepest descent method or the conjugate gradient method. So, the our function is like this way minimize f of x this function may be non-linear may be linear constant function may be what is called non-linear also. So, such that a subject to such that g j of x is less than equal to 0 agree and j is equal to 1 2 dot m and x below the dimension of x is n dimensional things. So, what you have to do it first that I told you first you convert into a unconstrained optimization problems. So, first you convert into a unconstrained optimization problems. So, this unconstrained optimization problem is obtained by adding a penalty function or term that measures the distance that measures the distance that is the what is called additional term what we are adding to the objective function agree. So, this additional terms you have to give weightage with a some factor agree. So, the our construct next is construct the penalty function as p stands for penalty function x is the decision variables of the optimization problems and tau is the scalar quantity agree that equal to f of x plus tau of k summation of this all inequality constraint I converted into a into this form j is equal to 1 to m g j of x whole square into u j g of g j of x g j of x and that bracket closed this is equation number 1. So, this tau of k is called penalty coefficient at that k indicate at kth iteration at kth iteration agree. So, what is the this is the our objective function is given the constraint is given this objective and constraint are non-linear and then construct the penalty function like this way objective function plus this now I will discuss what is this one that if the our initial point which is the exterior point that outside the feasible region then you penalize heavily that means you give punishment on the objective that constant function. If it is inside this one then you give weightage 0 of this one let us call how to do this one. So, and tau k plus 1 is greater than tau k that kth iteration value what is the value of tau k we have considered k plus 1 is the next iteration the value of tau k must be greater than at kth iteration that is the things then you u j g j g j of x this value is a function of that is the following function what is this I define this one u j g j of x is equal to 0 if g j of x is less than equal to 0 now see this our statement of the problem if you see the state plane of the problem this is the constraint this is the constraint and if this constraint if this constraint are satisfied that means our point is interior point this constraint is many inside the feasible region and if this constraint is not satisfied then we are exterior point means we are not in feasible region outside the feasible region. So, you penalize with one so you can see this is our in feasible region this is feasible region and this is infeasible region infeasible region. So, now what was the function now if you just look at this expression the equation number one if you consider this is the equation number one. So, equation number one can be written as equation one can be rewritten as p x tau of k is equal to f of x plus tau k summation of j is equal to 1 to m max g j x minus 0 this then whole square. So, let us call this is equation number 2 this is equation number 2 how I have written it this one see this one when it is the infeasible region that means g j of x less than 0 then this quantity value is 0 this quantity. So, this will be 0 whole part will not be there when it is the outside the feasible region this what I this values is one that means this is g j x whole square into tau k. So, that what will write it now you see maximum of this equivalently you can write it maximum of this. So, when it is not in the feasible region this value is greater than 0 is not greater than 0 then this what is the value is there then square of this one this is same as that one. So, this will be positive maximum of this one it means g j you are taking agree g j what is the maximum then that square. So, we can write it this one if the constraint are not satisfied and not satisfied then then they are penalized. So, this equation I can write at f x into tau k into f of x that whole thing I am writing this thing I am writing f of x and this you know this portion summation of this one. So, where I can write max of g j x 0 is equal to 1 this value because when it satisfy the our constraints that means this value is less than 0 maximum of this one is 0 out of this because this value will be 0 minus negative quantity when this g of x is less than that mean x is in the feasible region. That means if g j of x is less than equal to 0 and that is equal to 1 this value will be if g j x value this value will be what is when it is greater than 0 when this value is greater than 0 this will be a 1. Now this will be your g j of x again maximum of that one because that value is a greater than this. So, it will be g j of this. So, this is the equation number 3. So, we will explain this thing will explain with an example let us call our problem is minimize f of x is equal to x cube then subject to g 1 of x is equal to 2 minus x is less than equal to 0 and g 2 of x is x minus 5 is less than equal to 5 less than equal to this 2 equation commonly you can see it is nothing but x is less than 5 and 2. So, it is you say 2 minus x if you bring it to here is less than 0 this one and x minus 5 is less than this 2 things I combine together and what is the minimum value of function at what point. Obviously, you see x is equal to 2 it has a minimum value of the function if you plot this one x is equal to 0 then steeply it is increasing like this way and that is what will be that one x minus that will be like this way steeply it is just cubic from x this is f of x. So, this function value because this range is given x is equal to 2 to 5. So, it is minimum value of this one is here only. So, I will discuss this thing next class that physical interpretation of what is called penalty function while we will solve the x here solve the non-linear problem using the x here point method. So, I will stop it here today.