 we have discussed how to solve the non-linear programming problem optimization problem using the interior point method. That means, what are the algorithm states are involved here, first you take a point in inside the feasible region that is we call the interior point. And then you write the what is called the necessary condition and this necessary condition whatever you will get it depends on the number of decision variable involves in the optimization problem. If the decision variables are n then we have n such type of equation necessary condition equation set of equation which may be in linear or non-linear depending upon the objective function and the constants. And next step is you have to solve this set of equation which is obtained from the necessary condition by either analytically or iterative method or you have to solve by numerical methods. That we have mentioned it if you solve in analytical method express variables, if you want to express the variables x 1, x 2 in terms of the penalty coefficient it is very tough for higher decision variable, higher number of decision variables. So, after solving this one you check the what is called optimality condition. This is the optimality condition if the function value does not change and it is less than the epsilon then you can stop or you check tau k f of k is less than equal to epsilon tau k is greater than 0 and f of x k is also greater than 0 the way we have defined f k. Once this is satisfy you can stop that means we have reached to the optimal solution using the interior point method. That means it is approaching to the boundaries of the problem feasible region which is near to the boundary of the problems. Next if it does not satisfy this condition then go for next iteration by assigning the penalty coefficient tau k plus 1 is equal to c into t k and c is less than 1. If the initial value of c is selected some then you multiply it by for say 0.1 you reduce the penalty coefficient in turn you are approaching towards the boundary of the feasible region. Then you increment the assign k u k plus 1 and go to the step 2 and repeat the process to obtain the optimal solution of non-linear programming problem using interior point method. Let us taken one example and see how one can solve this using or adopting this procedure. So, our problem is solve example solve the optimization problem problem as given below minimize f of x which is 4 by 3 x 1 plus 1 whole cube plus 4 x 2 which is 4 by 3 x 1 plus 1 whole cube plus 4 x 2 which you can write it 4. Then you can write it this what is called 4 if you take it common then it is one third x 1 plus 1 whole cube plus x 2 is equal to this. This is our optimum a minimum value of this minimum we have to optimize this one subject to g 1 of x you have a 2 minus 2 x 1 which is less than equal to 0 and g 2 of x is equal to minus 2 x 2 is less than equal to 0. So, from this equation we can say the x 2 this indicates the feasible to be x 2 is greater than equal to 0 and x 1 here from this equation we can say in that x 1 is greater than equal to 1. So, this is our that region feasible region of this one if you plot it this one if a solution you can say if you plot this one that our x 1 is in this direction x 2 if you write it in this direction then our feasible region is that part of this one here is the feasible region all this thing that may x 2 is greater than 0 the upper part and x 1 is greater than 1 greater than equal to 1. So, this is our feasible region now see the values of this again. So, now we according to our this you find out that our solution is penalty function x tau of k what is this the objective function plus tau k summation of j is equal to there are inequality condition two inequality conditions are there at j of x agree this. So, which we can write for our example this one it is a x 1 third x 1 plus 1 whole cube plus x 2 this is our f of x is nothing but a f of x plus tau k i is j is equal to 1 j is equal to 1 is minus 1 and g j is equal to 1 this value is 2 minus 1 this value minus 2 x k. So, it will be a 2 common if you take minus 2 x 1 means x 1 this plus tau k into minus g 2 of x g 2 of x is equal to our g 2 of x is minus 2 x 1. So, minus minus 2 g 2 of x is minus 2 x 1. So, minus minus 2 x 2 this is 2 x 2 g 2 2 x 2. So, ultimately it will come like this way 4 that 1 third x 1 plus 1 whole cube plus x 2 bracket closed minus tau k 2 1 minus x 1 then minus minus plus. So, this will be a plus agree plus then tau k divided by 2 x 2 agree. So, this thing is our penalty function for this one and tau k is the our penalty coefficients for when in inequality constant is there. It is obvious that if you have a equality constants are there we cannot directly apply this one at this moment. So, let us see that one how to solve that quantity. So, now our necessary condition. So, del p with respect to del x 1 if you differentiate this with respect to del x 1 3 x 1 plus 1 3. So, 3 3 cancel this is 4 x 1 plus 1 whole square then x 2 will not come into the picture then x 1 is involved here that minus tau k then if you go up it will be a minus 1 into that tau k by 2 is constant every again minus 1 this then it will come 1 minus x 1 whole square then differentiation of that minus 1 minus x 1 is minus 1. So, ultimately it is coming is 4 if you consider that this equal to 0 agree. So, I can write this one x 1 plus 1 whole square I divided both side by 4 minus tau k tau k by 8 this minus minus plus. So, this into 1 minus x 1 whole square is equal to 0 or this. So, that is the equation we got it let us call this equation is 1. Then next equation because we have a two decision variables are there that next equation del p of this necessary next necessary condition is x 2 is equal to you are differentiating this with respect to x 2. So, there is only one x 2 is here another x 2 is here. So, this will be a 4 minus tau now let us see this one 4 then x 2 is here tau k by 2 tau k by 2 if you do and then x 2 1 by x 2 differentiation plus tau k by 2 1 by x 2 differentiation is minus 1 x 2 square x 2 square that equal to 0 or again divided by 4 both sides. So, it will be a 1 that is minus tau k by 8 x 2 square is equal to 0 this is tau k. So, if I multiply by both side 8 x square then it is x 2 square minus tau k is equal to 0 this. So, this is let us call equation number 2. So, from 2 one can write it x 2 from 2 from 1 can write it x 2 square is equal to tau k by 8. So, x 2 is equal to plus minus tau k by 8. So, this is the solution now question comes here which sign that means this is root 8 and root a. So, this is which sign this x 2 is plus minus root over tau k by 2. So, this is root 8 then which sign we will consider we have seen if you just see the problem of our original problem our x 2 is greater than 0 that is what we have seen x 2 is greater than 0. So, you want to make x 2 is greater than 0 8 is a positive quantity tau k is also positive quantity. So, we have to consider the positive sign of that one. So, consider positive sign only and this is we come to conclusion from the what is called our feasible region is given x 2 is greater than 0 on this one. So, therefore, consider so x 2 is equal to plus tau k square root of tau k divided by 2 root 2 this is our that one. So, now we can get it x 1 value from equation 1 from equation 1 from equation 1 we see this equation 1 now that one from equation 1 is this is 0. So, x 1 plus 1 whole square is equal to I can write it this is from equation 1 I can write it x 1 plus 1 whole square is equal to tau k 8 1 minus x 1 whole square. So, now if you just do it 1 plus a plus b a minus b. So, it is a coming this and this will come a x 1 square whole square is equal to tau k by 8. So, 1 minus x 1 square is equal to plus minus root tau k divided by 2 root 2. Now, look at this one which sign I will consider now this one if x 1 is less than 1 if let us call x 1 value is less than 1 that this quantity will be positive. If x 1 quantity is greater than 1 then this quantity will be negative, but our feasible region is x 1 if you see the our statement x 1 is greater than equal to 1. So, this quantity will be a negative for that feasible region. So, we will consider negative sign only. So, our 1 minus x 1 square is equal to minus square root of tau k divided by 2 root 2. So, from there I can easily find out x 1 square is equal to 1 plus tau k divided by 2 root 2 and x 1 is equal to once again this quantity you see tau k is positive this is positive 1 plus tau k is also positive. So, this will be a plus minus square root of 1 plus square root of tau k divided by 2 root k this whole quantity is positive quantity because of tau k is positive quantity. Now, once again what value of plus will consider minus will consider naturally we have considered plus because x 1 value is greater than equal to 1. And this since this is a positive if we consider negative that x 1 value is coming negative. So, consider what is called positive sign only based on our physical region of this one. So, our solution now x 1 is coming now if you see therefore, our x 1 is square root of I get 1 plus square root of tau k divided by 2 root 2. So, this is our x 2 and we have already got x 1 we have already got what is called x 1 value is if you recollect this one it is 1 by 2 root 2 square root of tau k this is the thing. Now, we have seen there that if you penalty coefficient if you start from high value and then decreasing this value to this it is approaching to the boundary of our what is called optimization problem you approaching. So, now to get the optimal solution optimum to get the optimum solution is obtained by setting or by setting x tau k tends to 0. So, if tau k tends to 0 this is 0 0 of this one. So, x 1 is 0 and this is 0 means x 2 is this is 1 sorry this is 0 1 is there. So, this will be 1 this is a 0. So, our optimum solution is x 1 x 2 star is 1 0 and if 1 to find out and corresponding objective function value one can find this one. So, this is analytical approach we have solved, but I told you once again if you number of what is called the variables are n then you have a n number of what is called that necessary condition. So, you have a n number of necessary condition but to express as x 1 x 2 x 3 in terms of that what is called the penalty coefficient it is almost what is called tough job. So, you cannot express how we can solve this problem what was the set of equation you got it from the necessary condition that you can solve our standard numerical methods solving that set of equation optimization problem by numerical methods. And that we have seen you can solve it by Newton Raphson method that is conjugate gradient method then you would steepest descent method then modified Newton Raphson method all this thing you can apply to solve a set of non-linear equation to get the optimum value of the function. So, this is let us call once you got the expression of the iterative this is we had then finally, we expect x tau k tends to infinity to 0 this when exterior point we have start from the low value of interior point that means we are starting from the far away from the it is approaching to the bounded of this one when tau k tends to infinity. But here is interior point is different both the methods from the exterior points we take the point from the outside the feasible region and it gives a sequence of solution and ultimately we are approaching the optimal solution of the problems. Whereas, in interior point method we take a point in the inside the feasible region and ultimately a sequence of iteration it will give you the what is called optimal solution of the non-linear optimization problem that means it is approaching to the boundary of the feasible region this one. So, let us solve this thing is using iterative method similar as exterior point using iterative method you know the x 1 x 2 expression in terms of what is called the penalty coefficients. So, that table we form it like this way table tau k then x 1 superscript tau k is equal to just now you have computed that expression 1 plus square root of tau k divided by 2 root 2 bracket then you find out x 2 of k that will be a tau k divided by 2 root 2 this is x 2 then you find out the penalty function where value at each iteration that one then you find out the objective function value as given in the problem. So, what is p I am writing p is if you refer to our earlier pages then you will see 4 that is one third x 1 plus 1 whole cube plus x 2 that is our problem p that is f of x and then tau k divided by 2 that x 2 see this one p what we have written p that p is written here you see this minus that one that is coming minus tau k 1 minus x k. So, it is a that you can write it tau k x 1 minus this our problem is this one this this thing agree. So, that will come tau k that means x 1 to x 1 minus x 2 minus x 1 minus plus tau k 2 x 2 see this one what is this it is actually minus I just change this order x 1 minus x 2 minus minus plus I change the order x 1 minus x 1 this and f of x value you know already this is our f of x if you see this is our f of x this value. So, let us call we started with a high value of tau k 1 1000 agree and immediately I can find out the value of this because tau k value is 1000 this is known. So, x 1 value is coming that 1000 means it is a 10 root 5 all this thing. So, this value will come 3.49 values and this value square root of 1002 by t that will come x 2 values will come 11.1803. So, we will just omit x 2 here x 2 because x 2 under this column only. So, x 1 value is this one x 2 value is this one. So, now immediately I can find out the f x this is our f x. So, this f x value is coming 165.412 put the value of x 1 x 2 in this expression that is our f x if you see that is our f x and it will get this one. Once you get this one f x you add this two terms with the f x tau k you know x 1 you know is 3.49 x 2 you know. So, you can find out and that give you will give you the 410.94. So, next what will go this is the value and this is our and you see if you see the value of f of x is that one. Now, you reduce the value of penalty coefficient by let us call one tenth. So, it will be 100 put the value of 100 is means 10 divided by 2 root 2 plus 1 square root of that one that will come 2.1296. Similarly, x 2 put the value of lambda tau k 100 then you will get this value is 3.535. Once you know the x 1 x 2 the f x value is that one this value you will get it 55.101 and similarly P you can calculate 1113.41. Now, once again you reduce this one let us call 10 one tenth of this one then this value will get 1.4553 this is 1.118 then this is 2421.1. So, this is 39.66. Now, this one you see function value is decrease. So, one can stop it when this part is almost 0 or P and f is same then only you can stop it or you can see our what is the stopping criteria you used we have used in the algorithmic steps that is the algorithmic you can follow it next is if you proceed like this way then if you just do next step is 10 to the power of minus 6 lambda tau k then this equal to 1.0002 and this will be a 3.534.354 into 10 to the power of minus 3 and once you know x 1 x 2 I can find out this one is 10.676 and which is nearly equal to 10.6792. So, if you further proceed and 0 we put it here is value is 1 and here value is 0 then this value is 10.666 and this value is 10.676. So, this value also 10.666, but in general I told you if you have a number of variables are n you have a n sets of what is called necessary condition and that x 1 x 2 all decision variables you cannot express in terms of our penalty coefficients. And this method you cannot apply it either it is between method or that analytical method this one what you have to do once you get it the set of equation necessary condition from the set of equation from the necessary condition that you solve numerically and get the results for optimum value of the function. So, this is about that it is and there is a another method is there what is called they called is a what is called logarithmic interior penalty. So, if you have a now in more general at this logarithm penalty function structure is like this one the penalty term can also be taken the same as earlier method 1 by summation of reciprocal of 1 by g i minus only we are expressing in terms of logarithm that as the logarithm of the constraints. So, our penalty function is like x then tau k then f of x minus tau k then your summation of j is equal to 1 to m we have a m such inequality condition minus of g j of x this and this tau k value is greater than 0 and it is a penalty coefficient. See this one because it is a interior point it is a interior point method we take a point in the inside the feasible region and that consider is a interior point which satisfy the inequality constraint of the problem statement. That means this we can consider this is a negative term negative positive. So, this quantity l n natural log of this one you will get some value then tau k is this one. So, logarithm and penalty term can be used for equality constraints. So, note this logarithm penalty function can be used in equality constraint after modifying the equality constraint let us call this is the equality constraint we have is equal to 0 for i is equal to 1 2 dot dot p our problem is this one. So, this inequality constraint I can convert into a 2 inequality constraint h i of x another is this and both are i is equal to 1 2 p. So, inequality constraint you convert into this and then this multiplied by minus 1 both side you will be greater than equal to. So, one you write it in place of that one you write it in terms of h i is greater than equal to 0 minus h i of x is again greater than equal to 0. So, replace equal to sign by this then apply the logarithm penalty function method this. So, let us see this one directly you can use the h of this here just like a exterior point method you cannot use directly this one. So, let us see what is the term associated what is this meaning of that term penalty term of that one will just investigate this term let us see agree this whole term will investigate. So, thus whole term let us denote is this term is I am calling logarithmic logarithm penalty function term that is equal to minus tau k ln minus g of x k this is that part only I have denoted by capital omega this part. Let us investigate this part. So, let us call I am plotting this one in this relation g k and in this direction is omega and plotting. So, this function if you see when g k value is when g k g g of x value is is constant is g of x is less than equal to 0. So, let us call g of x value is minus 1 minus 1 minus plus then this value is 0. So, at one this function value is here then further if it is less than 1 less than minus 1 minus 2. So, this will be what minus 2 minus 1. So, minus 2 this value will increase for a particular value of lambda k then further it is preceded with minus. So, this will be a something like this next point is here and further your g k is let us call it is less than minus 2. So, minus 2 minus plus this is further it will increase like this way. So, it is nature is like this way it will when g k is greater than minus 1 greater than minus 1 let us call point minus 0.5 then this minus minus plus 0.5 this value will be more than this minus and here preceded with minus. So, plus so this value will be positive value. So, I can write the plot the positive penalty the positive penalty term for omega. This is our penalty term this is the penalty term and when this penalty term is positive when the value of this minus 1 when the value of g k is greater than minus 1 the penalty term is positive. When the value of g k is less than minus 1 the penalty term negative penalty term. So, negative penalty term now note this one when you will get the negative penalty term for this one one can avoid this one by assigning omega is equal to 0. So, the negative penalty could be avoided by setting omega is equal to 0 that omega value is 0 we can set it. Because the penalty term if you want to keep it positive then this for all x such that g of x is less than minus 1 that is what we have seen g of x is less than minus 1. Now, let us see then how to solve the problem by using the logarithmic penalty function methods using the interior point interior logarithmic interior point method penalty function method. So, example use the logarithmic interior point interior point penalty function method to solve non-linear optimization problems to solve non-linear n l p problem n l p problem. So, this example is we have taken minimize f of x is equal to twice x 1 plus x 2 this objective function is linear. Whereas, the constraints subject to the constant g 1 of x is equal to g 1 of x is equal to x 1 square minus x 2 is less than equal to 0 and g 2 of x is equal to minus x 1 is less than equal to 0. So, this shows first this shows that x 1 is greater than equal to feasible region x 1 is greater than 0 and once x 1 is greater than 0 this shows that x 2 is both side you multiply by minus 1 minus x 2 plus x 1 is greater than equal to 0. So, minus you take it x 2 is greater than x 1 square since x 1 is greater than 0 similarly, x 2 also will be implies greater than 0. So, that is our and x as a dimension real and which dimension is 2 cross 1 2 decision variables are there. Then if you want to solve by what is called logarithm method interior point method then first what have to do exactly in same manner you generate or form a penalty function for the given problem which you are going to solve by interior point method. That means, you are considering a feasible point inside the region and that feasible point is the interior point and starting from this one point you are now solving the a set of equation which is form which are form the necessary condition of the problem. So, this penalty function I am writing is f of x plus here is minus tau k summation of j is equal to 1 to 2 l n minus j g j of x whole this one. So, f x you know this one 2 is x 1 plus x 2 minus tau k l n plus x 2 minus l n minus sign is this one g o 1 of x is that one g 1 is what x 1 square minus x 2 bracket bracket closed. Then minus j is equal to l n minus j is equal to is minus x 1 bracket close this is our penalty function agree. So, let us call this is equation number one or you can say that one if you can rewrite that one if you rewrite this one or say because instead of rewriting this one write equation one. Then our necessary condition for this one necessary condition so del p del x 1 now see this one with respect to x 1. So, that will be 2. So, is equal to 2 and x 1 is involved here agree x 1 is involved here and x 1 is involved here they are minus plus. So, if you differentiate with respect to this one tau k divided by 1 by x k minus sign. So, third second one what will write it see this one second term minus tau k as it is minus tau k it is then this one is reciprocal of that one reciprocal of that one differentiation of this one means minus x 1 square plus x 2 and the differentiation of that one is equal to into minus twice x 1 agree. So, this minus minus plus and there is another term is here if you differentiate with respect to this is minus minus plus this minus. So, it will be 1 by x 1 so tau k divided by x 1. So, that is equal to 0 agree. So, we can write it twice plus twice tau k into x 1 this minus minus plus minus tau k by x 1. But here denominator part is this one then minus x 1 square plus x 2 this equal to 0. Let us call this is equation number 2 and del p del x 1 I have done it del x 2 is equal to what now you differentiate with respect to 2 is here involved 2 is involved is x 2 involved here. So, we can write it now that 2 minus minus tau k you can write and this will be 1 by that one and this minus and this minus plus. So, there will be no coefficient will come here. So, that will be a minus of this agree then what will write it that one tau k divided by minus x 1 square plus x 2 into 1 see this one this part tau k I am differentiated with respect to x 2. So, divided by this one so it is minus x 1 plus x 2 into differentiation of x 2 is 1. So, this equal to 0. So, we have a 2 is equal to tau k x 1 square minus plus x 2 this let us call this is equation number 3. So, from 2 1 can solve the equation of for x 1 because see from there. So, you see tau k minus x 1 square plus x 2 tau k divided by x 1 square plus x 2 this portion I can replace by 2 this equal to I can replace by 2. So, if you replace from 2 now from 2 this 2 plus 2 into 2 into x 1 into x 1 minus tau k divided by x 1 is equal to 0. So, if you solve this one this equation because is now a value x 1 you can express in terms of tau k. So, now our solution of x k I am skipping this one you will get a quadratic equation like this x 1 4 x 1 square plus twice x 1 minus tau k is equal to 0. If you simplify that equation you will get it that one. So, therefore, x 1 is equal to minus b plus minus root over b square minus 4 a c c is minus tau k divided by twice a. So, ultimately we will get it this one after simplification of this one we will get it 1 minus 1 plus minus root over 1 plus 4 tau k. Now, tell me this is the x 1 what value of x 1 I will select that if you recollect that our original problem we have considered here that this. So, x 1 is greater than 0. So, if you consider minus sign this quantity is greater than 1. So, it will be a negative. So, 1 is to consider this minus 1, but this is greater than 1. So, we have to consider positive sign. So, that our x 1 consider because consider the positive sign because of that restriction or you can say that our g 2 is the same. Since g 2 of x is equal to minus x 1 is less than equal to 0 and you have seen x 1 is greater than equal to 0 this one. So, I have to consider their positive sign you have consider 1 fourth minus 1 plus 1 plus 4 tau k. This is our solution for x 1 once you know this x 1 then from 3 equation 3 this one from 3 put the value of x 1 here then you will get x 2. So, I can write it this one from 3 2 is equal to tau k divided by rewrite this equation minus x 1 square plus x 2. So, it is you will get it minus x 1 square plus x 2 is equal to tau k by 2. So, now I can if you take the x 1 square that side it will be x 2 is equal to x 2 is equal to x 1 square plus tau k by 2 at x 1 square just now you got it the x 1 x 1 is this one square of that one. So, it will be a 16 1 by 6 is equal to minus 1 plus root over 1 plus 4 tau k whole square plus tau k by 2. So, again now you put it tau k tends to 0 in order to get the optimal solution of this problem to tau k tends to 0. We get x 1 star is equal to x 1 you see here x 1 expression this x 1 tau k is equal to 0 that means it will be 1 minus 1 plus 1 0. So, that will be a 0 and x 2 star will be see x 2 star tau k is 0 this is 0 this is 0 and this is 0. So, it will be 1 1 minus 1 0. So, this is a 0. So, our optimal solution for this one x 1 star x 2 star x 2 star the optimum value of this one is equal to 0 0 and the corresponding things you can find out the value of and same thing you can solve it by using the iterative method the same problem you can solve by using iterative. So, I leave it this is an exercise you just do it at home, but the table I formed it. Then x 1 superscript k is equal to whatever the expression you got it minus 1 plus root over 1 plus 4 tau k x 2 k is equal to 1 by 16 minus 1 1 by 16 minus 1 plus square root 1 plus 4 tau k square plus tau k by 2. Then you write p expression you know already p expression p then you write it f expression this f of x superscript k. So, I will just do for you first iteration k is equal to 1 if you consider 4 put the value of 4 here then you will get 0.781. Then this value will come 2.61 and this value will come 0.173 because this expression you know already what is the expression we have considered here. This is the example if you see the example we consider that value of what is this expression this example and p is that one put this value of x 1 x 2 tau k in this expression that value you will get it this one. This is the expression for this one you put it x 1 x 2 tau k value in this expression p expression you will get it. Then f f f is the objective function value that you will get first you will get calculate the objective function value 6.76.782. So, this next you reduce the value of this one by one tenth let us call second iteration take 0.4 and you will get it 0.0192 and that value you are getting 0.2001 that function value you are getting it 0.4386 and this value you will get 1.406. So, if you do this one iteration if you go on increasing at tau k is equal to 0 you will get x is 0. This is 0 and this value you will get it 0 because x 1 0 x 2 0 what is the objective function twice x 1 plus x 2 this is 0 and ultimately this is 0. So, our solution is that our solution is coming x 1 star x 2 star is 0 0. Now, you may have problem here you see that when you consider the our penalty function of that one if you see tau k if you see this one tau k is minus l n x 2 minus tau k minus x 1 square plus x 2. This when you put x 2 is 0 then I told you when this coming is x 2 is 0 this is a coming l n you cannot find out this one. So, this you assign with this value is equal to 0 that you are coming to this one. So, this is the end of this when this is problem when tau k you assign to 0 that. So, we will stop it here for the next class we will just consider the. So, far we have discussed the single variable optimization problems, but in real practice and application you will see the number of what is called objective function may be more than 1 2 or more than 2. So, in that situation how to obtain the optimal solution of a multi objective optimization problems how to solve it for a non-linear or linear optimization problems. So, that we will discuss next class.