 the last class we have taken in one example for a non-linear programming problem that affects is equal to four one third x one plus one whole cube plus x two subject to this constraint and we have to solve this problem by using exterior penalty function method. Then we have seen this one that x one is greater than one and x two is greater than equal to zero these are the physical region. So, initially we have to form a penalty function method with the objective function and with the constraint like this way. Once you form this one then your job is to find out the necessary condition for the function to be optimized that we have seen it that first this is the analytical solution by finding the necessary condition del p del x one del p del x two is zero. So, after solving this after details expression for one and two we will get a set of two simultaneous equation and our job is to solve this one. And ultimately after solving this one we got x one value with this expression, but you have to take proper sign of when you are finding out the roots of this quadratic equation of that one you have to take the proper sign of this one because x cannot be negative because of feasible region solution. So, that plus you have to consider this one where this plus this term is greater than this one that we have explained last class. Similarly, x two when you solve this x two you have to take proper sign change, but our x one x two if you see this one x one x two our things should be a what is called that in the x what is called infeasible solutions is it not because infeasible solution x three point mean infeasible solution we are taking and then we are finding out the function value which is function value is decreasing with each iteration. And ultimately it will converge to a some optimum value of the function which is a feasible. So, that we have seen and we have got the results this one x one is equal to one x two optimum value is zero when tau k that penalty coefficient tends to infinity. Let us solve this problem by using what is called iterative method because we are using the necessary condition expression necessary conditions what we got x equation number one on two we express x one and x two in terms of tau k. And now tau k I will change from small value to very large value the infinity and if you use the iterative method how you will solve this problem that will see using the iterative method. That is we are considering the exterior point exterior penalty function method the same problem what was the we got the necessary condition for this problem and express x one in terms of tau k x two in terms of tau k that we are writing now. So, our iterative method we make the tabular form and this is the tau k penalty coefficients then we will expression for analytical expression for x one of k k superscript k indicates iteration which expression we got it one bracket minus one plus tau k see the expression plus root over tau k square plus four tau k under square root then x two k is equal to minus one by two tau k then penalty function value you write it which is function of x superscript k into tau comma tau k and then you find out the f if you like you can find out x of k capital x of k then you can function value f of x superscript of k this note that our f of x small f of x is equal to four one third x one plus one whole cube plus x two this is our f x and capital f x of superscript k is equal to max two one minus x one zero this then whole square for g one constraints and for another constraint max minus twice x two comma zero max of this and this square. So, this is our excess let us calculate this one first which start let us solve small value of tau k point zero zero one point zero zero one then this value put the value of tau k here in this expression you will get this value is point nine three seven seven then x two one by this if you put it this value you will get minus five hundred and once you know x one x two that function value you can compute that function value is minus two thousand then once I know x one x two f f of x I can compute that value will be hundred thousand point zero two. So, what is this this multiplied with this is multiplied by tau k also I included tau k mind it this is also I included tau k and then got it you can you can find out only f k, but I multiply this tau k into f of k tau k into f of k is that one. So, this then this is equal to what this minus tau k of this one that p then you will get this value is this then tau k see this expression that what we have written it that p expression p expression f of k tau k into this whole thing is capital f of k. So, this and this you add it this and this if you add it you will get it minus nine nine nine point nine eight then next choice of tau k is ten times of this one that is point one if you take because lambda k for exterior point method if you recollect lambda k plus one divided by lambda k is equal to c c is greater than one and we say that c value is ten. So, next is this value if you put this value is this you will get point eight zero nine minus fifty then you will get f is you will get minus one point one ninety nine point nine nine then lambda k into f of k is here f of x I know multiplied by lambda k you will get it this value is here that is you are getting hundred hundred point one three then this plus this will be your ninety nine point ninety nine point eight six but minus sign again ten times of this one point one you see the function value is decreasing two thousand two now it is coming near about two hundred minus two hundred. So, next value is this is point zero four five nine seven minus five and this value will be nineteen point minus nineteen point seven nine and this value will come ten point eight five because when you put this value is lambda x one x two value here and multiplied by lambda k this value is I am representing here. So, this plus this will be minus nine minus eight point nine three eight so now if you proceed like this way and increase in the value of lambda k you will see at one two three four at ten thousand lambda k value is ten thousand the value what will get it the x one value by applying lambda k here x one value will come point nine nine nine six three then x two value will come minus point zero zero zero zero five and f of x value will be coming is that is ten point six six one and lambda k into f of x value is coming point zero zero zero one two. So, that value will coming this plus this is ten point six six two now this see this point our feasible region is x greater than equal to one and x two is greater than zero but both are not satisfy this one still it is a infeasible region. So, we need further more iteration if you go on iterative like this very large value of lambda let us call say infinity then this will get one this will get zero and this will get this value will get it here thirty two by three this is zero then this is thirty two by three. So, this contribution of the penalty function is become zero when it reach to the optimum value of the function. So, our optimum value of the function is thirty two value thirty two by three and corresponding optimum point is x one is one and x two is zero these are the optimum point for which the function value is this. So, this we have and you see even though this this two points still is not is not in interior point it is a exterior point still interior point and since we know the our feasible region x one if you see the feasible region x one is greater than equal to one and x two is greater than equal to zero. So, both condition is not satisfying so it is not a interior point of that one. So, next we will see some of the problem associated with this one or you can say more general expression will discuss the problem later for general form of that one that how to solve the problem using that what is called exterior penalty method penalty function method. We have seen that minimize the function again subject to inequality constraint suppose if you have a equality constraint then how to tackle that problem. So, it is a general form general form of non-linear programming problem using exterior penalty function method what sense I am telling you we have not considered the equality constraint. So, our problem is like this way this is the minimize f of x subject to h i of x is equal to zero and we have a such equality constraint p equality constraint is there and equality constraint x j of x is less than equal to zero we have m equality constraint now how to introduce this one. So, this one can write it there are two ways of doing one can write it h i of x zero this one can write it into a inequality constraint h i of x greater than equal to zero h i of x is less than equal to. So, this equality constraint I can write it this and we can solve similarly as we have discussed earlier. So, in each equality constraint we get it two inequality constraints. So, in turn we will get two p inequality constraints and there is already m inequality constraints are there. So, there will be a m plus two p inequality constraints we will get it, but in generally we can write penalty function penalty penalty function we can write it. So, this p which is a function of x tau k and beta k tau k is penalty coefficient associated with the inequality constraint and beta k is the penalty coefficient associated with the equality constraints. So, this is equal to now we are writing objective function as it is plus beta k summation of all one to p equality constraint h i of x this whole square that in just like a inequality constraint we used to do gamma k summation of j is equal to one to p then we used to write it max of this max of g j comma zero whole square in place of this one j of j square plus gamma k summation of j is equal to one to m max g j of x zero whole square and this is let us call equation number one and these are these and these are the penalty coefficient gamma k tau k let us call we have written tau k throughout this one tau k is the penalty coefficient associated with the inequality constraint and beta k is the penalty coefficient associated with the equality constraint both quantity is greater than zero. So, we make some comments of this one that now how to choice choice of initial values of penalty coefficient that means tau k and beta k one is the simplest choice is there you consider a point which is outside the feasible region that is called the exterior point then beta k selection of beta k initial choice of beta k after that what we are doing we are incrementing we are increase the value of beta k or gamma k by 10 times. So, initial value of beta k penalty coefficient associated with the equality constraint and tau k associated with the inequality constraint how to select it. So, that value is so objective function value then penalty terms value corresponding to equality constraint penalty terms because we have a penalty terms associated with the equality constraint we have a penalty terms associated with the inequality constraint. So, this penalty terms then tau k is objective function value similarly then penalty terms value corresponding corresponding inequality corresponding to inequality constraints values corresponding to this. So, this initial value we have to select this is the one choice of this one next is one remark important remark is there that analytical solution that what we did it that we in order to find out the what is called first the initial that necessary condition we have found out then we have solved this problem for x 1 the solution we have solved this two set of equation or in general n set of equation by x 1 x 2 all these variable we have calculated in terms of our tau k and for the present problem in now you have at x 1 x 2 everything you have to express in terms of tau k and beta k, but it is really very tedious job to get such an expression. So, we have to look for what is called the what is called numerical method solution of a set of equation which is obtained set of equation which are obtained from the necessary condition of the problems. So, let us what is this I just mentioned in this one remarks of this one for a general non-linear programming problem there will be n equations because if the non-linear problem is n decision variables are there then when we are taking the necessary condition that after forming the penalty function del p of del p with respect to x 1 del p with respect to x 2 we have a x 1 x n variables are there. So, we will get x n equations given by from where you will get it the gradient of penalty functions x the penalty function is the function of tau k and beta k to be solved and there will be m max term in each of this equation in each of this equations because we have a m we have a m inequality constant in if when you have a inequality constant m inequality constant in penalty function you will get summation of max term of g 1 comma 0 whole square then g 2 max of g 2 comma 0 whole square in this way m terms will be there each equation agree it is almost that is what is the main stress thrust is there it is almost difficult to express to express x i which is a function of tau k beta k for i is equal to 1 2 dot dot because we have n decision variables and then take limit tau k tends to infinity very large value and beta k tends to infinity to get optimal solution solution of the decision variables x 1 x 2 x 3 dot dot x n. So, one should look for what is called the numerical method to solve a set of necessary condition if you have n decision variables are there you will get set of n non-linear algebraic equation that you can solve by some suitable numerical methods that is our stress. So, therefore, it is necessary to solve equations del p of x tau k beta k is equal to 0 that is we is the necessary conditions using some numerical. So, this is the remarks for this one. So, next we will solve that what is called non-linear problem solution of non-linear programming problem using interior penalty function method or it is called barrier function method. So, next is solution of next topics is solution of non-linear programming problem using interior penalty function method or this is called interior penalty function is method call is a barrier function method. So, what is this we will see first if you recollect this one our earlier problems the exterior penalty function method generates a sequence of infeasible solution that converts to a feasible solution in exterior penalty function method generate a sequence of infeasible solution which converts into a feasible solution again an optimal solution whereas in case of what is called interior penalty function method it generates a sequence of feasible solution and ultimately it converts to a optimal solution feasible and optimal solution. So, this is the difference between the two methods. So, exterior you can write the interior point method interior penalty function generates a sequence of feasible points that converges to a feasible and optimal solution that means x n x superstar of the source dimension is n cross 1 n if it is a n visible. So, our problem is we have a problem minimize the function subject to the constraint then interior point method generate a sequence of first initial guess is the interior point in the feasible region then it generates a another feasible point where the function value is decreasing and in this way is generate a sequence of feasible points and ultimately it converts to a feasible and optimal point optimal solution. So, let us see this one the problem statement how will do it. So, minimize f of x with the objective function you have to minimize such that g j of x is less than equal to 0 j is equal to 1 2 dot dot m agree. So, this is our problem now we have to solve by using a interior point method agree the interior point method means we have to take a initial guess which is inside the feasible region and satisfy these all these things feasible means it should satisfy the what is called the call all types of constraints. So, corresponding to this you problem construct the following interior penalty function or it is called in keeping a barrier function barrier means obstacle function barrier always is we understand the obstacle function the sense that it will not allow to move the decision variables boundary of the feasible regions it will not allow to cross the boundary of the feasible regions. So, now let us see this one what is the penalty function is considered for this one. So, p x tau of k is similar as earlier also exterior point method f x plus tau k and tau k is the our penalty coefficients. Since we have a since we have a m inequality constraint we have to give the penalize this one j is equal to 1 to m minus 1 by g j x agree now look this one our point is interior point means point is inside the feasible region when it is a feasible region g j of x value is negative agree when it is negative negative is a positive. So, this quantity is positive that means this quantity and also tau k value is greater than this tau k value is greater than 0 agree. So, this is our let us call equation number 2 and this set of equation what about the description of the problem it is a equation number 1. So, with so this term is this whole term is penalty function term this is the this and this is the penalty function term penalty function terms. So, with tau k plus one next iteration tau k value will be is equal to see where c is less than equal to 1 say c is equal to 0.1 we are decreasing the penalty function value if penalty coefficients value one tenth each iterations. So, let us see the interpretation of this things why the tau k value will be decreasing and ultimately it is approaching to 0 to get the optimal solution of the problem using interior penalty function method. Let us take this simple example minimize x such that or subject to 2 minus x is less than equal to this. So, this implies this implies x is this implies x is greater than 2. So, our feasible region is x greater than 2 if I plot it this x is in this directions and this direction is our penalty function p penalty function in this directions if you plot it. Then let us see f of x is what first f of x is what is nothing but a x and that f of x you have to minimize. So, this is nothing but a is a a straight line which is passing through a origin and our feasible region is if you see the our feasible x greater than 2. So, let us call this is 1 this is 2 so greater than 2. So, our whole feasible region is that portions. So, this is x is equal to 2 this is 0 and this is you can write it this nothing but a f of x is equal to x and with this one there is another function is added. So, our g of x for this one is 2 minus x. So, if you write it minus 1 by 2 minus x and our initial guess is we have we are consider in the feasible region. So, the x value will be greater than 2. So, let us see when x is equal to greater than equal to when x is equal to 2 the function value if you see this one the function value is x is equal to it is very large and then it will go on decreasing and it will approach to this asymptotically it will approach to the objective function values. So, this so let us call these values is I am plotting is that this function value that means our inner case if you see the p inner case if you see the p tau k is equal to f of x means x plus tau k only one inequality constant is that 2 minus x inequality constant is there. So, I write it into minus 1 by 2 minus x. So, I am I am now plotting I am now plotting this quantity again I am plotting this quantity. So, this value is infinity of that to that to that one and then if you go on increasing x then this value will be it is negative plus. So, this will be this way and it is approaching to the asymptotically it will approach along the objective function. Let us call you can think of it as if that our penalty coefficient value is one when you are plotting this one. Now, the if the penalty function value is made it half or one tenth now what will be this the nature of the curve will be exactly same on only at each point of x the magnitude will be reduced by one tenth if you are considering that tau k value is one tenth. So, this will be once again if you see this one once again this will be like this way. So, this is let us call tau k value is one again and I am plotting this is for tau k value I am plotting g of 1 by minus this is I am plotting 1 by g of k g of x I am plotting I am plotting tau is equal to 1 g of a and this is corresponding to tau of k is equal to 0.1 and this is minus 1 minus g of x I am plotting. So, in this way if you are reducing the value of tau k by one tenth all these things ultimately you will see this value will be decreasing ultimately you will see this will be like this way and it will be approaching along this one. And when x is equal to 0 is you say when x is equal to sorry when x is equal to x is equal to 2 or nearly equal to this infinite value is that this is right. So, it will not allow to cross the value of feasible region feasible region variables value cross the right left hand side of this one one that means it will not allow the this method will not allow to cross the boundary level of feasible region this method. So, that is why tau k value is slowly if you are increasing it will approach to the asymptotically asymptotically to the objective function near to the boundary of the feasible region that is our main objective. So, I will just write it the remarks of that one. So, remarks so that is the basic idea of using the interior penalty function method. So, now remarks if x is within the feasible region in the feasible region or space for feasible range space and far away from the boundary then g of x g j of x is less than 0 and penalty term penalty term is small and again. So, penalty term is small and p x of tau k will be nearly equal to f of x this is first and second observation remarks is when x moves towards the boundary of the feasible space when x is moved towards the boundary of the feasible space and constraints tends to and constraints tend to be binding or g j x will be 0. Then the penalty function then the penalty function will prevent or you write that then the penalty function will become very large and preventing x from moving out of the feasible point. So, what we have discussed in this figure you say the first statement is made when x is far away from this one x is far away from this one you say this quantity is large that means 1 by this one is small very small. So, in that situation penalty function value and object function will be same that is the we told that that one when the our variable x is closer to the boundary of that that one when it is closer to this one then this quantity value this quantity value that 1 by x 2 value will be very large that is when x is moves towards the feasible region space tends to binding g x is equal to 0. Then only this 1 by g x will be very large then penalty function will be very large and will prevent the what is called the variable to go to the infeasible region. So, moving out from the feasible point. So, this is the basic idea behind using the interior point method. So, our algorithm takes algorithm steps for interior point method interior point function method same as earlier exterior point first step is what we will select a initial point which must be inside the feasible region. So, select a feasible solution or feasible point x superscript 0 and that must be a interior point that satisfies all constraints that means g j of x is less than equal to 0 j is equal to 1 to dot dot m and choose and select tau 1 greater than 0 and k is equal to 1 this is the first step once you select this one using the what is called necessary conditions all these things you find the new improved value of the feasible point which reduce the function value from the previous value. So, second step take x superscript of minus 1 as a starting point use unconstrained optimization unconstrained optimization problems unconstrained minimization problem if it is minimization problem m or you can unconstrained optimization problem and use unconstrained unconstrained minimization problem solution using numerical methods. So, our numerical methods we have discussed what is called Newton-Raphson method steepest descent method conjugate gradient method modified real Newton-Raphson method we have discussed earlier. So, taking the initial starting point you get the first you obtain the necessary condition for the function to be optimized and you will get set of non-linear equation that you can solve by numerical method or analytically use optimum solution using unconstrained minimization problem solution using numerical methods to obtain to find the optimum of barrier optimum of the barrier function or interior penalty function denoted by x superscript k from x k minus 1 you get it that one next step is step 3 is check check if the solution is this to the optimum check if the solution x superscript k is the optimum solution of the original problem. How will you check it I told you one way of checking it you find out the value of the function at kth iteration then subtract with k minus 1th iteration value divided by f of x k value. So, mod of this one if you see if it is less than equal to epsilon epsilon is a positive quantity positive small very small quantity is this one or you can check it this or you can check it that tau k or f of x superscript k that value is less than equal to epsilon 2 epsilon 2 is less than greater than 0 and very small quantity tau k is the penalty coefficient whose value is greater than 0 and this value you will see when it is in the feasible region that value is also greater than 0. And this f of x is what that each term summation of f that is what is called summation of minus 1 by g j of x that is the f of x repeat summation of j is equal to 1 to m minus 1 by g j of x is the f x check this one if it is satisfied stop the iteration otherwise you go to the next step otherwise go to the next step. So, step 4 when you go to the step to update your penalty function coefficient c when x tier function method I told the value of c is greater than 0 that value of c is greater than 0 and most value it is a greater than 0 and less than 1 say c is equal to we consider 0.1 now we are decreasing the value with by one tenth that we have seen why you have decreasing this one it is the feasible region which is far away from the boundary of this feasible region it will approach to the near the boundary when you are do like this way. So, that things and set k is equal to k plus 1 and go to step 2 and step 2 is this one you start the iterative process once again this one and in this way you will do until and unless this condition stopping criteria condition is not satisfied either one of this is not satisfied. So, this way the interior penalty function method or barrier function method is solved agree and these two methods are efficient in solving the non-linear what is called optimization problems. Then we will see next that how to solve a problem by taking a numerical example employing that I am doing that technique what we have discussed the interior penalty function method or barrier function method. So, I will stop the lecture here only.