 what is convex optimization problem? And in convex optimization problem that objective function is a convex function and the your inequality constraint is a convex function and our equality constraint is a affine function. Then we have discussed that what you mean by the quadratic convex optimization problem. In that situation that our objective function is a quadratic convex function, then our inequality and equality constraints are affine function means linear function in a convex set. Then we have discussed that what is quadratic constraint, quadratic optimization problems. In that situation objective function is a quadratic convex function and the equality constraint also quadratic convex function and equality constraints is a affine function. Then problem is how to solve this problems? Then one way of solving is that you just apply the KKT necessary condition, then after that you one can solve it by different techniques. One of the technique I am discussing here is the linear programming technique. So, in linear programming problems that if you see this one that our a problem is a linear programming problem when the our objective function is linear. And as well as our all constraints and equality constraints and equality constraints and inequality constraints are also linear. In order to solve this problem by simplex method you have to convert first simplex method means iterative method numerical methods. You have to first convert the linear programming into a standard LP problem. What is standard LP problem is there minimize the function which is a linear function which is written c transpose x subject to a all our equality constraints. So, if you have a in the problem actual problem if you have a inequality constraints are there one can convert into one has to convert into a equality constraints. If you put it into this form equality constraint and the right hand side is the constraint and that constraints are greater than equal to 0 means non negative number. And our all design variables x 1 x 2 dot x n are also greater than equal to 0 the non negative number. So, this is our any linear optimization problems if you have to do it first you convert it into a standard LP problems. And what is standard LP problem we just mention it here. So, now see how to convert the standard optimization a linear programming problem to convert first into standard optimization problems. So, let us take some example and mention convert to standard LP problem. So, our problem is let us call maximize z is equal to 3 x 1 plus 4 x 2 and subject to 4 x 1 plus 2 x 2 is less than equal to 4. And 6 x 1 plus 8 x 2 is greater than equal to 24 and we have a minus 2 x 1 plus 2 x 2 is equal to minus 6. And we have given x 1 is greater than 0 and x 2 is unrestricted in sign unrestricted in sign. This is the situation that means that x 2 value can be positive negative and 0 whereas x 1 is greater than equal to 0. So, we have a different types of constraints are there that is less than equal to type greater than equal to type equal to type is there. So, first is if you want to solve this problem by simplex method means iterative method then first you have to convert into a standard LP problems. Let us see how to convert the standard LP problems standard LP problem convert. So, first is this is unrestricted in sign. So, the x 2 because all variables here must be greater than 0 greater than equal to 0 for standard LP problem. And all inequality constraints we have to convert into a equality constraints and the right hand side of the equality constraint must be positive. So, these steps we have to do it here. So, let us see I can always write that our you can just solution not the solution of the problem how to convert into standard LP problems. So, our x 2 I can always write it x 3 minus x 4 there is no other variable other than x 1 and x 2. So, I have written x 2 is x 3 minus x 4 where x 3 is greater than equal to 0 x 4 is greater than equal to 0. So, which in turn the value of x 2 depending upon the value of x 3 and x 4 it can be a positive it can be a negative it can be 0. So, we introduce in place of x 2 in this problem by x 3 and x 4 that means x 3 minus x 4. So, this part is now becoming x 3 is greater than 0 x 3 x 4 is greater than equal to 0. So, x 2 now we will replace by 2 new additional variables next is I have to convert this equation into a equality constraint. And the right hand side of the equality constraint must be positive. So, that how to convert that one. So, let us take the first equation 4 x 1 plus 2 x 2 is it is if you see this one it is less than equal to 4. That means this is a positive quantity this is a positive quantity that means left hand side left hand side of this expression is less than 4 it indicates that if you add some positive quantity with the in the in this expression of left hand side then I can make it this inequality constraint as a equality constraints. So, I can write it this plus a new variable here we have x 3 x 4 is there. So, I have to write it that x 5 I have to add some positive quantity. So, this is our values is greater than equal to 0 I told you whenever you will get it x 2 that will replace by x 3 minus x 4. So, I can write it this now this if you write it x 3 minus I will just rewrite this one once again because there is a 2 x 2. So, I will replace by x 2 by x 3 minus x 4 and that is less than equal to 4 that means right hand side is a positive quantity and left hand side is less than the right hand side. So, I have to add some positive quantity in the left hand side expressions. So, this is a x 5 x 3 is there x 4 the next variable is x 5. So, that variables are called slack variables and that variables is greater than equal to 0. So, this equal to your 4. So, first equation which is given inequality we have converted into a equality constraints equality constraint and as well as right hand side of this action is a positive quantity. So, next equation if you see this is 6 x 1 plus you see 8 x 2. So, x 2 will be replaced by x 3 and x 4 and it is greater than equal to 24 greater than equal to that means it indicates that this quantity right hand side quantity is positive. That means I have to subtract in the left hand expression something. So, that it will become equal to 24. So, it is 8 into x 3 minus x 4 I have to subtract some positive quantity in from this expression minus up to we have gone up to x 5. So, I will put it x 6 I will write this is equal to 24, but there is a some problem is here. If you write it this one let us call we have a this equation is there and 2 equations are there for the time being I asked you to solve this one for the time being if you see there are 6 unknown variables and 2 there are 2 equations are there how you solve it it has a infinite number of solutions are there. Suppose if you consider x 1 x 2 x 3 is 0 x 1 x 3 x 4 is 0 then x 5 is equal to 4, but x 6 is equal to minus 24, but x 6 value is positive. So, when you have a such type of inequality constant this is not enough just introducing a minus a variable. So, that variable called surplus variable surplus variable there is not enough just if you add any surplus variable because why your surface that this quantity is greater than that you see this quantity is greater than 24. So, something you have to subtract in surplus from this variable. So, that is why it is called surplus variable and just now I mention you if you ask to solve only this 2 equation it is a equally 2 equation agree then I can say there are 6 variable 2 equations are there. So, you have a infinite number of solution if you just consider that x 1 x 1 x 3 x 4 is equal to 0 then 1 case you can x 5 is equal to 4 it is satisfied because x 5 our positive quantity I have just written positive quantity satisfied and x 6 also our greater than equal to 0 positive quantity. Because I am subtracting some positive quantity to get 24 when I made it x 1 is 0 x 3 is 0 x 4 is 0 that x 6 value is coming minus 24 just contradicts. So, this is we need to add what is called another variable and that variable that is called after x 6 it is x 7. So, this variable is called artificial variable variable agree. So, when you have a such type of equation some expression greater than equal to some quantity this one if you get it then you have a 2 variables you have to introduce 1 is surplus variable another is artificial variable. So, this is the second now third equation you see we have a equality means this equation is there equality constraints are there, but only the things we have a right hand side is a minus we have to make it plus. So, both side you multiply by minus if you multiply by both side minus it is a twice x 1 minus twice x 2 is equal to 6 and our artificial variable I forgot to mention in our artificial variable also non negative number greater than equal to 0 artificial variable. So, you see this all this equality constraints all this thing we have converted, but what is this standard problem it is given convert into standard it is given maximization problem. So, our problem will be now if you see this one our problem is minimize f f is nothing, but I told minimization maximization problems I can if I know how to minimize a function I can use same technique here. So, it is nothing, but a what the function we have suppose to maximize we should do with minus that function is minus z you know that is 3 x 1 plus 4 x 2. So, I will x 2 I replace by x 3 and x 4. So, it is a minus I am writing x 1 minus that z 3 x 1 plus 4 x 2 I will replace by x 3 minus x 4. So, in all together it will be a minus 3 x 1 minus 3 x 1 then your minus 4 x 3 and x 4. So, I repeat once again so our that this problem we first we have converted into standard LP problem maximization minimization of functions same as minimization of this function same as that minus z of this you minimize subject to this constraint which is converted into a equality constraint. And we have introduced when there is a this type of symbols are there equality type we need only once like variables which is greater than equal to 0. When we have this type of inequality constraint are there we need two variables one is surplus variable another is artificial variable both are greater than equal to 0. And equality sign right hand side if it is not a negative positive quantity we have to make by some means to make it positive. So, our problem is minimize subject to this constraint and all right hand side constraints are becoming greater than equal to 0. Now, and same also all variables artificial variable then your surplus variable and our design variable all are also greater than equal to 0. So, the problem minimize this one subject to this and you can write it and what is this x i is greater than equal to 0 where i is equal to 1 2 dot dot in our case we have come up to 7. So, I am writing first step is here minimize this one subject to constraints this constraints which is converted into equality constraints with x i is greater than equal to 0 i is greater i is 1 2 n on all right hand side that means b i is greater than equal to 0 our i is how many equations are there 1 2 and 3 i is equal to i can write it this i is equal to 1 2 3. Suppose, if you consider this is b 1 this is b 2 and this is b 3 then this i can write it. So, at this moment we know how to convert a LP problem into standard LP problem linear programming problem how to convert into standard LP problems. Suppose, you may have a some constraints like this way x 1 say x 1 is greater than equal to 5 less than equal to 5 greater than equal to 3. So, this I can easily convert into a two inequality constraints I can write it here x 1 is less than equal to 5 and x 1 greater than equal to 3 this and you know how to convert this thing in equality constraint similar way you can write it because this is a what is called this type of things you can it is enough to introduce only one slack variable. But here is this type greater than equal to x 1 is greater than equal to some expression greater than equal to some constant quantity. So, that you have to introduce two variables agree. So, one can convert the linear general linear programming problem into standard LP problems. So, and in your case if is a x 2 is unrestricted sign the x 2 is if is recollect x 2 is related to x 3 and x 4. So, let us see this one next that. So, now basically if you see this one minimize f of x subject to a into x is equal to b this is n cross 1 if the dimension of the variables design variables this and if you have a m equations are there then this is the basic equation with b is greater than equal to 0 and x is greater than 0. This vector greater than equal to 0 means each element of this vector of dimension m cross 1 is greater than equal to 0 similarly x also each element of that x greater than 0 means greater than equal to 0 x greater than equal to 0. Now, our problem is you see our we are looking for solution of this algebraic equation linear equation. We have to find out what is the value of x. So, that this constant are satisfied this are satisfied this of course, when we are putting in the standard LP problem it will has to satisfy, but this constant also for any value of x is not x value is non negative number. So, solve this one and simultaneously this value of x should give the minimum value of the function. So, that is our job this one. So, it is nothing but a solution of a set of equations. So, if you recollect your the basic linear algebra what is called solution of a set of equation it is a x this is a m this is n cross 1 this is a m this is a m cross n and we are assuming number of equation this m is indicates the number of equation is less than equal to n. So, we get different solution of this one can write it like this way if we form you see our this is the solution we want to do it. We form a matrix augmented matrix with our system matrix and right hand side vector. So, this dimension will be m cross n plus 1 this a. So, if you just recollect our basic results of solution of algebraic equation that is a rank of a star is equal to if rank of a is equal to r both the ranks are this is the augmented system matrix is r then which r is less than n then our conclusion is that we have a many solution multiple solutions. So, if the rank a is not equal to rank a no solution exist in other words you can say the system equation r inconsistence. In other words we can say the system equation equations are inconsistent that means some equation is equal to let us call b 1 and same equation is equal to b 2 is inconsistent between the two equations and the third one if the rank of this a star if rank of a star is equal to rank of a then we have a unique solution. That means in other words rank of this equal to this equal to n sorry n this n is the number of columns of this matrix. So, this will be when it is a symmetric matrix sorry not symmetric matrix it is a square matrix. So, you have a square matrix and the rank is full of that square matrix then you have a unique solution. So, this basically we are solving that type of equation now. So, in order to solve this one if you recollect in under graduate class that we are solving a set of algebraic linear equation by some numerical techniques. That means Gauss elimination method Gauss Jordan method all this thing Gauss Seidel method this type of thing and it is a Gauss elimination method is nothing but a row operations you are doing to eliminate some of the variables from the some equations. So, here also we will do the same thing here for the I mean. So, first in simplex method simplex method is nothing but a simplex method to solve a set of algebraic equation simplex method solve a set of algebraic algebraic equations by using the numerical techniques or it is called the iterative process. So, what you have to do first that you convert the system equation system algebraic equation into a canonical form what is canonical form that I will just discuss here. Suppose we have a system A x is equal to m cross n and this is n cross 1 is equal to b m cross 1 and it is the equation number 1 and m is number of equation is much less than the number of design variables as n. So, what I will do it we have a m equation is there in first equation we can consider that variable x 1 is there only in first equation variable x 1 x 2 x 3 may be x n that I have just writing this one this is the this is the standard equation is there a 1 1 x plus a 1 2 x 2 plus dot dot a 1 n x n is equal to b 1 then a 2 1 x 1 plus a 2 2 x 2 plus dot dot a 2 n x n is equal to b 2 and if you continue we have a m set of equation is there m 1 x 1 plus a m 2 x 2 plus dot dot plus a m n x n is equal to b m. So, you say in all equation in general x 1 x 2 all these things there. So, what will do it in canonical form representation canonical representation of set of equation will keep x 1 only in equation number 1 and x 2 only in equation number 1 and will keep x 2 only in equation number 2 no other equation x 2 will be there. So, in this since we have a m equations are there we can make it m variables x 1 x 2 up to m x m this variables are present only in one equation only x 1 will be involved in one equation no other equation x 1 will be involved x 2 involved in this equation let us call two equation it will not involve in other equation similarly x m involved in lust equation and other equation x m will not be there. So, this I can do by elementary row operations how will do it I multiplied by this first you say a 1 divided by a you divide this equation by a 1 1 then you multiplied by that equation by a 2 1 subtract from equation 2. So, x 2 is out similarly multiplied by first equation by a 3 1 subtract from third equation that x 1 will remove from this one. So, in this way similarly x 2 will be only in this equation. So, x 1 is not there. So, you divide by this one what is the coefficient is divide by that one you convert into a only coefficient of this one coefficient associated with x 1 x 2 up to x n is 1 all coefficient. So, you eliminate x 2 from equation number 2 3 dot dot and so on you continue up to variable x m. So, after doing this elementary after doing elementary row operations one can write this above equation like this way x m the dimension of x is equal to m cross this dimension I am writing this is x m I am writing the dimension is m components are there x 1 x 2 dot dot m components. So, I am writing if you are writing this one it indicates m component m cross 1 then this remaining variables remaining variables will be your q is a matrix and that variables will be n minus m into 1. So, this or you can just this equal to b bar and that is dimension m cross 1. So, you see first equation or you can say it is nothing but I multiplied by this I dimension is m cross m this you can write it also. So, mind it our this or you can use some of the symbols if you write our x m is our variables are x 1 x 2 dot dot x m agree. And if you just put it let us call I put it to just to differentiate this one x bar m then it is cross 1 is x bar then this x bar then this and you know x bar is what this one then x bar n minus m cross 1 what is this one x of m plus 1 x of m plus 2 dot dot x n whose dimension is your n minus m into 1. So, this I can write it into this form. So, this I can write it into this form. So, this representation is called canonical form representation once you get it this canonical form representation you see we have n variables are there and whereas we have a n variables are there, but we have only m equations are there. So, you can get is infinite number of solutions let us say how we are telling the infinite number of solution of this one. So, at the moment I can say if you solve this equation 1 equation what or equivalently equation 2 if you solve it you have a either you solve the equation 1 or 2 which is a equation 2 is canonical form solve it one can solve if you solve one can solve one can solve equation 1 and obtain the following solution and can obtain the following solution. What is the solution one is I may get that unique solution when I will get it unique solution when any linear equations at when the order of this order or you say rank of the matrix is n and full rank then you will get unique solutions unique solution unique solution when you will get it when m is equal to n and rank of a is equal to n and 2 you have a infinite solution many solution infinite solution and I told you when you will get this infinite solution of this one when rank of a star is equal to rank of a is equal to r which is less than small n. That is this conditions if you see this one that condition many solution infinite then you have a no solution and then you have a unbounded solution. There are 4 types of solution exist with type of systems now note that that our equation if you just look at this expression a x m cross 1 sorry n cross 1 this is m cross n is equal to b m cross 1 and we are considering m is less than n not equal to n then we have seen this one that number of equation number of equation number of equation in this system is number of equations r is m and number of variables is n. So now you see this one if this matrix is a full rank this is a rectangular matrix if it is a full rank this then we can make it what is called that n of m number of linear equations linearly independent combination we can make it that means this is n factorial n divided by factorial m n minus m. So, how many columns are there in a matrix whose dimension is n columns are there out of m columns there is m columns are linearly independent if it is a full rank of matrix n then we have a n columns are there small n columns are there out of this small n I am picking up m linearly independent vectors. So, we can get it such type of number of linearly independent sets of that one. So if you now recollect that one you see this one our canonical form if you see this is our canonical form representation our canonical form representation I will write it here that is i into x bar of m. So, 1 is q x bar of m cross 1 is equal to b bar this is we have seen how to do it. So, from now onwards I am telling that as if I have a system itself is a canonical form m this is the m this is the m variables are there and this is the this dimension is n minus m cross 1 dimensions. Now, you see if you assign this n minus m variable assign is 0 then is values is nothing but a b. So, how many because you have a n variables I can I can select n minus m variables in what combination ways that you have a total variables as m and out of this I am taking is m variables which is not equal to 0 and remaining is 0. So, how many solution we can get it let us call I consider this is equal to 0. So, I will get it one solution I can consider another combination one variable from here it has gone to here and one variable is coming here. So, then there will be another solution. So, if you assign that our now our variables we can split up into two parts one is basic variable another is the non-basic variables and non-basic variables if how many non-basic variables are there there are n minus m non-basic variables are there. How many basic variables are there is a m basic variables are there when we will assign the non-basic variable is 0 the solution what we will get it is called the basic variable solution. So, if you see this one that the total number of you can the total number of variables that m b the m b the rank of a which is the full rank and every set of m columns m column vectors is linearly independent linearly independent. So, this tells us how many variables combination of linearly independent vectors will get it this determines the number of m basic variables with the variables and n minus m non-basic variables. So, that means, if we assign the n minus 1 variable value something we will we can get the remaining m variables by solving m equations because I told you there are n variables are there m equations there. So, you have to assign n minus m variables value and you can which in turn you will get it what is called m equations. Now, then m m equations m m variables are there what is m variables x 1 x 2 x m x m or any variables may be from x 1 to x 1 which is n x 1 x 2 x 2 x 3 x m m variables of x will be unknown that can be obtained by solving the set of equations. So, when you will assign thus this non-basic variable value is 0 then solution is called is a what is called basic variable solutions. So, total number of solution we have total number of solution we have total number of basic solution we have n m is equal to factorial n factorial m n minus m. So, let us see what is and this solution you say this solution if you have a m equations are there n variables are there basically we have as this number of solution are there out of this solution out of this solution some may be feasible solution some may not be feasible solution according to our optimization problems agree. So, let us take one example what how will you find out this solution of this one minus x 1 plus x 2 is less than equal to 0 twice x 1 plus x 2 is less than equal to 0 plus than equal to 2 x 1 is greater than equal to 0 x 2 is greater than equal to 0. So, let us call this two equation and this two equation I can convert into a what is called equality equality equation that means minus x 1 minus x 1 plus x 2 and this quantity is less than 1 that means I have to add some new variables let us call x 3 is equal to 1 and this is 2 x 1 plus x 2 this is less than this one then I have to add some other variable let us call x 4 is equal to 2 and we have a given the x 1 value x 2 value is greater than 0. Now, you solve it this equation and there is no restriction I put it in x 3 of course, this x x 3 is greater than 0 x 4 is greater than equal to 0 because this some positive quantity I am adding some positive quantity. So, in turn that all variables are greater than equal to 0 if it is why if it is why if it is violates this condition you will get a some solution with which violates this condition any one of this then it is not a feasible solution and it is not a satisfying this constraints. So, let us call how to solve this one. So, you have a three we have a four unknowns two equations are there. So, I can easily assign two equation values that two variables value what is called equal to 0. Let us call I assign x 1 is 0 x 2 is 0. So, I can get it here what is called x 3 is 3 x 4 is 2 and this is one of the solution agree and other value solution is what that x 1 is 0 I consider x 2 0 and x 3 0 all these things we have considered. So, if you just consider our this is one choice of we got it 0 then if you choice if you do the choice like this way let us call x 1 is 0 agree x 1 is 0 then you what will get it this is the and you see we have a x 1 if I put it 0 then are three unknowns are there any one of this you can put it 0 let us call you put it that x 3 is 0 then you will get another solution. So, in turn if you take the four variables are there two equations are there you assign two variables randomly anything any combination you assign 0 other variables you can get it. So, in our case now n is equal to 4 number of variables are 4 and number of equation is 2. So, this what is the solution how many solution will get it for this one first is how many solution you see factorial 4 2 this is m n minus m factorial 2. So, in total I will get this 4 that means it is 3.2 is a 6 solution, but it is not necessary that 6 solution will give you the will satisfy this equation and satisfy the constant also, but we will get the 6 solution you can try it here you will get 6 solution I can tell you what is the 6 solution I took let us call x 1 is 0 x 2 is 0 then one solution then I take x 1 0 x 3 0 then we have a x 2 and x 4 solve it you will get another solution. If you have a x 3 x 2 is 0 x 3 is 0 x 1 and x 4 another solution in this way that means any two variables you have to assign 0 and remaining two variables you find out and in this way you can get number of solution n n c m that means two variables at a time you are as what is called the if it is m m variables at a time you are grouping to get the solution of this one from out of n variables. So, in this way the if you see this solution I just I will not solve I just mention it the solution of this to find all basic solution please remember the basic solution I will call of that equation once you convert into a what is called canonical form canonical form because we have a in this case we have a two equations are there. So, I will if I convert let us call x 1 it present in equation number one it should not be present in equation number two similarly x 2 present in equation number one it should not be present in equation number two. So, our x 1 and x 2 are canonical form agree and you can consider this variables also canonical form because x 3 present only in equation number this x 4 is visibility equation number four only equation number two this also canonical form initially. So, in this way I can make it six combination of this one. So, now total to find the basic solution we take any two variables as basic variables any two variables we take basic variables means that two variables in this is two equation that two variables present only in one equation if it is a x 1 you take as the basic variable x 1 will present in first equation and x 2 you take another basic variable is present in second equation if you take x 2 and x 3 then x 2 you can think of x 2 present in first equation x 3 present in only in second equation in this way that we have a two variables at a time I am taking the basic variables. So, you have we will get a solution of find the basic solution we take two basic variable we will get the basic variable solution set. So, our solution we have we will have a solution one third four three zero zero this is x 1 you can say x 2 this is x 3 that x 4 this is x 3 that x 4 this is one solution you can get it please you can check it also another solution zero one zero two zero another solution I am getting is minus one zero zero four another solution I am getting it zero two minus one zero and another solution is zero one zero one and another solution is zero zero one two. So, you see we have a four variable two variables at a time two variables at a time we assign zero and remaining two variables we can find out by solving the two equations. So, we got it this this this and out of the six basic variables out of the six basic variables as a six solution you say that there are four feasible solution is there this is not valid this is cross this we cannot accept because our x 1 x 2 dot x 3 x 4 all are greater than zero, but it is showing x 1 is negative similarly this is not acceptable this is x 3 is negative. This is also not acceptable. So, we have a our basic solution we have a six out of six I told you in the beginning that when you are discussing that we have a basic solution again it does not mean all the basic solution will give you the feasible solution feasible solution means that this solution must satisfy our what is called equality constraints. And also our that constraint that x x i is greater than zero it in it may not satisfy also and if it is a feasible solution then it satisfies all the constraints and the all side what is called constraints also. So, out of the six basic variables we got four is the feasible solution four we have a feasible solution. So, what is the basic variables then basic variables is the set of variables we have a set of variables out of this thus if you get you have a what is called that set of variables is there in the basic solution in the basic solution the corresponding variable which is non zero the corresponding variable which is non zero they are called as basic variables. So, basic variables the set of variables in the basic solution in the basic solution that have set of variables in the basic solution that have non zero values non zero values are called basic variables. So, basic these are and the variables whose values are all zeros then we will call that variables are called non basic variables. So, I told you that if you have a n variables are there m equations is there we have a m basic variables and we assign the non basic variables value non basic variables value is zero and immediately we will get the what is the basic variable solution. And we have that type of combination we have a n c m that is what we have seen n factorial m factorial m n minus m factorial. So, what is the basic feasible solution you have a basic solution out of this which solution are feasible then it is called basic feasible solution. And next is optimal solution you got a base what is the feasible solution out of this feasible solution which solution will give you the optimum value of the function it is called the feasible optimum feasible solution. And non degenerative solution basic non degenerative basic variable solution non degenerative basic variable solution what does it mean you got the feasible solution of this one. If the basic variable if the basic variable values are all non zero then it is called non degenerative basic variable solution. So, next class we will just discuss how to solve the l m what is called the linear matrix equation in the matrix form. And then we will solve in tabular form.