 first we have discussed that from the description of the problem how one can formulate the multi objective optimization problem in mathematically. Then we have discussed the what is called the primaries of vector calculus, the function is given which is a multivariable function how to find out the gradient of a vector. And simultaneously we have found out the gradient of a vector if you once again differentiate with respect to x then we have seen it becomes a matrix which may matrix is a symmetric matrix. And its dimension is n cross n where the n is the number of variables involved in that function and that matrix is a symmetric matrix we have seen. And with a example we have seen how to find out the gradient of a vector today we will just discuss how to find out that Hessian matrix that means if you have a differentiate that function twice second order differentiation you do it you will get a Hessian matrix how to compute that matrix. So, let us take one example that example same example what we have considered last class for a to find out the gradient of a vector. So, the example calculate the second derivative of the function f of x is equal to x 1 square plus twice x 1 x 2 plus 3 x 2 square plus 4 x 1 x 2 x 3 plus x 3 square calculate the second derivative of the function at point x is equal to x star here the dimension of x is 3 cross 1 that means this has a x 1 x 2 and x 3 and this values are given 1 to 0. So, in order to find out the Hessian matrix of this function that means the gradient of this function once again if you differentiate then this is called the second derivative of this function for a multiple derivative function. So, that can first you have to find out the gradient of this function f of x that last class we have found out f of x value at x is equal to x star first you let us call you find out the f of value f of x gradient of this one is nothing, but a del f of x with respect to x 1 then del f of x with respect to x 2 then you have to find out the del f of x with respect to x. Since we have a three variables are there function is a this function f of x is a function of x 1 x 2 and x 3. So, this if you differentiate this function with respect to x last class we have seen it is 2 x 1 recall this expression x 2 plus 4 x 2 x 1 then differentiate of f of x with respect to x 2 that will come 2 x in square plus 2 2 x 1 plus then 2 x this is x 1 with respect to x 2 this is sorry this is 2 x 2 sorry this is 2 x 1 then 6 x 2 then 4 x 1 x 3 and then if next is if you differentiate f of x with respect to x 3 then is a function of this 2 terms a function of x 3. So, it is a 4 x 1 x 2 plus 2 this is a this is a cube not square please see recall the last x function it is a 3. So, 3 x 3 square so this. So, this function you have to once again you have to differentiate with respect to x that means vector this gradient of this vector once again we differentiate with respect to x. So, if you differentiate this with respect to x that is is denoted by this symbol f of x. So, this one we have seen it is like this way. So, del f of x del x 1 square del f square of x del x 1 del x 2 and del square f of x del x 1 del x 3. Similarly, this is del square f del x 1 and del x 2 this will be del square of f x x this is del x 2 square del square of f x then del x 2 del x 3 and last of this one is then del square f of x is equal to del x 1 of 3 del x 3 then del square f of x is del x 2 of del x 3 then del square of x is equal to del x 3 square. So, this is a 3 by 3 matrix and this is a symmetric matrix because a 1 2 position a 1 2 position is a 2 1 a 1 3 position a 3 1 and a 2 3 position is a 3 2 these are all identicals this out diagonal positions of this one. So, one has to calculate since we know the if you see this one one can calculate this one we know del f of x with respect to x 1 that value we know this value is 2 x 1 square 2 x plus 2 x 2 plus 4 x 2 x 3. So, once again you differentiate this with respect to x 1 that value will be coming 2 only. Similarly, you differentiate this del of x del x 1 already you have differentiate with respect to x 2 then it will come 2 then what is this with respect to this is 2 plus 4 it will come 4 x 3 then again you can differentiate del f x plus del x 1 del x 3 this one with respect to x 3 you differentiate that will come your 4 x 2 next is we know that del f of x in del x 2 expression this is the expression with 2 x 1 plus 6 x 2 plus 4 x 1 and x 3. So, you differentiate del f of x with respect to del x 2 square. So, differentiate with respect to x 2 so that will come if you do 6 then del square f of x del x 2 and del x 3. So, differentiate this thing with respect to x 3 so that will come 4 x 1. So, only left is now differentiation of del f of x with respect to x 3 we know this expression is 4 x 1 into x 2 plus 3 x 3 square. So, now you differentiate once again this one with respect to x 3. So, this will be del f of x del x 3 square will be equal to 6 x 3. So, put these values in the that matrix this one and find out this matrix value. So, you have already calculate these values all partial derivative second order partial derivative it is all this element we have calculated earlier. So, now if you put these values there then our matrix which will call Hessian matrix of del square of x is coming is 2 2 plus 4 x 3 plus 4 x 2 2 plus 4 x 3 plus 6 4 x 2 x 1 then 4 x 2 4 x 1 and 6 x 3. So, now you have to find out the Hessian matrix at a point that means we have to find out the Hessian matrix that matrix at point x is equal to x star and that matrix dimension is 1 3 cross 1 and that is given value is x 1 is 1 x 2 is 2 that is 0 0. If you put in this expression that is 2 as it is this x 3 is 0. So, this will be 2 then x 2 is your 2 that will be 8 then x 3 is 0 2 6 as it is then x 1 is your 4 this x 2 value is 2 8 that is x 1 is 1 4 then x 2 is 0 this one. So, this is our Hessian matrix this matrix is Hessian matrix that means if you differentiate gradient of a vector once again with respect to x that matrix you will get a Hessian matrix and then that matrix for this particular example when you use x the value of at point x result x star is 1 2 3 this is our Hessian matrix and it is a symmetric matrix of this one and symmetric you can write it. So, next is before we go to this our what is called in order to find out the optimum value of this function which is a function function is a several values of x 1 x 2 dot dot x n that is called the function f of x is a function which is a function of x 1 x 2 dot dot x n. In order to find out the optimum value of the function at a point then we must have few more mathematical preliminaries. So, that is next is quadratic function. So, let us call that we have a function with two if we have a function we have a function two variables x 1 and x 2 x 1 and x 2 the quadratic form of the function is of the function is a x 1 square b x 1 x 2 plus c x 2 square it can be a in addition to the constant term plus the linear terms. Let us call for the time being is this one it is a quadratic form quadratic function. Then this quadratic function our problem is the this is the quadratic form. So, this quadratic form one can express into a matrix and vector form. So, let us call this is equation number 1. So, one can express the equation 1 one can express the equation number equation 1 as like this way f of x equivalently one can express like this way this equal to x 1 x 2. We have considered for the two variable case then we extend for n variable case this we can write it like this way you see this what is the coefficient of x 1 square is a that will write in a 1 1 position of the matrix which are going to form. So, that is is a then a 1 2 position is will be your this matrix a 1 2 position means 1 2 position coefficient is b that we can write it here then 0 then what is the coefficient of x 2 square that will write it a 2 2 position that means c into x 1 and x 2. So, if you just do the operation matrix vector operation then whatever the results will get and with the row operation with this one will get exactly what are the function is given this is one one choice or one can write it like this way x 1 x 2 then a 0 b c the a 1 2 position or a 2 position I have just written here this one and this is x 1 and x 2 and also this if you do the operation you will get exactly whatever the function is getting. Now, one can write also like this way there are infinite number of choices are there to put that matrix this matrix. So, I can write it here because x 1 square coefficient is a. So, that will remain in a 1 1 position x 2 coefficient x 2 square coefficient is c that will remain a 2 2 position and this is x 1 x 1 x 2 coefficient and this is also one can write it x 2 1 position that this x 1 coefficient x 1 and x 2 coefficient can be distributed with this of diagonal things that some must be equal to the coefficient of x 1 and x 2. So, I can write it also like this way there is no problem if I write it minus b 2 b and c is like this way. So, you see if I write this expression it will a 1 a x 1 square then c x 2 square then I will write it here minus b x 1 x 2 again there is a 2 b x 1 is 2. So, if you simplify it will come only b x 1 x 2. So, this is one the most of the cases it is selected the matrix p is or if one can write it in symmetric matrix form x 1 x 2. Now, see how I am writing in symmetric matrix form this one coefficient of x 1 and x 2 is b the half of this b by 2 I will write in 1 2 position and b by 2 I will write it 2 1 position. So, it will be a x 1 square a x 1 square coefficient is a x 2 square coefficient is c then coefficient of x 1 is b b I divided into two parts to make the matrix symmetric. So, it will be a b by 2 and b by 2. So, you can easily realize that we have a infinite number of choices, but this choice has a I mean most popular choice of this one because we can conclude something from if the matrix is symmetric matrix regarding the function. So, our conclusion is p is not if you consider this matrix is p or this is also p this matrix this matrix also p this matrix of p p is not unique matrix to represent the function quadratic function into this form. What form this I can write it if you consider x 1 x 2 is a vector form you see here I can write x transpose this is vector. So, it is row vector. So, x transpose I can write it that p into x this is 2 cross 1 for this example. Similarly, I can write x transpose, but here p is different this form here also I can write x transpose p x of t, but p is different, but all these cases it is same as equivalently same as function of x 1 and x 2 and in this case p is symmetric matrix in this case p is symmetric matrix is not unique. So, let us see that in general we have seen that out of the for this example out of the 4 p matrices which will give the exactly same value of expression of f of x 1 and x 2 out of this one is we have considered the symmetric matrix choice. So, in general for n variables function is a n variables f is a function of x 1 x 2 dot dot n x n a quadratic form of n variables n cross 1 which is a n quadratic form of it n variables. What we can write it f is a function of x 1 x 2 dot dot x n which in vector form we can x of n whose dimension is n cross 1 is I can write it this is equal to x transpose p x where x is equal to x 1 x 2 dot dot x n whole n is equal to x transpose means it is a column vector of dimension n cross x is a dimension n cross n. Suppose the p is not symmetric suppose p is not symmetric suppose p which is n cross n this is n cross 1 this dimension will be n cross n. Suppose p is not symmetric matrix one can always suppose this one f of x just now we have written x transpose p n cross n into x whose dimension n cross and p is not symmetric. And look this expression this is a scalar quantity the function is a scalar quantity this. So, I can always write there is no harm I can always write that x transpose p cross this x and half x transpose n cross n. Suppose p x because suppose this constant it is 10 I can put it here 5 here 5. So, it is a 10 ultimately scalar. So, this quantity is scalar. So, this further we can make it after some manipulation of this one I can write it p into x plus since this is a scalar quantity then I can take the transpose of that one. So, this x transpose p x that transpose here since I have considered it is not a symmetric matrix that will be a p transpose note how I have written this one. If a is a matrix of dimension m into p b is a matrix of dimension proper dimension p cross let us call n c is a matrix whose dimension n cross r. So, we can multiply a b c this you see they are with proper dimension if this is this then this product if I result resultant of this matrices product of this matrices after multiplication. If you take transpose this is we can write c transpose reverse order you write c transpose b transpose a transpose and this property I have used it here to take the transpose of that one taking the transpose of that one or you can say just you can forget about this I am now taking the transpose of that one. So, this equal to now I can write it x transpose p x plus half this is half is there half then x transpose you can say this is a this is b this is c c transpose may x transpose then b transpose p transpose that this transpose of a transpose that means x. So, if you make the simplification of this one half you take common then you can write x transpose common then you write p of p transpose by 2 sorry 2 you push it inside then x. So, this matrix is a p is a that is our non symmetric matrix take transpose of this one and divided by 2 and this matrix we know if a is a matrix non symmetric matrix if add with a transpose result is a symmetric matrix and we are dividing by scalar by quantity 2. So, all elements of p plus p transpose will be divided by 2 only. So, this is a symmetric matrix and you see this we have derived from this expression. So, x transpose p x if p is not symmetric is not symmetric we can write it this one x transpose p plus p transpose by 2 into x where this matrix is symmetric this matrix is symmetric. So, whatever the results we will get it again the expression in terms of what is called f of f function f which is a function of x 1 x 2 the same expression we will get it here also that may x transpose p plus p transpose by 2 into x same, but only the advantage is we are getting this matrix is now become a new matrix which is a symmetric matrix, but scalar this function value scalar value expression in terms of x 1 will be same expression as this one. So, this is our conclusion of this one that even if p is not symmetric I can make the x transpose p plus p transpose by 2 into x is an expression whose values are same as this values in terms of x. So, let us call now we define some of the function positive definite matrix definition of definition of positive definite quadratic function. So, let us let us call we have a nth order quadratic function is there f of x is a function of n variables which is in general we am writing p 1 1 x 1 square p 1 2 x 1 x 2 p 1 2 x 1 x 2 then p 1 3 x 1 square x 1 x 3 and dot dot we have a n variables are there p 1 n x 1 x n plus p 2 1 let us call this is x 2 x 1 p 2 2 x 2 square p 2 3 x 2 x 3 plus dot dot p 2 2 x 2 x 2 x 2 x 2 x 2 x n and in this way if you consider the all these things last term of this one will be p n n x 3 x n x n square again which we can write it into matrix and vector form is a matrix and vector for this dimension n cross n and this dimension n cross 1 where p is c clearly p 1 2 p 2 1 is not a symmetric matrix. These two things are not same a 1 p 1 3 and p 3 1 third equation will come is not a same quantity. So, you will get a non symmetric matrix non symmetric again. So, our definition of positive definite matrix is that first the function f of x which is equal to x transpose p x n cross 1 this is said to be positive definite positive definite bracket when you will read the bracket we will read all in bracket terms or we will get positive semi definite definite quadratic function. The function will be said to be a this function f of x means this function which you have written into matrix form matrix and vector form is said to be positive definite quadratic function. If f of x is equal to x transpose p x is greater than 0 or bracket term will be said to be positive semi definite when it is greater than equal to 0 for every x and x is not equal to null vector. When x is equal to null vector x 1 is 0 x 2 is 0 this function value is 0. So, this will be so I repeat once again the function is said to be f of x is said to be positive definite function. If f of x transpose x that means f of x function value will be greater than 0 for all values of x except x is equal to if it is not null vector always 0. If it is a semi definite the function value may be 0 or greater than 0 for every value of x which is not null vector if it is x is null vector. So, this is the definition of this one then second definition of this one that function is negative definite. Similarly the function f of x is equal to x transpose p x which is a this is said to be negative definite or negative semi definite definite quadratic function if f of x is f of x is a function of x 1 x 2 which can be expressed in terms of matrix and vector form x transpose p x agree this value will be if it will be negative definite this value function value which is a scalar is always less than 0. If it is a semi definite this function value will be less than equal to 0 it function value may be negative or 0 this. So, there is a negative for every x which dimension n cross 1 that may n cross 1 is not equal to 0. Now, this function may be a indefinite that means a function is said to be indefinite when this x function value can be greater than 0 or less than 0 or may be 0. So, we cannot say anything that for all values of x which is not equal to 0 the function value is either positive negative or 0 it can be anything. So, that is called what is called function is indefinite the function f of x which is equal to x transpose p x is said to be indefinite quadratic function function. If f of x is x transpose p x is greater than 0 for some value some values of x agree which is not equal to 0 null vector. And f of x which is a x transpose p x is this value is negative for other values of x for other values of x which not equal to. So, this is called indefinite quadratic functions. So, we can see next. So, now we know the what is a positive definite quadratic function negative definite quadratic function positive definite positive semi definite quadratic function and negative semi definite function. Along this things in the same line what is called definition of positive definite matrix it is just link with this only definition of positive definite matrix. So, let us call for a p which is n cross n matrix is positive definite matrix which is n cross n matrix is positive definite matrix if and only if. If you multiply it by p matrix agree whether p is a symmetric matrix or non symmetric matrix whether it is a positive definite matrix you multiply it by p matrix with proper dimension pre multiply it by a row vector post multiply it by a column vector of same. So, it is a column vector post multiply column vector and pre multiply it by same vector which is row vector agree. So, this if this quantity is greater than 0 when x is not equal to this then we will call the matrix is positive definite. But see this is a this is a become a quadratic function now x is this. So, what is this means if p is a positive definite matrix you pre multiply by x transpose any vector x and post multiply it by x. If this quantity is greater than 0 for all values of x except x is equal to null vector then we will call matrix is symmetric matrix agree. So, next is positive definite you can say this if p is equal to p transpose that means p is a symmetric matrix then we will call that this p is a symmetric matrix and that symmetric matrix is if it is greater than 0 and one thing is there p positive definite in short it is written like this way p greater than 0 means p is a symmetric matrix p positive definite in short it is written p greater than 0 means p is positive definite matrix. So, if it is a p transpose greater than 0 it indicates that p is a symmetric matrix, but this does not indicate p is a symmetric matrix p may be non symmetric matrix also. So, next is the definition of positive semi definite matrix or positive semi definite matrix or negative definite matrix or you can say negative definite matrix definite matrix definition negative matrix is a same for a function p whose dimension is n cross n is negative definite matrix that means in short it is written p less than 0 is I will read as a p is a negative definite matrix if and only if that x transpose p of x is less than 0 when x is not equal to null vector agree that this is not a null vector. So, again this is a quadratic function and p may be a symmetric matrix may not be symmetric matrix, but that p is if it is a non symmetric matrix or symmetric matrix is a negative definite if this x transpose p x which is a quadratic form in terms of variables x x means is set as a n components are there x 1 x 2 dot dot action if this value will be less than 0 for all values of x accept this one. So, next is definition of positive semi definite again if p is positive semi definite positive semi definite semi definite in mathematically it is written positive semi definite p is greater than 0 I will read it p is positive semi definite if and only if x transpose p x that is a scalar value this value will be greater than equal to 0 when x not equal to a null vector. So, this is the definition of that one again is there we can also the definition of negative semi definite matrix again definition of negative semi definite matrix negative semi definite matrix. So, the matrix p n cross n is said to be negative semi definite matrix in short it is written like this way will read p is semi definite matrix negative semi definite matrix sorry if and only if the quadratic form that means that matrix you pre multiply and post multiply p multiply x transpose and post multiplied by x x is a vector of dimension n cross 1 if this values this values is a square value is either negative or equal to 0 this for when x transpose p x is equal to 0. x is not equal to a null vector x dimension is n cross n c. So, this is that now question is coming that there are infinite number of vectors are there exist then how we check this thing whether the matrix p is a positive definite matrix which in turn that x transpose p x is positive definite quadratic function or not. So, there is an infinite number the test for now the test for positive definite matrix and this is done for Sylvester criteria, Sylvester criteria criterion and this is valid Sylvester criteria you can apply only if matrix p is symmetric matrix matrix this is for if p is symmetric matrix then only you can symmetric matrix. So, a matrix is positive definite or not that test one can do by using Sylvester criteria again you can do using Sylvester criteria provided this matrix is symmetric matrix and according to the dimension according to the definition of symmetric over the positive definite matrix we know that x transpose p x transpose p must be x transpose p x transpose p must be x transpose p positive definite matrix this must be greater than 0. Now, if p is given p is may not be known what is called symmetric. So, we can always express that one I just shown you we can always write it this one x transpose p by 2 x transpose p plus p transpose by 2 x whatever the value will get it this one and this value are exactly same. So, in other words that if p is not symmetric matrix will convert into a this form p plus p transpose by 2 and then test with this matrix which matrix is symmetric that p plus p transpose by 2 I am denoted by q. So, I am testing with this matrix q matrix if q is symmetric in turn I can say this function if q is symmetric x transpose p greater than 0 will be there by using the Sylvester criteria test I can find out whether q is positive definite or not if positive definite then this will greater than 0. So, how to check positive definite matrix that q. So, from now onwards I will consider our p is a symmetric matrix. So, Sylvester test for positive definite let me say p n cross n is symmetric matrix even it is not symmetric I will convert into a p plus p transpose by 2 that one I will consider as a p which is a symmetric matrix symmetric matrix. Then Sylvester theorem tells if it is a symmetric matrix p that p is positive definite provided first check is all the diagonal elements of p must be positive all the diagonal elements that means p i i i is equal to 1 2 dot n must be positive and p i i is equal non zero elements this is the first if this is there all diagonals of p is like whatever the p matrix is given convert into symmetric matrix like this way p plus p transpose by 2 and then check all the diagonal matrix are positive one and non zero if it is so further you proceed like this way all the leading all the leading leading principle minors means determinant must be positive. So, let us note what do you mean by the leading principle minors the leading principle minors the leading principle leading principle minors minor of order k of n n cross n matrix. Is obtained by deleting by deleting last mind it last n minus k k is the order n minus k rows and columns last n minus rows and columns again. So, this is the leading principle minors. So, let us example take an example and check how this test can be done whether a matrix is positive definite matrix or not. So, if you consider this one example this determine the nature of the quadratic function the nature of the quadratic function means whether the function is positive definite negative definite or positive semi definite or negative semi definite this nature you have to know. So, that function is given once again I can write into this general form which is a function of your three variables x. So, I am writing dimension x dimension is the so 7 x 1 square plus 4 x 1 x 2 plus 10 x 1 x 3 plus 5 x 2 plus 8 x 2 x 3 plus 9 x 3 square and where we have considered x is equal to a vector consist of x 1 x 2 and x 3 7 x 1 square 4 x 1 10 x 1 x 3 5 x 2 plus 8 x 2 x 3 and 9. So, this thing I can easily convert into a matrix and vector form that you would. So, x 1 x 2 this are there then this p matrix this is the word this you have to filled up from this information I told you the x 1 square is 7. So, it will go in 1 1 position first I will diagonal elements I x 2 square is 5. So, it will come here x 2 to position x 3 square coefficient is 9. So, it will come 3 3 position now you say x 1 x 2 product of x 1 x 2 agree. So, you can put it here 4 here 0, but problem is I want this is a symmetric matrix. So, that I can test the Sylvester inequality condition whether p is this p matrix if it is symmetric this value will be greater than 0 provided p is positive definite. So, this I will equally distribute between the a 1 2 position and 2 1 position. So, 4 2 2 equal. So, that it will become symmetric then 10 what is the x 1 x 3 1 3 position 5 is here and 5 I am giving here 1 3 3 1 position. Then your 2 3 position 2 3 position is 8 this is 4 and this is 3 2 position 4. So, this is our p matrix and this p matrix is a symmetric matrix. So, if we can show if p is a symmetric matrix if you can show the p is a positive definite matrix means p greater than 0. If you can show positive definite matrix then this function value is always greater than 0 for all values of x except x is not equal to null vector. So, let us say by using Sylvester inequality Sylvester criteria then what you have to find out first we have to find out the leading principle minor of order 1 leading principle minor of order 1. Then what is this then our how many rows and columns we have to delete from the last row you see I have written n minus k n is equal to universe 3 and order is k order is 1. So, I have to delete 2 rows 2 columns from the last rows 2 rows and 2 columns. So, only this elements is left. So, that is 7 order is 7 and that is greater than 0 next is leading principle minor of order 2 k is equal to 2 here. Here is k is equal to you can write k is equal to 1. So, this is what then n minus k n is equal to 3 k is equal to 2 this is 1 last row and last column you delete from the matrix p last row last column. So, you have only this matrix this matrix is if you see this matrix this will be a 7 last row last column if you delete it then your matrix is determinant of that one determinant of that 7 2 determinant of this matrix 7 2 2 5 and that determinant if you see this is 35 minus 4 is equal to 31 which is greater than 0 and last because if an order is n I have to consider up to nth order minors. Leading principle minor of order k is equal to 3 that means it indicates n minus k n is equal to 3 k is equal to 3 this is no rows no columns you have to delete. That means you have to take the full matrix what is p is given. So, determinant of that determinant of that matrix you have to find out this one that matrix is 7 see this one 7 2 5 2 5 4 5 4 9. So, the determinant is you have to find out. So, that below if you find out this determinant you will get this determinant value is you know the how to find out the determinant value is greater than 0. So, according to Sylvester theorem if you see this one he is telling first what is the matrix p is there if it is a symmetric matrix only our case we have formulated this thing in the symmetric matrix check it this one that all the diagonal elements are positive 1 or this all are positive then proceed for the all leading minors must be greater than 0. So, all leading minors is order 1 is got 7 greater than 0 order 2 is we got greater than 0 order 3 is also greater than 0. That means what you can say this we can say the p is a positive definite matrix once p is positive definite matrix we by definition we know x transpose p x transpose p x is always greater than 0 this x transpose p x is always greater than 0. So, this function is a quadratic function is a positive definite matrix similarly we can go for what is called positive semi definite. So, next class we will discuss positive semi definite and negative definite and negative semi definite how to test using the Sylvester criteria which Sylvester criteria agree. So, thank you.