 Good morning. From this lecture, we start the third module of our course, which is on numerical analysis, numerical method. So, in this lecture, we will concentrate on solution of equations and their systems. And next lecture onwards, we will start our study of numerical optimization technique. So, first we start with polynomial equations. The fundamental theorem of algebra tells us that a polynomial of nth degree has exactly n roots. And with those roots x 1, x 2, up to x n, you can factorize this polynomial in this manner. Now, in general, the roots are going to be complex. Now, as a special case, it may happen that you get real roots. However, with real coefficients, if complex roots appear, then they always appear in conjugate pairs. So, this is one important piece of information which helps us in many of the methods to solve such polynomial equations. So, this is a polynomial. This polynomial equated to 0 gives us the corresponding polynomial equation. So, we can talk of the roots of the polynomial or the solutions of the corresponding equation. Now, in the equation solving process for polynomial equations, there is a lot of theory which helps us or which guides us in several different manners. One of them is Descartes rule of signs and then the other is bracketing and separation of different roots. And then there is a process of synthetic division in which if we have a polynomial p x and we have another polynomial q x of lower degree, then with q x, we can divide the polynomial p x and the quotient is f x and the remainder is r x. Now, f and q, they are interchangeable. If you take f x as the divisor, then q x will be the quotient and r x is the remainder. So, the remainder is a polynomial of a degree lower than the divisor by which you try to divide the original polynomial. It is exactly like ordinary arithmetic division. Now, one more important point is the multiplicity of a root. All the roots need not be different. There can be some roots which are repeated till they are counted in this n root. So, for a cubic polynomial, the roots could be 2, 3 and 3. So, they are counted as 3 roots if 3 appears twice in this factorization. Now, the simplest polynomial equation that you can think of is of course, linear equation which is extremely simple. So, from the next step onwards, the thing becomes slowly more and more difficult. For linear equation, you have got quadratic equation which all of us studied in school. So, this quadratic equation has this solution that is there are two solutions with plus minus sign. Now, depending upon what is b square minus 4 a c, the solutions could be complex or could be real. Now, the way this formula is arrived at is of course, through completing the squares and from which taking the square root from here, we get this expression. Now, the same thing if we try to do in the case of cubic equation, then we would be thinking of completing the cube. So, that you can take a cube root and get the values of x. Now, this is comparatively much more difficult compared to quadratic equation and that is why it took a long time for mathematicians to arrive at a good way to solve this problem analytically. Cardinal solution proceeds like this that is for completing the cube in some sense, the process succeeds if this x square term is missing from the equation. Now, how to make it missing? How to get rid of this x square term? So, the way he handles this problem is by substituting y equal to x plus k here, which will transform this equation to y in this manner. Then we say that let us choose that k which makes this y square term vanish and that k is a by 3. So, by choosing a by 3 as k, we get this term as 0 and then y cube plus something into y plus something equal to 0 that becomes the transformed equation in y. So, in this manner now solution of this is possible in a process which is something like completing the cube. So, for that y is assumed in this manner u plus v and if you take the cube of this y cube, then you get this expression in which there is another u plus v appearing here. So, if we write this u plus v as y, then you note that this equation and this equation can be compared y cube y cube p y and as you bring this on this side you get minus 3 u v y. So, p becomes equal to minus 3 u v that means u v is this and then q here and negative of this term for this equation. So, u cube plus v cube become minus q. So, in u and v you have got two equations. It is not very difficult to solve this because here you have got u cube plus v cube and with the help of these two equations you can develop another equation in for u cube minus v cube. So, that is a plus b whole square minus 4 a b is a minus b whole square. When you use that formula you get this the square root of this second one gives you u cube minus v cube is equal to plus minus the square root of this. So, there are two equations the first one gives you the sum of u cube and v cube and the second equation will give you the difference of the same two things. So, half the sum gives you u cube and half the difference gives you v cube. So, you solve u cube and v cube from these two and get this. Let us call it call these two as a and b plus minus you get. So, you get two values a and b. So, one of them you can call u cube and the other you can call as v cube. After that you take the cube roots of a and v b and while taking cube roots keep in mind that if this thing becomes complex then you have to appropriately take it is angle and in the cube root the angle will get 1 by 3 1 by 1 by 3 2 by 3 and full so like that. So, that means in the cube root. So, if the description of that number in the complex plane turns out to be like this then it is cube root will have one third of this angle plus 120 degree and then plus 120 degree wherever that goes. So, these will be the arguments of the three cube roots. So, even if this is real the three cube roots will be one which will be real the other will be at 120 degree and the third will be along 240 degree. So, that way we take the three cube roots of a and b. So, you can show it as if one of the cube roots is a 1 the other one will be a 1 omega the third one will be a 1 omega square. Similarly, on this side now with this with u chosen from here and v chosen from here you can combine u plus v in 9 different ways, but all of them will not satisfy this equation only three of them will satisfy you can verify this fact. So, if it happens that a 1 and b 1 taken from these two and added together becomes one root of this equation satisfies this equation then the other two will be a 1 omega plus b 1 omega square and y 3 will be a 1 omega square plus b 1 omega this is the way it is and you will also note that out of these three roots one of them must be real because this is a cubic equation. So, the solutions of it the roots of a cubic polynomial if they are complex must appear in complex conjugate pairs. So, that will be able to account for at most and even number of roots. So, only two of them can be complex the third one the odd member out will be essentially real. So, this is one situation that you will always find when the degree of the polynomial is odd. Now, after the second after this cubic equation was solved by Cardano then mathematicians tried for quite some time to solve the next higher degree equation that is the quartic equation four degree equation and Cardano's own student Ferrari found a method again based on completing the square to solve for the four degree equation. So, suppose this is the four degree equation given of course, the leading coefficient can be taken as one because whatever else appears there you can divide the entire equation by that number and put it in this manner. Now, if you take if you keep on this side on the left side only the fourth degree and third degree terms and complete that as a complete square compensating by the term a square x square by four on this side and take all these other terms there then you get this expression. On this side you get a complete square it would be nice if at the same time you got a complete square here also then both sides you could take square root and reduce the degree of the equation, but then that cannot be always possible because this expression is dependent on the a b c d coefficients given in the original problem that is not in our hand. So, in general this will not be a perfect square of a linear expression. So, we need to make it a square by doing what by adding certain things here and adding compensatory terms on this side also. So, inside this complete square we add a term from our side we add a term y by 2 and the corresponding compensations there will be three compensations y into x square which is here and then y into a by 2 x which is here and y square by 4 which is here. So, by adding these compensatory terms we extend this perfect square on this side and then ask the question what is the value of y for which this side also will be a complete square of a linear expression of x. Here we could not say that because we had no freedom now we have introduced y. So, we can ask for y to satisfy some requirement that we supply. So, we ask under what condition this will be a perfect square and that condition we know that condition is that square of this coefficient is equal to 4 times the product of these two terms pantheric terms. When we impose that condition like this b square minus 4 s e equal to 0 then and simplify this then we get a cubic equation in y. That means any solution of this cubic equation taken as y if we insert in this equation then the right side will be a complete square. So, now the question is how to solve this cubic equation. We have already seen how to solve a cubic equation. So, by that method Cardinal's method or by any other method if we can solve this cubic equation and take any one of the three roots any one of the three solutions for y. And typically we will take the real solution because cubic will always have a real solution. So, typically we will pick up a real solution from this cubic equation dump it here and then you will find that this right side becomes a perfect square. So, these are the steps first frame the cubic resolvent this is called the cubic resolvent of a quartic. So, in that a b c d we know from the original problem. So, we insert those things and frame this equation. So, that is the cubic resolvent of a quartic equation solve this cubic equation pick up one solution as y insert this y into that equation to get the right side in the form of a perfect square. The moment we have come to this point we immediately take square root on both sides. So, this turns out to be plus minus e x plus f and if we take the plus sign we get one quadratic equation if we take the minus sign we get another quadratic equation. And these two quadratic equations we can separately solve we get two roots from here and two roots from there and that completes the solution of the four degree equation. Now, after the four degree equation general quartic was also solved in this analytical manner then people started thinking in the next stage. That how about higher degree equations till it was proved by the Galois group theory that a general quintic or higher degree equation is not solvable in this analytical style. Only iterative numerical methods one can use for solving general equations of degree higher than four and that means we have to rely on iterative processes, but then iterative processes are of two kinds one kind of iterative processes for equation solving are general. They are for general non-linear equations which will apply to polynomial equations also, but then there are some methods which are specific to polynomial equations and they operate with much higher efficiency on polynomial equations. So, when you need to solve a polynomial equation then try to use one of those methods which are specific for polynomial equations. On the other hand for general non-linear equations you have no option you have to apply the general iterative numerical processes. So, the specific methods for polynomial equations are somewhat analytical they have an analytical base and partly they are numerical partly iterative. So, one method is through the companion matrix in one of the exercises in one linear algebra chapter in the text book you might have found the a problem in which it was asked to find the characteristic polynomial of this matrix. If you try to find the characteristic polynomial of this matrix by the standard process by lambda i minus this matrix determinant u equal to 0 and then expand the determinant then you get the characteristic polynomial exactly like this. The typical form in which you would supply a polynomial equation polynomial. So, then you find that if the characteristic polynomial of this matrix turns out to be the polynomial that we have in hand then to find the roots of the polynomial we can try to frame this matrix and call it the companion matrix and then solve an Eigen value problem for this matrix. So, you find that if the matrix is very small say up to size 4 then the correct way or simplest way to solve the Eigen value problem is to frame the characteristic polynomial and find its roots on the other hand if the Eigen value problem is of a much larger size the matrix is very large then that is found not to be a good method. For that purpose we studied a lot of other Eigen value problem methods. Now, if we find a polynomial of large degree and we want its roots then one way is to frame the companion matrix and then find its Eigen value. So, this is one of the best methods for finding the solutions of large degree, high degree polynomial equations. There is another method which is Bayer-Stokes method which also works very nicely on polynomial equations. What it does is that from a given polynomial it tries to separate out factors of small degree. Now, if you try to attempt separating real linear factors like a x plus b then that may not succeed always because a priori you do not know whether the equation has real solutions whether the polynomial has real roots, but then you know that if it has complex roots then complex roots always will appear in conjugate pairs. And when you multiply two such factors in which alpha and alpha bar are conjugate pairs then the product gives you a polynomial which will be in the form x square plus something x this is real plus something this is also real. So, some of the conjugate pairs is real product is also real. So, like that you will find. So, this quadratic factor will certainly have real coefficients if the original polynomial has real coefficients. So, we do not attempt to separate out real factors real linear factors, but real quadratic factors. So, after separating out a real quadratic factor what we do we propose a factor like this x square plus q 1 x plus q 2 and conduct a synthetic division of the given polynomial. So, after completing the synthetic division there can be a remainder if this is actually not a factor then there will be a remainder. If this is a factor if we if our guess is perfect then this is a factor and in that case remainder will be 0 and it will confirm that this is indeed a factor. On the other hand at the starting point in general we would not be able to make a correct guess. So, there will be a remainder remainder will be a linear remainder of degree less than this. So, the linear remainder will be like this now let us examine these two numbers R 1 and R 2 that will appear in this linear remainder R as a result of these two choices q 1 and q 2. So, if we choose different q 1 q 2 values then we will get different R 1 R 2 values. So, that way we can say that R this vector R 1 R 2 is actually a function of this vector q q 1 q 2 that means this R is a vector function of this vector variable q this also has two member this also has a two member has two member. So, then we say that now we try to write that as if this is a function. So, R of q equal to 0 this system of two equations in two unknowns the unknowns are q 1 q 2 and equations are these R 1 equal to 0 R 2 equal to 0. So, these two equations in these two unknowns we try to solve that means iteratively we change q 1 q 2 values in our guess which will make the remainder vanish. Now, this is actually analogous to the typical process of Newton Raphson method which is Jacobian based and in the case of polynomial equations there is a specific way to develop the Jacobian and apply iterations which is much more efficient compared to a general case. So, in the exercises of this chapter there is one problem one exercise in which the step by step process has been given to establish this particular method and then use it. I strongly advise you to carry out the operations once to be at home with the method how it operates. Next we continue into the case of two simultaneous equations for that matter one purpose of including the equation solving process in this course is basically to include methods for systems of equations. So, if we have two simultaneous equations in two unknowns. Now, polynomial based methods operate in a particular way in which we try to eliminate one of the unknowns and get a single equation in the other unknown. So, if there are two polynomial equations in x and y like this then rearranging we can express these two equations as if they are equations in x only. In the coefficients you will get terms containing y a 1 and a 2 are simply p 1 and p 2 as appearing from here and b 1 and b 2 are linear expressions of y including this and this term and c 2 are quadratic expressions of y including this term this term and this term whatever they are for the time being we club them together. Then we approach our school knowledge of common root for these two polynomials common solution for these two quadratic equations and the result is given by Cramer's rule x square by b 1 c 2 minus b 2 c 1 is equal to minus x by a 1 c 2 minus a 2 c 1 is equal to 1 by a 1 b 2 minus a 2 b 1. So, that we write now if we consider these two the equality of these two then from there we get one expression of x that is x square by x that gives us one expression for x that is minus this thing divided by this thing that is this. On the other hand if we use the equality of these two terms then we get x by 1 which is minus this by this that is this. Now the condition for common solution of these two quadratic equations for x is that these two must indeed be equal and that means that product of these two cross product cross product these two so product of these two is the same as product of these two that means this into this is equal to this whole square. So, this is the condition for common solution of these two quadratic equations. Now for x that is the condition, but for y what is it as you equate this to 0 and insert b 1 b 2 c 1 c 2 as expressions of y from which they were earlier found then this will run out to be a 4 degree equation in y. So, if you use that 4 degree equation to if you solve that 4 degree equation for y then in general you get 4 different solutions each of those solutions as you insert here you get one solution for x. So, that way for x and y combined you get 4 solutions which will be the case because these are both quadratic equations and the maximum number of solutions that they can have together is 4 only. In general you can say that two equations in x and y of degree n 1 and n 2 out of that if you can eliminate x the x eliminate turns out to be an equation of degree n 1 into n 2 in y. This number is called the Bezos number there are several methods for elimination of unknowns from simultaneous polynomial equations two of those methods are quite effective one is called Sylvester's dialectic method and the other is Bezos method. We do not go into that in detail because those who are interested in this polynomial system of equations they can look up particular texts and courses for theory of equations itself. Now, after this we will continue into the discussion of those methods which are general which will solve polynomial equations as well as transcendental equations they are purely numerical methods. A typical general non-linear equation can be expressed in this manner f of x equal to 0 and the practical problem now in the case of general non-linear equations transcendental equations we do not ask how many solutions this will have typically we do not ask because quite often enumerating the solutions that is working out the number of solutions that has that equation has is quite cumbersome. So, the practical problem is to find one real root of f x or one real solution of this problem. So, this entire study is typically conducted for real solutions unlike the case of polynomial equations in which case we are quite often interested in the complex solutions as well in the case of general non-linear equations we typically look for real solutions only. The examples of f x it can be a polynomial or it can be something like this and so on. So, for this kind of problems for solving general non-linear equations if you background processes will help us in tackling the correct interval in which we would expect a solution. So, for that we use the method of bracketing if we can identify two points say x 0 and x 1 in which the function value f of x is of different signs then if the function is continuous then we can use its continuity to claim that between x 0 and x 1 there must be a point where the function crosses 0 and that point gives us a root. So, now after bracketing the root in that manner all that remains is to hunt out the root in that interval one straight forward method would be bisection that is given the interval x 0 x 1 which gives a bracket we look for the sign in x 0 plus x 1 by 2. Now depending upon what is it what is its sign this midpoint will replace either x 0 or x 1 or x 1 and then continue. So, bisection itself is one viable iterative method for solving an equation like this, but it is costly there are much more effective methods for solving the problem, but you can always consider bisection method as a fallback if in some iteration the other sophisticated methods somehow get into some trouble. So, bisection method remains one method on which you can always rely as long as you have a bracket to start with. So, what are the other sophisticated methods? One is the fixed point iteration method. So, what we can do is that we can rearrange the given equation in a manner such that x is expressed as an expression of x itself. Now for example, take this example of f x. So, the solution of the equation tan x minus x cube minus 2 equal to 0 is required. So, f x is this. Now if this equal to 0 then we can arrange that equation in the form of an expression for x in terms of x itself for that expression as g x. There are several ways of doing it. One possibility is to take these two on the other side of the equation x cube plus 2 and then say then x is tan x minus 2 inverse that. The other possibility is take x cube only on the other side and then on this side you have tan x minus 2 and then cube root of that will be x or dividing tan x minus 2 by x square you get another expression for x. So, all three of these are candidates for suitable g x that you can frame out of this equation and try to iterate. Now what is iteration? Iteration is very simple. The iteration has two steps starting from a guess value. All these iterative processes will start from one guess value you see. So, starting from a guess value x 0 evaluate this function and since we are trying to solve x equal to g x then whatever is the value of this function that becomes the next value of x 1. So, then that x 1 goes here whatever comes out that becomes the next iterate for x that is x 2 and so on. So, this continues hoping for convergence. Now as I say hoping for convergence for an arbitrary arrangement like this there is no guarantee that the process will converge. If the solution is say at 5.2 then if you make a wrong kind of reorganization then it may happen that even starting from 6 it might go to 10 and then 16 and then 79 and it may diverge. So, convergence as it is is not guaranteed. Then we ask what is the question what is the condition under which the convergence is guaranteed there is a condition. So, to discover the condition see the way this process will work. Suppose after organizing x equal to g x we have got this curve as the graph of g x and. So, y equal to g x is this graph this curve and y equal to x is this 45 degree straight line. Now y equal to g x and y equal to x intersect at a point where x equal to g x is satisfied those are the points which we are looking for. As shown in the figure there are two such points one is x star and the other is x bar at which this curve and this straight line intersect. Now suppose we start from this value x equal to p as we were just discussing the iteration is just two steps at the current point evaluate the function g of x and then whatever is the value assign it to x and this process is continued. So, at this point x equal to p evaluate the function that way we get this value which is a. So, then from there this value now has to be assigned to x that means parallel to the x axis that is horizontally if we move and then meet the y equal to x line that means this y value will be here. Same as the corresponding x value so at this x value then we evaluate the function g x and I have at the point on the curve and then again horizontal vertical horizontal vertical we do converge. On the other hand if you start at q extremely close to the other solution x bar you would expect that quickly it would converge to x bar, but that does not happen directly from q we evaluate the function reach this point l. That value of g x that is y we assign to x that means we come here at that value we evaluate g x again we come here look we are not going from q towards x bar, but we are going away. If we start on the other side of x bar say at r again evaluate it here come here go there u v w we are again diverging on the other side. So, what is wrong here? So, so close to x bar we are not converging to x bar whether you start from this side or that side on the other hand so far away from x star we could happily converge. So, what is the difference? So, if you analyze the situation you will find that if there is an interval bracketed situation in which there is a unique solution x star then and apart from that if the reorganization has been done in such a manner that g prime has an absolute value which is less than 1 in that interval then any point started with in that interval will converge to x star. That means the slope of this curve in the interval if the slope of that is not greater than this slope of this line then the process will converge with guarantee otherwise not. The next method for solving a non-linear equation is the Newton-Raphson method this has a connection with the Newton's method of optimization theory as well. So, this method relies on the first order Taylor series of the function f x which is like this this is the first order truncated Taylor series. Now, the understanding is that we have got a value of x where we have evaluated the function and its derivative and we want to move to a new point x plus delta x where the function value is 0. So, if we want f of x plus delta x to be 0 then for that purpose we equate this right hand side to 0 and from there we get a value for delta x which is minus f x by f prime x. Now, when we add this delta x value to the original current x x k then we get this expression for x k plus 1 this is the typical update formula. If you examine this formula carefully you will find that this is one way of writing x equal to g x on the left side you have the next value of x on the right side you have an expression of the current value of x. Now, for that g x if you work out the same convergence requirement as the fixed point iteration formula then you get the convergence criterion to be this. If this is satisfied in an interval then the starting point from within that interval will converge to the root in that interval. The geometric interpretation of the working of Newton's lesson method based on this formula is the following. You start from a guess point and evaluate the function and reach this point at this point make a first order approximation of the curve that is a tangent approximation tangent based approximation. So, that means draw a tangent to the curve at this point and wherever that tangent crosses the x axis take that as the next point again from there evaluate the function reach the point on the curve draw the next tangent and so on. So, started from x 0 you will travel as a b c d e f and like this you can converge to the value x star which is the root of this particular function f x. Now, the merit of this method is that it has a quadratic speed of convergence that is at every iteration the accuracy gets improved by some out two orders this the accuracy gets improved by two orders at every step. So, this is called the quadratic speed of convergence the demerit is that if the starting point is not appropriate then there is no guarantee that the process will converge. So, sometimes Newton's method is found to wander haphazardily and go away from the solution it might diverge diverge or sometimes it might oscillate from here to on this side then on that side again on this side and sometimes the Newton's method is found to oscillate around the solution also rather than converging. So, these are some of the demerits that Newton's method has, but when it works if started close enough and if the function derivative is good the rate is good the slope is good then when it converges it converges extremely fast. Now, in this particular iteration formula if we replace the derivative by a finite difference kind of first order derivative then what you get is the method of the second method rather than tangent base method you get a second base method that is through a chord cut through two points on the curve. So, replacing f prime by this expression you get this iterative formula that is called the second method. So, the way it works is that start from two points x 0 and x 1 at these two points evaluate the derivative evaluate the function values no need of derivative here because derivative is replaced by this. Then through these two points draw a chord of the curve wherever this chord cuts the x axis take that as the next point then out of x 0 x 1 and x 2 retain two points and continue the process. Now, this is the second method and a particular case of second method in which the initial two points are taken in such a manner at the function value is positive on one side negative on the other and at every step that bracket is maintained that method is called that method is a special case of second method and that is called regular falsi or method of false position. In this particular case since we are maintaining a bracket always so convergence is guaranteed this is a very reliable method. Now, there are other methods also to solve non-linear equations one of them is quadratic interpolation or Muller's method in that what you do is that you do not talk of derivatives at all for evaluation you take three points at which the function is evaluated and through three points you can develop a quadratic model for the function a local quadratic model and then say we want to find out at which value of x this is 0 and that point is the next point that we have. That means started from three points this model gives a quadratic approximation for the function in the local neighborhood. So, we equate that to 0 and solve for x and then with the old three points we have got a fourth point out of these four points depending upon the function values we retain three and throw away the worst point and continue this process. So, this is quadratic interpolation of course, in this we have to solve a quadratic solution of a quadratic is not difficult, but then questions that arise is that what if that quadratic has two solutions which one to take then if the both the solutions are complex then we have nothing in hand in comparison to that there is another method which is called inverse quadratic interpolation which is found to be much more straight forward from the same three points that you start with do not frame y as a quadratic expression of x, but x as a quadratic expression of y and then for y equal to 0 you get x equal to a take that value of x as the fourth point and again out of the four points available drop one retain the three best and continue this called inverse quadratic interpolation. Combining this there is a combining this with many other aspects you have got a very good professional method for non-linear equation solving that is called one Weingarten Decker-Brent method or in brief Brent method. So, see here through the same three points x 0 x 1 y 1 x 2 y 2 the quadratic interpolation will give you this curve from which you might take this point or this point as the next point the inverse quadratic interpolation on the other hand would give you a y quadratic interpolation gives you this curve y as a quadratic expression of x and inverse quadratic interpolation will give you this parabola both are parabolas. So, the same three points the same three points this is the parabola that you would get in the case of quadratic interpolation this is the parabola that you would get in the case of inverse quadratic interpolation which will cut the x axis only at one point. Now in the Brent's method there are quite a few aspects that are combined first of all it starts with a bracket out of the three points that we have in hand the three points must fall on two sides of the solution that is at least in one of the at one of the points the sign of the function must be opposite to what is the sign in the other two points. So, and that bracket is always maintained. So, you never drop the solitary point on the other side retaining all three points on one side of the root you never do that in Brent's method you always maintain the bracket second you use inverse quadratic interpolation that is this that is this model. And whenever you find that the next suggestion of a point that is that you get out of this turns out to be within the bracket interval you accept it on the other hand if you find that the suggestion x equal to a turns out to be outside the current bracket that means rather than squeezing the bracket it would enhance the bracket which is not desirable. Then you say that for that particular iteration we do not accept the suggestion of the inverse quadratic interpolation rather for that particular iteration we replace that step by a straight forward by section straight. So, whenever we find that the sophisticated method suggest something good we accept it and if it suggests something undesirable then you do not accept it and for that particular iteration we use our time tested reliable and guaranteed method of bisection for only that step. So, when the bad time comes we look for a slow, but assured step and when good times proceed we approach the solution faster. So, this is a kind of opportunistic manoeuvring between a fast method and a safe method this kind of situations we will come across quite often when we go into the optimization method. In this context it can be noted that many of the equations solving processes are deeply related to many of the corresponding optimization processes they have the same background of the iterative schemes. Now we proceed to the system of non-linear equations if you have got a number of equations in a number of unknowns then coincidentally this system can be written like this where f is a vector function of a vector variable x. Now in general it could be that the dimension of x is greater than the dimension of f or the other way round if the dimension of x is greater that means you are trying to solve less number of equations in more number of unknowns that is an under determined problem and you would expect infinite solutions if there are solutions at all if the system is consistent then you would expect infinite solutions. On the other hand if the number of x's is less then in general you would expect conflict and then you talk of least square problems that is to the extent possible you can try to satisfy all the equations if the number of equations is larger. The most often encountered situation is when you have n equations in n unknowns so that means in which case you will expect a finite number of solutions not necessarily unique but a finite number of solutions that is what you would expect but you might find one of the other cases also in that case but typical expectation is a finite number of solutions in that situation. So, now one comfort of a single equation is lost you cannot talk of bracketing at all because it is not a one dimensional line anymore it is a multi dimensional space in which you have to make the search and if you try to work out fixed point iteration schemes like this then there will be so much of possible reorganization of these equations that you cannot enumerate them out. So, you try to establish those kinds of reorganization schemes which come from some straight forward theory rather than haphazardly trying the recombination. The most obvious candidate for that purpose is again Newton's method or Newton-Abson method which we get from the first order truncated trailer series of this vector function of a vector variable and that is f of x plus delta x is equal to f of x plus the Jacobian multiplied with delta x the higher second order terms for the time being we neglect. Now, if f of x plus delta x is roughly equal to this and if we have a starting value starting vector x take 0 and we want at the next point this function value to be all 0 then we equate this to 0, 0 vector and then take f x on the other side pre multiply with the inverse of j x and that gives us delta x that delta x we add to the current x and get the next x. So, the next value of the vector x is current 1 minus j inverse f. So, this is the typical iteration of Newton Abson method for solving a system of non-linear equations. This matches exactly with what we had earlier for single equation for a single equation we had this. Now, again f of x k is here f of x k and division by the first derivative is now replaced with pre multiplication with the Jacobian inverse division by a matrix you cannot conduct. So, the corresponding operation here is pre multiplication with inverse. So, further matter you could write it as minus 1 by f prime into this that is f prime to the power minus 1 into this. So, that j inverse is here of a Jacobian that is the first derivative. Now, this method of solving a system of linear equation is non-linear equations has its usual merits and demerits merit of first convergence and demerits of the lack of guarantee of convergence. It might wander away, it might oscillate, it might diverge such things may happen and you can figure out that such things are most likely to happen when this matrix is close to singularity because in that case the corrections will turn out to be so large up to which you cannot rely on a first order truncated Taylor series. So, what are the other methods? There is a modified Newton's method and then there is a Broiden's second method. So, just like the second method of a single equation for multiple equations also for a system of equations also you have a second method in which the same Jacobian theme is used, but Jacobian is not evaluated at every iteration because you see that it is costly because for n functions you have to determine n derivatives. So, n square derivatives you need to evaluate. So, that is costly. So, in Broiden's second method the Jacobian is not evaluated at every iteration, but it is developed through some suitable updates at every iteration. Through the steps that we conduct we get some idea about the derivative and based on that idea we update, we keep on collecting the derivatives, we keep on updating the Jacobian matrix and that way towards the later iterations we expect to find the Jacobian without actually evaluating derivatives. We try to develop a Jacobian which is close to accurate. The best formulation for non-linear equation solving is through the method of optimization. You see that rather than asking for f x equal to 0, f 1 0, f 2 equal to 0, f 3 equal to 0 and so on, if you pose the question as an optimization problem and you say we want to minimize this function, we know that a global minimum of this function is at that point where each of the squares is individually 0 because it is a sum of squares. So, this kind of an optimization based formulation for a non-linear system of equation is typically the best among the non-linear equations solving processes and for this there can be after you formulate it like this in principle you can use any optimization method to minimize this function, but in practice we find that some methods perform better in this kind of a problem, some other methods do not do so well. Levenberg-Marquardt method is one optimization method which is particularly suitable for this kind of equation solving processes and least square minimization problems which we will discuss later because the proper appreciation of this method will be possible only after some discussion on the optimization method, methods of non-linear optimization, a topic which we start from the next lecture. In the current lesson, these are the important points that we should keep in mind the iteration schemes for solving single equations and Newton-Nurson method which in most situations turns out to be adequate. If it is not adequate, then we look for certain other optimization based approaches and other methods also have a connection with non-linear optimization method. So, next lecture we start our study of optimization methods and along that we will see at the appropriate time what is the optimization based formulation for solving systems of non-linear equations. Thank you.