 So, we have considered system of linear equations. Now, we consider single equation f x is equal to 0, where f is any function. We are going to consider methods such as bisection method, then Newton's method, secant method and a variation of secant method which is known as regular falsie method. The bisection method is simplest method, but the drawback is the convergence is going to be very slow. Newton's method when it converges, it is going to converge fast, it is going to converge what is known as quadratically. In the case of Newton's method, one needs to know what is a function value and what is a derivative value. So, if the computation of derivatives is difficult, then the Newton's method is not advisable, then one considers what is known as secant method. The price one pays is the secant method will not converge as fast as the Newton's method. Now, these methods they are not going to converge unconditionally, it will be only under certain conditions. If you have chosen your starting point appropriately, then it is going to converge, but when it converges Newton's method converges fast and that is going to be the best method. Now, this Newton's method secant method these are based on the idea that f is a general function. So, finding its zero is difficult, but if f is a linear function that means if it is a straight line, then we know how to calculate its zero, we have to see where the straight line crosses x axis. So, one approximates the given function by a straight line and that gives us the Newton's method and secant method. These methods they are going to be useful when one considers the Eigen value problems. So, in case of Eigen value problem, one needs to find zeros of polynomial. So, let me first explain what is the bisection method. Also, when we want to analyze the convergence of Newton's method convergence of secant method, it is useful to connect the zero of a function to a fixed point of a function. So, if f of c is equal to zero, then we say that c is a zero or root of our function f. If f of c is equal to c, then we say that c is a fixed point. So, in order to calculate fixed point approximately, we will be considering Picard's iteration and then we will see how the Newton's method, it can be put in the setting of fixed point iteration. So, today we are first going to describe what is bisection method. So, today we are first going to describe what is bisection method. Then I will describe what is Newton's method, what is secant method and then we will go to fixed point. We will define the Picard iteration. We will prove conditions for existence and uniqueness of fixed point and then we will see how the Newton's method can be put in this setting and the convergence of Newton's method secant method, we will be considering later on. So, now our setting is f is a function defined on closed interval a, b taking real values. Our aim is to find c such that f of c is equal to zero. If the function is f of c is equal to zero, then f of c is equal to zero. So, f of a into f of b is equal to zero. So, f of a into f of b is equal to zero will mean that either f of a is zero or f of b is zero. If it is strictly less than zero, then it means f a and f b they will be of opposite signs and then by the intermediate value theorem f of c is equal to zero for some c belonging to interval a, b. So, if you have got f a into f b to be less than or equal to zero, we know that a root of function f lies in the interval a, b. Now, this is the simplest method, bisection method. f is from a, b to r is continuous f a into f b is less than zero and f has a unique zero c in a, b. So, this much is given to us f a and f b can be of opposite signs and they can have more than one zero like look at this is say point a, this is point b. So, I have got f a into f b is less than zero and then there is this zero, this zero and this zero. So, there are three zero. So, in the bisection method we assume that we are given that f has a unique zero c in a, b. So, the method is look at the midpoint. If f of m is equal to zero, then find c is equal to m. Otherwise, either f a into f m is less than zero or f m into f b is less than zero. This will mean that our zero lies in the interval a to m. f m into f b less than zero will mean that the zero lies in the second half that is in the interval m to b. So, you choose a 1 to be equal to a, b 1 to be equal to m. In this case, in the other case you choose a 1 is equal to m and b 1 is equal to b. So, we started with the interval a b, we subdivided it into two equal parts. We know that our function has got only one zero. So, if the midpoint is zero, then find, then we have found the zero. Otherwise, our zero will be either in the interval a to a plus b by 2 or it will be in the interval a plus b by 2 to b and that can be checked by looking at sign of f of a f of a plus b by 2 f of b. So, we have found that when the two end points are of the opposite sign, that interval is going to contain our zero. So, starting with the interval a b, we have found the interval of half the length which contains our zero and then you continue. Suppose it is in the first half, that means a to a plus b by 2, divide it into two equal parts. If the midpoint is the zero, well and good. Otherwise, there will be one of the interval which will contain your zero. So, take that interval. So, it is a very simple method, easy to apply, only the convergence is going to be slow. Like this way, we are constructing the sequence of intervals a n b n and both a n and b n, they are going to converge to c as n tends to infinity. But at a time what we are doing is we are dividing the interval into half. So, at every stage you are going to gain only one binary digit. So, that is the drawback of this bisection method. But still it is useful to do some like when we want to apply Newton's method, we need a starting point. So, one can do a few steps of bisection method, try to get a smaller small interval in which our zero is going to contain and then take x zero to be a point in that interval. So, let me describe the bisection algorithm. So, it is a zero is equal to a b zero is equal to b and then for n is equal to 0, 1, 2, m is going to be equal to a n plus b n by 2. If f of a n f m, if they are of opposite signs then a n plus 1 is equal to a n, b n plus 1 is equal to m. If this is not the case then a n plus 1 is equal to m, b n plus 1 is equal to b n and then you continue. So, in this algorithm one can give a statement that if at any stage f of m is equal to 0 then you stop. Another stopping criteria one needs to give is when to stop. So, then either you have to specify the value of n, the maximum value or you can give it in terms of the say length of the interval like when the length of the interval say it is less than 10 raise to minus 6 then you stop. So, you have to give the stopping criteria. So, this is the simplest method which gives us an approximation to a 0 under I have said under the condition that it should be continuous function and it should be given to us that there is only 1 0, but it can be modified like suppose it has got more than 1 0 like if they have opposite signs the number of 0s is going to be odd and all. So, this is the method one can modify. So, this is by section method in the simplest form and the convergence is going to be very slow. So, now here is an example f x is equal to x cube minus x minus 1, x belonging to 1 to 2 it being a polynomial it is continuous. f at 1 is going to be equal to minus 1 f at 2 is equal to 5. So, f 1 and f 2 they are going to be equal to the derivative of opposite signs. The derivative of f is given by 3 x square minus 1 and in the interval 1 to 2 it is going to be bigger than 0. Now, if the derivative is bigger than 0 that tells us that f is strictly increasing. If it is strictly increasing it has a unique 0 in 1 to 2. So, all the 3 conditions are satisfied it is a continuous function the end points have opposite signs and it has a unique 0 in the interval 1 to 2. So, let us look at the midpoint value f of 3 by 2. So, f of 3 by 2 is 7 by 8 which is 0.875. So, this being positive and f of 1 is equal to minus 1 these being of opposite signs our 0 is going to lie in the interval 1 to 1.5 and then one can continue this and obtain an approximation to 0 of our function. So, this is bisection method and now let us describe what is Newton's method. So, we have got a function let us assume that the function is differentiable and at no point the tangent is becoming horizontal. So, that means the derivative is not vanishing I am telling you sufficient conditions. So, you have interval a b you start with a point x 0 at x 0 look at the tangent to the curve at x 0. See where the tangent cuts the x axis wherever it cuts that is going to be our point x 1. The next step is consider tangent to the curve at x 1 see where it cuts x axis at x 1 that is going to be our x 2 and so on. So, this is going to be our Newton's method you have to start with a some point x 0 and then you approximate your function by a tangent look at the 0 of that tangent or where the tangent crosses x axis that gives us our next point. Now, whether this converges or not we are going to study this later on in detail at present I just want to describe what is Newton's method. I said that the tangent should not be horizontal because if the tangent is horizontal then it won't cut the x axis it will be parallel to the x axis and then now we cannot proceed that is why when I assumes this thing. Another important thing will be that I start with x 0 my function f is defined on interval a b I am starting with a point x 0 I am looking at the tangent to the curve at point x 0 and then I look at the point at which the tangent intersects the x axis. So, this point of intersection can be outside the domain of our function. So, these things we will see more in detail. So, at present let us assume that we don't face such difficulties and then that describes our Newton's method and another thing is in the Newton's method we will assume that it has a simple 0 that means f of c is equal to 0, but the derivative does not vanish. So, here is the graphical representation this is the curve. So, this is our function f the intersection is the point in which we are interested in you start with x 0 look at the tangent to the curve at x 0. So, this is the tangent. So, this tangent cuts x axis at x 1 in the next step look at tangent to the curve at x 1. So, this is the tangent it cuts x axis at x 2 then look at the tangent to the curve at x 2 it will cut the x axis as x 3 and so on. Here is the equation of the tangent at x 0 or if you want at x 0 f x 0. So, y minus f x 0 divided by x minus x 0 is equal to f dash x 0 that is the slope of our tangent and that gives you y is equal to f x 0 plus f dash x 0 x minus x 0 that is the equation of the tangent at x 0 its intersection point with the x axis will be given by equating this equation to 0. So, when you equate it to 0 and suppose that intersection point is x 1 you will get x 1 is equal to x 0 minus f x 0 upon f dash x 0 and in general x n plus 1 will be equal to x n minus f x n upon f dash x n x 0 is the starting point it is given or which is the starting point x 0 and then you get this. Now, I said that the Newton's method is we are going to consider it for a simple 0. So, we have got f of c is equal to 0 f dash c not equal to 0. So, suppose our function f dash is continuous then if f dash c is not equal to 0 in a neighborhood of c f dash x also will not be 0. So, this condition that our iteration is x n plus 1 is equal to x n minus f x n upon f dash x n. So, whenever f dash x n is 0 our procedure is going to break down, but then if you are assuming that f dash c is not equal to 0 your function f dash is continuous and if you are remaining in the interval around c then this condition will be satisfied. Now, whether you remain in the interval or whether you remain in the neighborhood that is another point. So, for that we will have some sufficient condition. So, in fact for the Newton's method this starting point x 0 it is going to be a crucial point. Now, in the Newton's method we need f dash of x n. Now, the function may not be easily differentiable in that case what we can do is we can replace f dash x n by a divided difference. We had considered the numerical differentiation we had f dash at a is approximately equal to f of a plus h minus f a divided by h. In a similar manner we can replace f dash x n by now let us look at the points x n and x n minus 1. So, f dash x n will be approximately equal to f x n minus f of x n minus 1 divided by x n minus x n minus 1. So, if you do this substitution then what you are going to get is the secant method. So, in the Newton's method x 0 is the initial case in secant method we need to have two points x 0 and x 1. You replace f dash x n by divided difference based on x n minus 1 and x n. So, that is going to be equal to f x n minus f x n minus 1 divided by x n minus x n minus 1 that defines the secant method to be x n plus 1 is equal to x n minus f x n divided by this divided difference. So, let us see what it means graphically. So, you are starting with two points. So, suppose I take the end point x 0 here and x 1 here. Look at the equation of the secant through x 0 f x 0 and x 1 f x 1 equation of this straight line which passes through these two points it will be given by y minus f x 0 divided by x minus x 0 is equal to the slope f x 1 minus f x 0 divided by x 1 minus x 0. So, this is the equation of the secant or the straight line which passes through these two points. When you simplify this what you get is y is equal to f x 0 plus f x 1 minus f x 0 divided by x 1 minus x 0 into x minus x 0. Notice that this is nothing but divided difference based on x 0 x 1 equate this to 0 and when you equate this to 0 whatever you get call it x 2 and that is going to give you here this should be x 2 not x 1. So, we are going to have y is equal to f x 0 plus f of x 0 x 1 the divided difference based on x minus x 0. So, I am going to equate it to 0. So, I will get f x 0 plus f of x 0 x 1 into x 2 minus x 0 to be equal to 0. When I solve it for x 2 I will get x 2 is equal to x 0 minus f x 0 divided by f of x 0 x 1. And in general I can write x n plus 1 to be equal to x n minus x 1 minus x 2 minus x 0 f x n divided by f of x n minus 1 x n n is equal to 1, 2 and so on. So, this is the secant formula and now we are going to look at the fixed point of a function. So, here is the definition g from a b to a b it is a function a point c is said to be a fixed point if g of c is equal to c. So, that means the fixed point is going to be intersection of the graph of g and the straight line y is equal to x whatever is the intersection that is going to be our fixed point. Now, fixed point is related to 0 of a function for example, if I define f x to be equal to g x minus x then f of c will be 0 if and only if g of c is equal to c there can be more such functions. But this is just I want to tell you that fixed point is not something totally different than the 0 of a function the two notions they are related. So, now let us look at conditions under which we know existence of fixed point. Suppose your function g from a b to a b is continuous then g has a fixed point c is equal to c in a b. So, what is important is interval a b should map into interval a b it should be continuous and then g has a fixed point c in interval a b. The proof is simple we are going to make use of intermediate value theorem for continuous function if g of a is equal to a then a is a fixed point. If g of b is equal to b then b is a fixed point and now consider the case when g of a is not equal to a g of b not equal to b. Since g maps interval a b to interval a b when I look at g of a g of a has to be in the interval a b it is not equal to a that means I will get g of a to b bigger than a. When I look at g of b so g of b again it has to be in the interval a b and it should be it is not equal to b. So, g of b is less than b. So, now look at f x is equal to g x minus x g of a is bigger than a g of b less than b will imply that f of a is greater than 0 f of b is less than 0 f from a b to r continuous because g is given to be continuous apply the intermediate value theorem to obtain c such that f of c is equal to 0 and this implies that g of c is equal to c. So, we have conditions that your function g should be a continuous function it should map interval a b to interval a b then g is going to have a fixed point. Now, the condition that interval a b should be mapped to interval a b that is necessary that one can easily imagine now what will happen if instead of interval a b closed interval a b if I take open interval a b or instead of the bounded interval a b if I take infinite interval whether the result will still hold that means whether g will have a fixed point. So, it is not true so we will give counter examples. So, the first is the result is not true if closed interval a b is replaced by open interval a b or one of the counter example is g x is equal to x square x belonging to 0 to 1. So, this g is a continuous function it is mapping open interval 0 1 to itself, but g x is equal equal to x will mean that either x is equal to 0 or x is equal to 1 and both the points they are not in the domain. So, the statement about existence of fixed point is not true if you replace the closed interval a b by open interval a b. Next if the interval a b is replaced by infinite interval a to infinity still it is not true and here is a counter example if you look at g x is equal to x plus 1 x belonging to 0 to infinity. So, this function is continuous it maps interval 0 to infinity to itself, but it does not have a fixed point. In this case the function had fixed points, but they were not belonging to our domain here g x is equal to x plus 1 it is the translation. So, it does not have fixed point at all and the third is continuity that if g is not continuous then it not have fixed points. So, this you can define the easily what I have done is I define g x is equal to x square x belonging to open interval 0 to 1. I know that these functions fixed points are 1 and 0. So, what I do is I define g x is equal to 1 when x is equal to 0 and 0 when x is equal to 1. So, I do not need the function to be continuous. So, this function when the continuity is violated then it need not have fixed point. So, this is about the existence. Now what about uniqueness whether a fixed point is going to be unique. Now, there can be more than 1 fixed point like look at our example g x is equal to x square on the closed interval 0 to 1 then it has got 2 fixed points. If you look at function g x is equal to x on intervals again say closed interval 0 to 1 then it has got infinitely many fixed points and you can easily construct the example when it has got a unique fixed point. So, now what we want to do is we want to find a sufficient condition which will guarantee uniqueness of the fixed point. Now, that condition is going to be in terms of the derivative. So, for the existence of fixed point continuity was enough it should map the interval a b to a b, a b should be a closed and bounded interval and it should be continuous. Now, we are going to show that if in addition g is differentiable on open interval a b then and if the derivative modulus is less than 1 then it has a unique fixed point and the proof is straight forward what we are going to do is use the mean value theorem that if you have got g c 1 is equal to c 1 g c 2 is equal to c 2 then consider g c 1 minus g c 2 apply mean value theorem and conclude that c 1 has to be equal to c 2. Here is the uniqueness of the fixed point g is a map from a b to a b continuous map it is differentiable on open interval a b with modulus of g dash x to be less than 1 for x belonging to open interval a b then g has a unique fixed point in a b existence is already proved. So, let us look at uniqueness. So, let g c 1 is equal to c 1 g c 2 is equal to c 2 for c 1 and c 2 in the interval a b c 1 minus c 2 will be equal to g c 1 minus g c 2 the function g is going to be continuous on the closed interval c 1 to c 2 differentiable on open interval c 1 to c 2 because g is continuous on a bigger set a b and it is differentiable in possibly bigger set open interval a b. So, by the mean value theorem this is going to be equal to c 1 minus c 2 into g dash d where d is some point in the interval c 1 to c 2 modulus of c 1 minus c 2 take modulus of both the sides. So, you will have mod of c 1 minus c 2 is equal to mod of c 1 minus c 2 into modulus of g dash d if modulus of g dash d is less than 1 because that is our assumption then c 1 minus c 2 has to be 0 because if it is not 0 then what we will get will be modulus of c 1 minus c 2 is strictly less than modulus of c 1 minus c 2. So, using this fact one gets c 1 is equal to c 2. So, we have obtained now the necessary and or we have obtained conditions for the condition the uniqueness and existence of fixed point. These are sufficient conditions that if these conditions are satisfied then definitely g has a fixed point. Now, next comes the question that how to find an approximation to this fixed point as I told you that fixed point of a function can be a 0 of another function. So, if we have a way of finding approximation to fixed point this can be translated into a method for finding 0 of a function. So, we are going to now define what is known as Picard's fixed point iteration scheme. The convergence of this scheme we will prove under slightly stronger condition that on the interval open interval a b modulus of g dash x should be less than or equal to some constant k which is less than 1. For the uniqueness our condition was modulus of g dash x should be less than 1. Now, we are saying modulus of g dash x should be less than or equal to k less than 1. In the first case modulus of g dash x less than 1 means g dash x can go arbitrarily near to 1. In the second case when we are saying modulus of g dash x should be less than or equal to k less than 1 it means it has to stay away from 1. Now, this stronger condition we are assuming for the convenience that result is true also when modulus of g dash x is less than 1, but if we assume this condition then life becomes easier the proof becomes simpler. So, under this condition we define Picard's iteration scheme which is nothing but go on applying g start with x 0 apply g. So, g of x 0 that will be our x 1 and then you continue that gives you iterates and then we will show that these iterates they converge to the unique fixed point c and once again we will be using the mean value theorem. So, here these are our assumptions g is from a b to a b continuous differentiable on open interval a b modulus of g dash x less than or equal to k less than 1 for x belonging to a b then g has a unique fixed point c in interval a b this part we have already proved start with x 0 in a b and define x n is equal to g x n minus 1 n is equal to 1 2 and so on x 0 can be any point in the interval a b. Then x n converges to c as n tends to infinity. So, let us prove this result. So, consider x n minus c x n minus c is equal to g of x n minus 1 minus g of c because x n is equal to g x n minus 1 by definition c is equal to g of c because it is a fixed point apply the mean value theorem. So, you will get this to be equal to x n minus 1 minus c g dash of d n where this d n is going to lie between x n minus 1 minus c because x n is equal to g of x n minus 1 and c. Since modulus of g dash x is less than or equal to k less than 1 you get mod of x n minus c to be less than or equal to k times modulus of x n minus 1 minus c. Apply the same argument and get this to be less than or equal to k square modulus of x n minus 2 minus c and like that less than or equal to k raise to n mod of x 0 minus c. Now, it is here that we use the fact that k is less than 1. So, k strictly less than 1 implies k raise to n will tend to 0 as n tends to infinity and this implies that x n converges to c as n tends to infinity. So, we have proved this Picard's fixed point iteration that if the derivative is less than or equal to k less than 1 then for any starting point x 0 if you define x n as g x n minus 1 then x n is going to converge to c as n tends to infinity. Now, we are going to talk about the order of convergence. I had mentioned in the beginning that Newton's method has got quadratic convergence then the secant method it will have convergence better than linear convergence. But less than quadratic convergence so let me now make these statements precise. So, we are going to define what is the order of convergence. So, we have suppose you have got a sequence x 0 x 1 x n is a sequence which is converging to c. We set en to be equal to x n minus c if there exists a constant m not equal to 0 and a real number p such that limit as n tends to infinity modulus of en plus 1 divided by mod en raise to p is equal to m then m is called asymptotic error constant and this p is called the order of convergence. If p is equal to 1 what it means is limit of mod of en plus 1 by mod en should be equal to m. If p is equal to 2 we want mod en plus 1 by mod en square as n tends to infinity and infinity is equal to m. So, this m is asymptotic constant and what we are saying is that limit mod en plus 1 upon mod en raise to p as n tends to infinity is equal to m. This means mod en plus 1 is approximately equal to m times modulus of en raise to p. So, if you have got p is equal to 2 then modulus of en plus 1 will be approximately equal to modulus of en square. This is the order of convergence. So, this is the order of convergence. So, this is error at nth stage. This is error at n plus first stage. Our error is going to be small. So, I can assume mod en to be less than 1. So, here whatever is the error at nth stage at n plus first stage it is becoming approximately square of that. Since mod en is less than 1 mod en square will be still smaller. So, this will be the quadratic convergence. If you have got p is equal to 1 in that case we are going to have p is equal to 1 means modulus of en plus 1 is approximately equal to m times modulus of en. So, let us look at some illustrative examples which will make the idea little more clear. Look at x n is equal to 1 by n. It tends to 0 as n tends to infinity. En plus 1 is going to be equal to x n plus 1 minus 0. So, that is 1 upon n plus 1 divided by en which is 1 by n. I do not need to take modulus because mod en plus 1 is equal to en plus 1. This is equal to n upon n plus 1 which converges to 1 as n tends to infinity. So, that means here p is equal to 1 and the asymptotic error constant m is also equal to 1. So, this is example of linear convergence. Look at 1 upon root n. This 1 upon root n also will tend to 0 as n tends to infinity. When I look at en plus 1 by en this is going to be equal to root n by root of n plus 1. The limit of this as n tends to infinity is again equal to 1. So, for 1 upon root n also it is a linear convergence with p is equal to 1 and the asymptotic error constant m is equal to 1. Here is the third example. If I define x 0 to be equal to 1 by 3, x 1 to be 1 by 3 square, x 2 to be 1 by 3 raise to 4. So, I am defining x 1 is equal to x 0 square, x 2 is equal to x 1 square, x n plus 1 is equal to x n square. Then this x n tends to 0. When I look at en plus 1 upon en square, en plus 1 is equal to 1 by x n plus 1. That means it is x n plus 1 minus 0. So, that is x n square. En is going to be x n. So, en square will be x n square which is equal to 1. So, here the asymptotic constant m is equal to 1 and n p is equal to 2. So, this is the example of a quadratic convergence. Now, in our next lecture what we will show is that if the fixed point which we are looking at. So, g is a function and g of c is equal to c. If g dash c is not equal to 0 then the Picard's iteration has linear convergence. If g dash c is equal to 0 then we will get quadratic convergence and that is going to be the case for the Newton's method. So, in the next lecture we are going to look at conditions, sufficient conditions under which Newton's method converges. Then the rate of convergence of Newton's method, rate of convergence of secant method and we are going to define a method what is known as regular falsely method. Thank you.