 Hi, we are discussing secant method for capturing isolated root of a non-linear equation. In the last class, we have introduced the method and in this class, we will study the convergence theorem of the secant method. We have two assumptions on the convergence of the secant method. One is that the function f which defines our equation is a C 2 function. What is mean by a C 2 function? Well f should be a continuous function, f dash should exist and f dash should also be a continuous function and f double dash should exist and f double dash should also be a continuous function. This is what we meant by saying f is a C 2 function and also we have to assume that the root that we are going to capture denoted by R is a simple root of the non-linear equation f of x equal to 0. What it means? It means f dash of R should not be equal to 0. That is what we meant by saying that the root R is a simple root. Once if these two assumptions are satisfied by the function f, then we can say that there exists a delta such that for every x naught and x 1 in the neighborhood of R that is the delta neighborhood of R, we can say that the secant method iterative sequence is well defined and the second thing is that we can say that the sequence belongs to the interval R minus delta to R plus delta. That is the delta neighborhood from where we have taken our initial guesses and the third one which is what we are mostly interested in is that the sequence x n will converge to R as n tends to infinity and finally, the conclusion is also that the order of convergence of the secant method is something approximately 1.62. That is what we meant by saying that limit n tends to infinity mod x n plus 1 minus R divided by x n minus R to the power of alpha is equal to some constant. Remember f dash of R is not equal to 0. Therefore, this is some constant non-negative constant. So, if you recall what is mean by the order of convergence, we say that a sequence converges with order at least alpha if there exist a constant such that x n plus 1 minus R is less than or equal to the constant times x n minus R to the power of alpha. Another way of defining order is limit n tends to infinity mod x n plus 1 minus R divided by mod x n minus R to the power of alpha is equal to some constant. So, this is what the condition that we are putting here and that says that the secant method is going to converge with order alpha and in fact, in our derivation we will also get the value of alpha as root 5 plus 1 by 2 which is approximately equal to 1.62. That shows that the secant method has a super linear convergence. In fact, bisection method has a linear convergence. In that way you can see that secant method is slightly faster than bisection method. Of course bisection method has its own disadvantage also that it is a bracketing method. Whereas, secant method is a open domain method. Therefore, the initial guesses can be chosen arbitrarily in secant method. This is a very good advantage of secant method. In addition to that it is also nice to see that secant method has super linear convergence. Let us try to prove this theorem. If you are not very good at calculus maybe in the first go you can omit this proof. However, I will strongly recommend you to go through the proof of this theorem very carefully because the techniques that are used in proving this theorem is very important for you to do any convergence analysis on a iterative method especially iterative method for non-linear equations. Therefore, it is very important for you to understand the proof of this theorem at least once. With this note let us go into the proof of this theorem. As a first step let us claim that for every n equal to 2, 3 and so on we can find a zeta n and a xi n such that this expression holds. That is x n plus 1 minus r into f dash of xi n is equal to x n minus 1 minus r into x n minus r by 2 into f double dash of zeta n. This expression will be used in arriving at all the conclusions that we have listed in this statement of the theorem. Therefore, first deriving this expression is very important for us. Let us see how to derive this. Note that to derive this we have to also assume that x n is not equal to x n minus 1 and both are not equal to r. Now, let us start our derivation with the formula of the secant method. Recall that the secant method is given by this expression and this expression can be rewritten like this. You can easily see this. Now, what I will do is in this expression I will replace x n plus 1 by x and then I will define this expression that is the left hand side expression as a function denoted by g of x. Now, you can see that if you plug in x equal to x n plus 1 then clearly this is equal to 0 because that is the way we have defined the function g. Therefore, g has a root which is x n plus 1. So, let us keep this in mind and go ahead also see that g is a linear polynomial. So, it depends on x with degree 1. So, these are the two observations that is g is a linear polynomial and g has the root x equal to x n plus 1 with these two properties in mind let us go to define a function psi as f of t minus g of t. Remember psi is a function of t where we have fixed x n minus 1 and that is not equal to x n. Of course, both of these are not equal to r and now we are also fixing x in such a way that x is not equal to any of these two numbers and then defining psi as a function of t. Therefore, x is fixed as for us this expression is concerned that you keep in mind and how psi of t is defined psi of t is defined with this expression and now let us see some important properties of the function psi. First thing is psi is a c 2 function why it is so because f is a c 2 function this is what we have assumed in our theorem and g is a linear polynomial and also you can see that this third term involves a quadratic term. Therefore, psi is a c 2 function very clearly also you can check that psi of t equal to 0 that is this equation psi of t equal to 0 has three distinct roots what are they one is t equal to x is a root of this equation t equal to x n minus 1 is a root of this equation and t equal to x n is a root of this equation they all are distinct because we have chosen x not equal to x n minus 1 and that is not equal to x n. Now, apply rolls theorem what rolls theorem says if you have a function which has three roots something like this then its derivative that is psi dash of t equal to 0 will have at least two roots distinct roots that is what rolls theorem says because you have three distinct roots for psi. Therefore, in between these two if you apply rolls theorem you will see that there exist at least one point at which psi dash of t equal to 0 and similarly in between these two points you can find at least one point at which psi dash vanishes. So, rolls theorem says that there exist at least two points at which psi dash vanishes. Now, psi dash is something like this at least two points psi 1 and psi 2 at which psi dash vanishes. Now, again apply rolls theorem on psi dash to see that there exist at least one point say eta at which psi double dash vanishes that is what the rolls theorem when applied to psi dash will give us. So, again applying rolls theorem on psi dash we can get a zeta n such that psi double dash of zeta n equal to 0 that is what the rolls theorem says. Now, we got zeta n now what are we going to do with this let us see well we got the zeta n let us compute psi double dash from its expression and plug in zeta n and see what is happening you can see that psi double dash is equal to f double dash right and g double dash that is equal to 0 because g is a linear polynomial. Therefore, second derivative of g is 0 and remember this is a constant because x is not a variable for us as far as this expression is concerned right therefore, this is a constant. So, do not differentiate it because we are differentiating with respect to t not with respect to x and what will happen to this term this term will simply give you two if you differentiate it twice right. Therefore, we have psi double dash of t is equal to f double dash of t minus this expression times 2 right. Now, from there what you can write is f of x minus g of x that is this term is equal to 1 by 2 2 is going to the other side right into this term is also going to the other side after shifting this to the left hand side right. Therefore, we will get f of x minus g of x is equal to 1 by 2 into x minus x n minus 1 into x x minus x n into f double dash of zeta n. What I am doing I am just plugging in t is equal to zeta n and then doing this simple manipulation to get this expression right. Now, let us see how to go ahead I will put x equal to r in this expression remember I have chosen x not equal to x n minus 1 and not equal to x n already these two are chosen such that this is not equal to r. Therefore, I may choose my x equal to r right that is not a problem for me now when I choose x equal to r in this expression you can see that this will go to 0 because f of r equal to 0 right. So, the first term will go off and you will have minus g of r is equal to 1 by 2 into instead of x I am putting r here right that is what I got here. Now, let us see how it goes if you recall we have defined our g such that g of x n plus 1 equal to 0 this is how we have defined g in our first step. Therefore, I can as well write this expression as g of x n plus 1 minus this equal to this because this is just going to contribute 0 here. Therefore, no problem I can write like this and now what happens I got this expression where g is 0 keep in mind. Now, what I will do is I will put mean value theorem for this expression because x n plus 1 is not equal to r that is our assumption as well. Therefore, you can put the mean value theorem and get x n plus 1 minus r into g dash of xi there exist a xi between x n plus 1 and r such that the left hand side can be written like this that is what the mean value theorem and that is equal to I am just keeping the right hand side as it is well we got this expression now. Now, we will also see that g dash of xi is equal to this how will you get that well you directly differentiate the expression g that we have taken in the first slide that is this one differentiate with respect to x and put x equal to xi here. You will see that this is 0 because this is a constant. So, when you differentiate with respect to x this will be 0 and you will be left out with this term only that is what I am writing g dash of xi equal to f of x n minus f of x n minus 1 divided by x n minus x n minus 1. Now, you apply the mean value theorem for this term you can see that there exist a xi n between x n and x n minus 1 such that this is equal to f dash of xi n into x n minus x n minus 1 right this numerator equal to this therefore, x n minus x n minus 1 will get cancelled and you will have g dash of xi equal to f dash of xi n. Now, you put this term into the expression that you got here you will see that x n plus 1 that is this term. Now, instead of g xi you are putting f dash of xi n and the right hand side is kept as it is. So, this is what we wanted to derive right. So, our first part of the proof is over we have derived a expression remember this expression is basically coming from the secant method formula. How we have taken the secant method formula from there we have defined g and through that we have derived this expression therefore, the secant method formula gives us a sequence right that sequence will surely satisfy this expression for some zeta n and xi n that is what we have seen. Now, remember r is a simple root of our equation and f dash is continuous it means what you have a function f say its graph is like this and r is the root of the equation f of x equal to 0 right this is the graph of the function f of x and we know that f dash of r is not equal to 0 right since f dash is continuous by intermediate value theorem you can see that in a small neighborhood of r f dash will remind non 0 that is what the intermediate value theorem tells us that is what we are writing here you can find a small neighborhood of r say r minus delta naught to r plus delta naught in which the function f dash will remind non 0. Remember in the statement of our theorem we have to find a delta neighborhood of r right you remember we have to find a delta neighborhood of r in which if you start your iteration that is if you choose your x naught and x 1 in that delta neighborhood then all these conclusions will hold. Therefore, our aim is to find such a delta that is what we are trying to do now. So, the intermediate value theorem and the assumption that r is a simple root will give us a delta, but remember this is not the delta that we want something which is lesser than this. So, we have to choose some other delta suitably which is less than this delta such that all our conclusion should hold. Now the question is how to choose this delta that is the question let us see how are we going to get this delta let us have some notation here I am just assuming that I got a delta and then I am just defining the notation that is all. This is not something which I am deriving or something it is just a notation I am fixing small m means the minimum of f dash over the delta neighborhood that I am supposed to get now. So, delta is yet to be found, but whenever there is a m it means I am just taking the minimum of f dash in that neighborhood and similarly capital M is the maximum of f double dash. Now let us see how are we going to choose our delta this is where precisely I am going to choose my delta I will choose my delta in such a way that whenever I choose x naught and x 1 in that neighborhood such that x naught and x 1 are not equal to r then the maximum of this term should be less than 1. So, this is how I am going to choose my delta. Now if I choose my delta in such a way then what happens let us see recall we have already derived this expression just as the first part of our proof. Now I am going to use this expression how am I going to use this expression well I will take n equal to 1 in this expression then you can see that mod x 2 because I took n equal to 1 and I took the modulus and that is actually going to be less than or equal to this term why it is so you can see that this is less than or equal to norm x 2 minus r into minimum of f dash what is minimum of f dash minimum of f dash is denoted by m here and similarly I will take the modulus on the right hand side and that is going to be less than or equal to I will replace this by the maximum. Therefore, I will have m by 2 into mod x 1 minus r that is this term and mod x naught minus r that is this term because I have taken n is equal to 1. Therefore, we have this inequality by taking n equal to 1 and now what happens let us see this term is less than epsilon right because epsilon is taken as the maximum of these two. Therefore, this term is something less than epsilon and the other term is surely less than or equal to delta why because x 1 is chosen from the delta neighbor root therefore, mod x 1 minus r is less than or equal to delta. Therefore, this whole quantity is less than or equal to epsilon into delta and remember epsilon is less than 1 that is the way I have chosen my delta right. So, therefore, with that delta my epsilon is less than 1 therefore, this is less than delta and now what does this mean? This means that x 2 belongs to the delta neighbor root of r right that is what it means by saying that x 2 minus r is less than or equal to delta or may be less than delta means it is a open interval that is not important here x 2 lies in this delta neighbor root that is more important and similarly from here you can show that x n also belongs to this delta neighbor root for each n. So, if you go back you can see that this is what we wanted to show as one of the conclusions that the whole sequence belongs to the delta neighbor root remember this delta is chosen such that f dash of xi is not equal to 0 for all xi is not equal to 0. So, f dash of xi in this neighbor root r minus delta to r plus delta right this is how we have chosen delta. So, you keep that in mind you can see that f dash is not equal to 0 in this interval and therefore, f dash is either positive or negative in this neighbor root right it means f is a strictly monotonic function. If f is strictly monotonic then it has this situation where x n minus 1 is not equal to f of x n right because the second conclusion says that all x n belongs to this neighbor root and since all this x n are belonging to this neighbor root f dash will not vanish at x n's it means f of x n minus 1 will not be equal to f of x n because f will be a monotonic function. So, therefore, this conclusion is also done now we have to prove the convergence and this order of convergence also. Let us see how to prove this. So, we have already discussed this let us go to prove the convergence first you observe that mod x 2 minus r is less than or equal to we have already shown here mod x 2 minus r is less than or equal to this quantity. Now, I am multiplying both sides by m by 2 m therefore, you will have a square here that is what I am writing here m by 2 m into mod x 2 minus r is less than or equal to m by 2 m square into this right and now if you recall m by 2 m into x 1 minus r is less than or equal to epsilon and another m by 2 m into x naught minus r is also less than or equal to epsilon. Therefore, this term is less than or equal to epsilon square now take x 3 mod x 3 minus r is nothing, but m by 2 m into x 1 minus r into x 2 minus r and then you multiply both sides by 1 m divided by 2 small m and that will give you epsilon this is less than or equal to epsilon and this is less than or equal to epsilon square how we have just now proved here. Therefore, this entire term is less than or equal to epsilon cube similarly, you can see that m by 2 small m of mod x 4 minus r is less than equal to epsilon 5 and so on. In general, you can write m divided by 2 small m into mod x n plus 1 minus r is less than or equal to epsilon to the power of q n plus 1 what is q n plus 1 well q n plus 1 is a sequence such that q n plus 1 is equal to q n plus q n minus 1 how can you see this you can see that m divided by 2 m into mod x naught minus r is less than or equal to epsilon that is my epsilon to the power of 1 therefore, q 1 is equal to 1. Now, you take m divided by 2 m into mod x 1 minus r which is less than or equal to epsilon 1 therefore, q 2 is also 1. Now, q 3 from here you can see that q 3 is equal to 2 which can be seen as the sum of these two terms now what is q 4 q 4 is coming from here q 4 is equal to 3 that can be seen as the sum of these two terms here and what is q 5 q 5 is equal to 5 that can be seen as the sum of these two terms in general q n plus 1 is seen as the sum of its immediate previous term that is q n plus the previous to previous term that is q n minus 1 with the understanding that q naught is equal to 1 maybe I should start with 0 here 1 2 3 4 and so on sorry therefore, q naught is 1 q 1 is 1 and q 2 onwards we are defining like this. What is this sequence this is the well known Fibonacci sequence right and once you realize that you have a Fibonacci sequence as the power of epsilon now you recall that Fibonacci sequence tends to infinity as n tends to infinity also recall that we have chosen our delta in such a way that epsilon is less than 1 therefore, you have epsilon to the power of something and that is going to infinity as n tends to infinity that implies that the right hand side goes to 0 as n tends to infinity and what is on the left hand side you have constant times this term this is fixed therefore, you can in fact push it to the right hand side and therefore, you can see that this term goes to 0 as n tends to infinity that is precisely what we want as the convergence. Now all reminds is to get the order of convergence get the order of convergence how are we going to get the order of convergence let us see again I will recall what is the definition of order of convergence I have already told this at the beginning of the lecture the definition of the order of convergence is that there should exist a constant c such that mod x n plus 1 minus r is less than equal to c times mod x n minus r to the power of alpha then we say that the sequence converges with order at least alpha that is what we have seen and that should happen as n tends to infinity. Another way to define order of convergence is to use the following definition called asymptotic order of convergence this says that limit n tends to infinity this term should be constant right this both are often used for defining order of convergence let us try to derive or find the constant c such that this happens with an appropriate alpha let us see both of this again you start with the expression that we have derived at the beginning of the proof and then you just take this f dash on the other side with the understanding that f dash never vanishes in the neighborhood that we are working with right remember we are working with r minus delta to r plus delta this is the neighborhood in which we are working all our x n belongs to this neighborhood therefore, xi n which lies between x n and x n minus 1 and also between r in whatever it is it belongs to this interval therefore, f dash of xi n will not vanish right therefore, you can just divide both sides by f dash of xi n and that gives you this expression now take modulus on both sides and divide by mod x n minus r to the power of alpha we have to find what is alpha, but just take alpha and divide both sides you can see when you take modulus you are taking modulus here and also here since you are dividing both sides by x n minus r to the power of alpha you will have x n minus r to the power of 1 minus alpha right where alpha has to be chosen appropriately. So, we have this expression now what we will do is we will write this term in this form I am just rewriting it with an appropriate beta that is all if that is so then what should be our beta you can see that beta should satisfy that beta should be equal to 1 minus alpha why because x n minus r to the power of beta will come and that should be equal to 1 minus alpha from this expression therefore, we want beta to be 1 minus alpha and in the denominator we have alpha into beta that when you compare with this term you want alpha into beta to be minus 1 right now you just have to find alpha such that these two things happen simultaneously that implies that alpha should come as the root of this equation why because alpha should be equal to minus 1 by beta or beta equal to minus 1 by alpha that value of beta you can put here to get this equation. Therefore, the alpha that we want should come as the root of this quadratic equation remember this quadratic equation has two roots and one root is given like this and the other root of this quadratic equation is a negative number we are concerned about the order of convergence therefore, we will not choose the negative value of the alpha. So, we will choose this alpha and this number will surely satisfy this expression which is precisely satisfied by the secant method right because the expression that we have derived if you recall we started with this expression and this expression is derived from the secant method and from this expression only we have landed up with this and further rewritten that expression in this form and that is further written like this therefore, everything is coming from the secant method and the alpha is therefore, chosen very naturally from the secant method and that shows that the secant method if it converges will converge with order alpha and what is this value this value is approximately 1.618 all reminds is to now find the constant c right how will you find it recall that xi n will converge to r and also zeta n will converge to r how can you see you can see it from the sandwich theorem because how we have chosen xi n and zeta n well xi n and zeta n both lies between x n and x n minus 1 right. But we know that x n converges to r that we have already shown and also x n minus converges to r right and these two numbers always lie between these two numbers therefore, they will also converge to r as n tends to infinity that is what is very clear from the sandwich theorem. Now you take the limit n tends to infinity in this expression you can see that this is equal to f double dash of r divided by 2 into f dash of r now putting that into your expression because you are taking limit as n tends to infinity it will become like this into this term remember we had beta here, but beta is equal to minus 1 by alpha that is what I am putting here and therefore, we want the constant c such that see remember you want the whole thing to be equal to some constant right and that constant should be such that you have c which is this one this should be equal to c therefore, c is equal to this term and this is nothing, but this right this to the power of minus 1 by alpha therefore, we want the constant c such that c equal to this term into c to the power of minus 1 by alpha that will immediately give us what is c, c is given by this this is precisely what we wanted to show in the statement of our theorem well this is little long, but very interesting and important for us to understand with this I will end this lecture thank you for your attention.