 Hi, we are discussing iterative methods for approximating an isolated root of a non-linear equation. In this, we have covered bisection method, regular falsi method and secant method. In this lecture, we will discuss Newton-Raphson method. Let us first get a motivation for the Newton-Raphson method starting from the secant method. If you recall, in the last class, we have proved that the order of convergence of the secant method is 1.62. Now, the question is can we modify this method to get a quadratic convergence? Well, the answer is yes. Now, the question is how to do that? Let us quickly recall from the secant method, you can see that for a large n, if the sequence is converging, then you can see that x n minus 1 will approach x n. In that case, the expression given like this can be more closer to f dash of x n, because as n increases, you can see that x n minus 1 is approaching to x n. Therefore, this is actually equal to limit x n minus 1 tends to x n. Therefore, as n increases, this term will be more like f dash of x n. Now, look at the formula for secant method. You can see that the secant method involves 1 by this term. You can see that that is what is sitting here. 1 divided by this term is what is given here. Therefore, you can see that this term is more behaving like 1 by f dash of x n for large n provided the sequence is converging. Now, the question is why not we replace this term by f dash of x n itself right from the first term of the sequence. That is instead of having this term, you just replace that by 1 by f dash of x n. And that is what is called the Newton-Raphson's method. Newton-Raphson method is highly preferred in the practical applications mainly because of the two reasons. One is that the Newton-Raphson method has quadratic convergence. We will also prove this later in this lecture. Also, Newton-Raphson method gives us a process called linearization process. This is particularly very important because in many applications we come across complicated non-linear models. One good idea for this models is to first look for the linearized model of those models and then recover the non-linearity through an iterative procedure. A Newton-Raphson method is precisely based on this idea. Therefore, understanding Newton-Raphson method is very important. Newton-Raphson method is first introduced by the well known scientist Sir Isaac Newton. He formulated this method for obtaining roots for polynomials. He first started with an initial guess and then formulated sequence of error corrections. And this way he obtains the roots of polynomials. He introduced this procedure both for numerical computation and also for algebraic calculations. In the case of algebraic calculations, he in fact showed that he can recover the Taylor polynomial of a given non-linear function. But his procedure is quite complicated and that was further simplified by a mathematician called Joseph Raphson. Not much is known about this mathematician, but he gave a much simpler idea of the method which was introduced by Newton. But he also formulated the method for polynomials. Both Newton and Raphson never connected this method to calculus and gave a closed form of the formula that we have shown in the last slide. It was Thomas Simpson who connected this idea to calculus and gave the formula that we showed in the previous slide. And he also introduced this method to solve any non-linear equation. With this small historic note, let us go to give a formal derivation of the Newton-Raphson method. Well, let us start with a initial guess x naught. Now, around that point x naught, let us write the Taylor's formula for the function f of x. If you recall f of x, where x is in a small neighborhood of x naught can be written as the Taylor polynomial of order 1 of x plus the reminder term. Now, if you just neglect this reminder term, then you can see that f of x is well approximately equal to this linear polynomial. So, this is what we meant by linearization of a non-linear function. The non-linear term is inbuilt in this expression which is well unknown because this xi is unknown quantity lying between x naught and x. But the non-linearity is inbuilt in this expression which is neglected in that way we are only taking the linear part of the function f. And this idea is what we call as the linearization. Now, the idea is instead of looking for the root of the equation f of x equal to 0, we will first look for the root of this polynomial T 1 of x equal to 0. Well, you can find the root of this polynomial without any effort. Let us call the root of this polynomial by x 1 and this x 1 can be written as x naught minus f of x naught divided by f dash of x naught. Well, this can be obtained by directly rewriting this linear polynomial equation that is how we get the first term of our iteration sequence x 1. Now, from here you know how to find the next term of the iteration that is x 2. How will you find it? Well, you will plug in this x 1 into the right hand side expression replacing x naught by x 1 and you will get x 2. Once you get x 2, again you plug in x 2 on the right hand side get x 3 and like that you can now generate the sequence x n right. Let us see the geometrical interpretation of the Newton-Raphson method. Let us take a function f whose graph looks like this. This is y equal to f of x and it has a simple root r here. Now, what we are doing is we are starting our initial guess as x naught and what we are doing is instead of going along this non-linear function we are going along this straight line right. What is this straight line? This straight line is precisely the tangent line at the point x naught right. So, this tangent line if you recall it is given by y is equal to f of x naught plus f dash of x naught into x minus x naught right. So, this is the equation of this tangent line and what you are doing precisely is taking y equal to 0 that is how you are getting this equation and correspondingly this x is taken as x 1. What is the geometrical meaning of this? This is precisely the point of intersection of this tangent line with the x axis that is x 1 is precisely the point of intersection of this tangent line with the x axis right. That is the geometrical interpretation of this formula. Now, if you generalize this you will get the formula for Newton-Raphson method. Let us put the Newton-Raphson method in the form of an algorithm for that the inputs are the function f which defines the equation f of x equal to 0 and since we want f dash of x which is involved in our formula. Therefore, we will assume that f dash exists and for the theoretical reason we also assume that f dash is also continuous function also generally we are interested in computing the simple root of our equation. This is also just for the sake of theoretical purpose and then we have to take the initial guess that is the real number x naught. Generally from the implementation point of view x naught has to be chosen arbitrarily, but theoretically we have to choose x naught sufficiently close to the root r in order to have the convergence, but that cannot be achieved practically because we do not know r. Therefore, this condition is only theoretically feasible. However, practically we have to choose x naught arbitrarily once we are given f and x naught then the algorithm goes like this for any given x naught you will compute the iterations using this formula that is given x n you will find x n plus 1 using this formula where n runs from 0 to infinity right. So, if you take n equal to 0 you have x naught minus f of x naught divided by f dash of x naught and that gives you x 1 then once you get x 1 you will take n equal to 1 therefore, this expression will give you x 2 then and so on. So, in this way you can generate a sequence for any given n and on a computer this procedure can go on without end right, but practically we have to stop the iteration at some n for that we need a stopping criteria if you recall while discussing regular falsely method we have discussed some stopping criteria in order to break our iteration computationally one can follow any one of the stopping criteria we have introduced in that lecture. For instance relative Cauchy criteria is preferred of course along with the residual error criteria also, but practically it is often enough for us to check the relative Cauchy criteria to stop the iteration well let us take an example let us take this equation x cube x plus 1 by 2 x power 10 minus 0.85 equal to 0 here f of x is nothing, but this expression once this is given we have to first compute f dash of x remember f dash of x is involved in our formula right. Therefore, we have to compute first f dash of x and then also to start the iteration we need an initial guess I have chosen in this example the initial guess x naught as 1.25. Now, we are ready to set up our iterative procedure recall the Newton-Raphson method formula that generates the iteration sequence is given by this in our example this can be written like this where this is our f of x and the denominator is f dash of x right. So, that is what we have substituted from this expressions with x equal to x n and we got the iterative formula for the given equation using Newton-Raphson method. Now, let us go to compute the iterations for that first take x naught and plug in x naught into this expression with n is equal to 0 remember sin x naught is computed with radians and that will give us x 1. Let us also see this geometrically the graph of the function f given like this is shown in blue solid line and the root that we are interested is highlighted here in green color and its value is approximately 0.9529 and remember we decided that we will start our iteration with x naught equal to 1.25 and it is here with all this we can now compute x 1 using this expression right and the value of x 1 is given like this and let us see the geometrical interpretation of this x 1 well we started with x naught and you have to take this point and draw the tangent line at this point if you recall we have already mentioned this geometrical interpretation in our previous slide and what is x 1? x 1 is precisely the point of intersection of this tangent line with the x axis and that is given by this point right. Once you get x 1 you go back to that expression now replace x n by x 1 in this expression and that you can put the value of x 1 into this expression and you can use a calculator or computer to obtain the value of this expression with x 1 is equal to 1.12767 and that gives you x 2 as approximately 1.02953 geometrically what is happening well this was x naught where we started from there we approach to x 1 and now to get x 2 you have to take this point draw the tangent line and see the point of intersection of that tangent line with the x axis and that is your x 2. This is the geometrical viewpoint but analytically you simply take x 1 substitute in the formula and get the value x 2 by computing the expression right. Once you get x 2 again substitute x 2 into the Newton Raphson formula for our example and that gives you x 3 what is x 3 again you take this point x 2 draw the tangent line which is given like this and see the point of intersection of this line and that is your x 3. Now you see you started from x naught you got x 1 and from there you got x 2 and from there you got x 3 you can see how this sequence is converging towards the root of the equation right. So, that is the geometrical viewpoint of the Newton Raphson method in this particular example. Let us go to the next example let us take the equation sin x plus x square minus 1 equal to 0 and let us take the initial guess as x naught equal to 1. If you recall we have taken this example to compute a simple root using secant method also just to compare Newton Raphson method with secant method I am considering the same example and if you recall in secant method we have taken x naught equal to 0 and x 1 equal to 1. Recall in secant method we have to give 2 initial guesses therefore, we have chosen the initial guesses like this, but in Newton Raphson method it is enough to give 1 initial guess therefore, I am taking x naught as 1. Now let us see how Newton Raphson method sequence goes on I have computed first three iterations of the Newton Raphson method in this example starting with x naught equal to 1. You can see that in the third iteration we got the root exactly up to around 6 significant digits. One can see that the root of this equation is up to 6 significant digits is given like this and we have captured the root exactly up to this accuracy very well with just three iterations. If you recall in secant method the third iteration gave us this value which is still slightly away from the root. So, this is what we meant by saying that Newton Raphson method is slightly faster than the secant method. If you recall we have proved that the order of convergence of secant method is 1.62. In fact, we can prove that the order of convergence of Newton Raphson method is 2 that is Newton Raphson method has quadratic order of convergence and in this example also you can see that Newton Raphson method is going little faster than the secant method. With this let us now state the convergence theorem for Newton Raphson method let F be a C 2 function recall to set up the Newton Raphson iteration we just need F dash of x n that is we just need the first derivative of the function F, but for the convergence we need one more order of smoothness of F that is why we have assumed that F is a C 2 function and also we will assume that the root R which we are interested in is a simple root. It means what it means F dash of R is not equal to 0. If these two conditions hold for our function F then the theorem concludes that there exists a small neighborhood of the root R that is what we meant by saying that there exists a delta such that R minus delta to R plus delta is a small neighborhood delta neighborhood of R and if you start your iteration that is if your initial guess is taken in this small neighborhood remember we have to find this neighborhood delta ok. There exist means we have to find this delta neighborhood such that if you start your iteration with your initial guess x naught in this neighborhood then everything will go nicely that is what the theorem says what is mean by everything will go nicely let us see first thing is each term of the Newton Raphson sequence is well defined ok. What is mean by this well what happens if you started with a x naught then you went to x 1 then you went to x 2 like that you kept on going at some x n you see that your x n plus 1 is infinity then what happens well then the Newton Raphson iteration sequence is not well defined. The theorem says that if you start your initial guess pretty close to the root then this will never happen that is what it says well the question is when such situation will happen let us see suppose we are working with a function f whose graph is like this you can see that it is a nice smooth function at least visually you can see that and suppose we are interested in capturing this root R. Now you started with some x naught say and you started computing x 1 x 2 and so on at some n say x n is this then what happens your f of x n is this and now x n plus 1 is nothing but the point of intersection of the tangent line with x axis right. What is this tangent line tangent line is parallel to the x axis and therefore, it never intersects x axis that means you can never get x n plus 1 at all if you face such a situation this is a geometrical illustration what is happening analytically you see in this case you will have x n plus 1 is equal to x n minus f of x n divided by f dash of x n right what is f dash of x n here that is equal to 0. So, this becomes 0 here and that will give you infinity this is how the Newton-Raphson iteration sequence may fail to exist the theorem says that I can find a small neighborhood of R as long as I start my initial guess within that neighborhood then such a situation will never occur that is what it says suppose you see if you restrict yourself to this neighborhood and start your x n here x naught here then you see your x 1 will be somewhere here and your x 2 will be somewhere here and your x 3 will be somewhere here everything will lie in this interval only that is all your x n will lie in this interval only and they all will be well defined ok. So, that is what we are going to see. So, as long as you start your iteration with the initial guess x naught in a small neighborhood like this then your Newton-Raphson iteration sequence will be well defined the sequence x n will remind within this interval this is what I have shown geometrically in the previous slide and the important point is that the sequence will also converge to the root and finally, this gives us the quadratic convergence of the Newton-Raphson method well it means what this quantity is equal to constant right some finite number that is what we have to show in order to conclude that the sequence converges with this order you can see now here it is 2 therefore, the order is 2 why this term is finite because we assumed f dash of r is not equal to 0 right and f double dash is a continuous function that is what we assumed here that f is a c 2 function and we are always restricting to a closed and bounded interval therefore, this is a finite quantity this is also non-zero finite and therefore, this entire thing will be finite. So, if you prove this expression it precisely mean that the Newton-Raphson iterative sequence converges quadratically I hope you understood the statement of the convergence theorem of the Newton-Raphson method we will prove this theorem in the next class thank you for your attention.