 In the last segment, we discussed the bisection method for finding roots. This is a very simple method. In this segment, we are going to discuss Newton-Raphson which is perhaps the most sophisticated amongst the methods that we have discussed in this lecture sequence. So the Newton-Raphson method, it is again a method for finding the root of some function f of x that is finding x such that f of x equals 0. This method is going to work if the following conditions are met. We should be able to evaluate f of x and its derivative f dash x and we should have available a good initial guess. So sometimes this condition is not quite necessary but it definitely helps. So if we are close to the root, then this method will quickly take us even closer, very close. But if we are far then this method occasionally may wander around too much. So again, we are going to use this method to find the square root of y. So as before, we are going to say that our function is f of x squared is f of x is x squared minus y. Now in this case, we want the derivative as well and so we might as well write this down immediately. So f dash of x is 2x. So notice that given an x value, we can evaluate f of x as well as f dash of x quite easily in 2-3 arithmetic operations really. So we also need an initial guess and for this square root finding problem, it turns out that x not equal to 1 is a good guess. Actually for this problem, the guess requirement is easy limit. I think essentially any number will be okay. Alright, so this is again an iterative method and the key step over here is to get a better approximation of the root than the one that has been given to us. So let us say we have been given xi which is our current approximation to the root. So now I am going to tell you how we can get a better approximation and that better approximation we are going to call xi plus 1. So initially we know this point A. So A is xi 0, x coordinate is xi and y coordinate is 0. It is a point on the x axis. Next we are going to calculate f of xi. So f of xi is just the function value at this point or it is this height, height up to this point. So in fact it is exactly this point. The B point is the point over here because it is x coordinate is the same as that of A and the height is f of xi. So we can determine the point B if we know xi because we can get the coordinates of point B. Next step is we are going to approximate this function by a tangent at this point. So we are going to approximate it by a tangent and we will ask where does that tangent need the x axis. So let us say xi is the intercept of that tangent on the x axis. So c is going to be our new guess. So its coordinate is going to be xi plus 1, 0. I have not told you exactly what that value is but that is what it is going to be and of course the y coordinate is going to be 0. So all that remains now is that I have told you how to geometrically get to c but I have not told you how to algebraically get to c. So that is all that remains. So you can observe that xi plus 1 is this distance from origin to c. So it is this distance origin to a minus ac. So that is what this first equality gives me. xi plus 1 is xi minus ac. Now I am going to write ac in this funny form. I am going to write it as ab upon ab upon ac. Of course if you simplify this you will get ac but then why did I write it in this funny form? Well I did that because I know what the value of ab is for one thing. What is the value of ab? Well it is just f of xi. So I am going to put f of xi instead of this ab over here and what is ab upon ac? Ab upon ac is this height upon this and that is simply the slope of this tangent or it is the value of the derivative at point xi. So that is what I have written down over here. So ab upon ac is f dash of xi. So what have you got? We now have a formula for xi plus 1 in terms of xi f of xi and f dash xi. So xi plus 1 is xi minus f of xi upon f dash xi. So this is the formula which we are going to use to get to a better approximation. And in this picture you can see that c is likely to be a much much better approximation. In fact if the curve over here is almost a straight line then we would get to the root in one shot. But of course it will not be a straight line in general but you can see that we are likely to get closer to the root. So let us now use this formula xi plus 1 equal to xi minus f of xi upon f dash xi to find the square root of a number. So square root of a number y. So that is our formula and we want to find the square root of y. So for that we said that our function f of x is x squared minus y and we said that its derivative is 2x. So all that remains is now to substitute these quantities into this expression. So what do we get for xi f of xi xi squared xi squared minus y and for every dash of x we get 2x. So we get xi plus 1 is xi minus xi squared minus y upon 2xi. And this if you substitute is going to turn out to this y. So xi squared upon 2xi is going to give me xi upon 2. So that will take off half of xi from this. So I am going to get an xi but notice that there is an upon 2 over here. And what about this side? So here I am going to get a y but there is an upon 2xi over here. But this y on the other this minus sign and this minus sign will cancel each other out. So I am going to get y upon xi with that 2. So xi plus 1 is just going to be xi plus y upon xi divided by 2. So our basic iteration is quite simple. So starting with x0 we are going to use this to compute x1 then compute x2 and so on. And it turns out that if we keep on doing this we can get as close to square root of y as required. Okay? The proof is a little bit involved but calculus, a little bit of calculus will help. Certainly first year calculus of science and engineering should be adequate to prove this. But anyway it is not a part of the programming course. The proof is not a part of the programming course. So the code is very simple. So we are going to have a y in which we are going to read the value whose square root we want to calculate. Okay? Then our xi is going to be kept in a variable xi which we are going to start off with 1. This is a guess and as I said for this square root almost anything works and we are going to stick with x because this does work. And then first I am going to repeat 10 times xi equal to our basic iteration. So this is the old value of xi that we are going to use. We are going to calculate the new value of xi and right away we are going to put it into xi. So we are just going to do this 10 times. Okay? And at the end we are going to print out the result. Okay? So this is fairly straightforward. Okay? Now we have said that we are going to do this 10 times. Okay? So if we are going to do this 10 times we do not really know how much the error is going to be. Okay? Ideally we would like to say that look do this until the error is small. Okay? Let us see if we can get not exactly that condition but some similar condition. Okay? So here what I have done is I have plotted my function f of x. Okay? So this curved line is my function f of x. Okay? What is that function? Xi squared minus y. That is my function. Okay? This curved line. Okay? Now my current xi is this. The actual xi that I want, the actual root that I want is this. Okay? So I want this distance because that is my error or I want an estimate of this distance. I can never get the exact error because if I got the exact error I would just add it to my current estimate and then I will get the exact answer. So that is not, we are not going to be so lucky. Okay? So but if we get an upper bound on the error that is also fine. But here even determining an upper bound on distance is a little tricky at least not immediately obvious. Instead of that what we can see is that this distance is relatively easy to calculate. Well almost. So how much is this distance? So this function is xi squared minus y. So it is the distance between the square of my current guess and the root. So it is sort of an error. It is sort of the vertical distance from this point to this curve. What I really want is the horizontal distance from this point to this curve whereas this value is the vertical distance from this point to this curve. So I can do that. I can use that if the slope is not too bad and in case of the square root it is okay I guess. So if the slope is not too bad then I can in fact get a good estimate from this okay. So here is how my function might change. So my program will again ask for y. It will again set xi to 1. But now instead of running for 10 iterations it is going to say okay have I come close to this curve okay. Has my xi come close to this curve along the y axis in the y direction. I really want in the x direction but have I come close in the y direction. And if I have not come close and whatever error we want we can put over here then we are going to iterate again and again okay until this condition is not, it becomes false okay. At the end of it we know that our vertical error has become smaller than this and even though we want the horizontal error to become small that is okay at least we are making some effort okay. But you have to remember that this is not the actual error. This is some kind of a very heuristic estimate of what the actual error might be. So in any case at the end of it we are going to put out our current xi as our answer okay. Now the error analysis can be done and in fact it turns out that the number of correct bits essentially doubles with each iteration. So now that is a remarkable result. Its proof is not easy but with a little bit of effort and you should be able to understand it, it is given in several places okay. But let me just tell you what that result is saying. So it is saying that suppose you start off with one bit correct then if you run it once you will get two bits, if you run it twice you will get four bits, if you run it three times you will get eight bits, if you run it four times you will get sixteen bits, if you run it five times you will get 32 bits, if you run it six times you will get 64 bits correct okay. So basically it says that you have to run this just about six times, only six iterations have to happen okay. So that is that is really fast convergence to the correct answer okay. It is not exactly doubling so it may be it will take seven or eight but not much more than that okay. So let me make some remarks on this some interesting points. So one interesting point is that Babylonians use this method to find square root 3500 years ago okay. They did not know this whole tangents, tangency and they do not it does not seem that they knew that they could be that this could be used to find the roots of any function for which we can evaluate f and f dash okay. So for that for that realization we had to wait for Newton and Raphson. But 3500 years ago getting square roots as accurately as you want is an amazing achievement okay. Now I should mention that Newton's method is very commonly used okay many many places okay. And it is also useful in multiple dimensions okay. So what is the multiple dimensions problem? We are given several functions of several variables okay. So we want to find values to x, y, z, w okay. One set of values to x, y, z, w such that this equation is satisfied, this equation is satisfied, this equation is satisfied okay. So x, y, z, w the set of numbers defines a point in a high dimensional space and what we want is a single point in this high dimensional space at which all of these functions f, g, h get take the value 0 okay. So essentially we want to find a point in high dimensional space which is a simultaneous root of all these functions f, g, h. So you can use Newton's method to solve such problems. However solving such problems is very tricky okay. So you have to use Newton's method very carefully and sometimes it just does not work okay. But anyway Newton's method is a great method very very useful and it is also remarkable that one part of it or one aspect of it was known to Babylonians 3500 years ago okay. So that concludes this lecture sequence. Let me just make some final remarks. So in general what we talked about here or sort of the major problem that we talked about here is to evaluate f of x where f of x is some interesting mathematical function but which is not easy to calculate directly. So if you want to do that then you can use Taylor series if f and its derivatives can be evaluated at some point x naught of your choosing okay. And that point should be close to x or x should be in that radius of convergence that point x naught okay for that function f okay. Then we also said that we can express f as an integral of some easily valuable function not easily integrable easily valuable function. Then we can numerically integrate this function g and we can calculate f. We can also calculate f we can express it as the root of some easily value valuable function g and then you can use the bisection method or Newton Raphson method okay. We also said and it is worth definitely keeping an important keeping in mind that all these methods are iterative okay. So the accuracy of the answer improves with each iteration okay. So we do not get the exact answer but we can get answers which are as close to the actual answer as we want. And since we are representing numbers only to a small number of bits or say a large number of bits whatever it is still we do not really care for answers which are more precise than what is possible in that many bits. And therefore these methods are actually quite good enough for using on a computer. We do not really have to worry about deriving some kind of exact methods because there is no such exact thing on a computer anyway at least if we use the standard data types. Now we also said that the ideas of these chapters are very fundamental and in fact if you do numerical calculation you will see them again and again and even in the book they appear in 2 places chapter 19 and chapter 29. And I would also like to say that the ideas might be mathematically involved okay. The programs are actually quite simple once you understand what is to be done the programs are quite simple and they essentially have one loop okay which could be a which you could use as a for loop or you could use even the repeat loop sometime but certainly a while loop or a for loop and I suggest you get practice in writing all these things using for loops and while loops and though that was also one of the motivations we had in doing all this to get practice and learn programming and get practice in writing loops, writing simple loops with a small number of variables. Okay, so that concludes this lecture sequence. Thank you.