 Hi, I'm Zor, welcome to a new Zor education. Among many different theorems, which we were discussing, which are related to derivatives, there is one which I call method, well, not I call, everybody calls it method, Newton's method. Well, it's related to finding zeros of the function. Now, obviously, in those cases when you can find zeros of the function just analytically, the method doesn't really make much sense, because this is actually the method which allows you to find zeros of a wide range of functions, especially in those cases when there is no analytical solution. Alright, as always, let me just make this preamble that this particular lecture is part of the course of advanced mathematics, which is presented on unizor.com. I suggest you to watch this lecture from this website, because it has the text for notes for each lecture, which basically constitutes like a textbook for you. Plus, science student can take exams, for instance, site is completely free and no advertising, so feel free to use the website rather than just the YouTube video if you have found it on YouTube. Alright, so, again, the purpose of this so-called Newton's method is to find the solution of function of equation f of x is equal to zero. Now, in some cases, obviously this can be found without any kind of specific methodology just analytically, but in many cases analytical solutions is difficult, in which case the Newton's method is really applicable. It is heavily related to derivatives, and that's why actually this particular method is part of the so-called main theorem related to derivatives. The word of warning, however, this is not a universally working method under any circumstances, etc. There are certain restrictions, and there are certain rules to apply this particular method, and I will go into this in a little bit later in some details. So, now to present this method, I have chosen actually the way where analytical solution is possible. It's a very easy example, but it illustrates quite well the methodology, and then we can just talk about some other functions, for instance, where the method can be applied. So, let me start with a very, very simple equation, which I'm sure everybody can solve even without using paper and pen. Obviously, x is equal to two is a solution of this particular equation. But I would like to approach this as if I don't know this solution, so let's pretend that I don't know how to solve this analytically, and I would like to solve it using this Newton's method. So, first of all, let me just explain graphically what it actually means. Well, graphically, this particular function, so it's one, two, three, four, five. One, two, three, four, that's enough for me. So, let me just draw this particular graph of this function. It goes like this. At point two, it's equal to zero, at point four, it's equal to eight minus four, four, and this is a straight line. So, what I'm looking for, I'm looking for this value, where the function intersects with the graph of the function, intersects the x-axis, right? That's exactly where the function is equal to zero, and the value of x at this particular point is the solution, which I pretend that I don't know. So, here's how we can approach it. Let's consider this triangle A, B, C. What do I know about this triangle if my point A is chosen completely arbitrarily? So, I just chose point A at point x is equal to four. So, let's start with x zero is equal to four. I've chosen it completely off the blue. Well, I can definitely substitute four as x and get the value of A, B, right? So, point A has coordinates zero, four. Point B has coordinates, oh, sorry, it's four zero. Four zero, not zero four. I fixed the x-coordinate. Four, what am I writing? Four zero. Point B, let's call it inertia. Okay, point B has coordinates four four. How I got that? I substituted x is equal to four into this equation. I mean, not the equation, the function actually. I don't know when it's equal to zero. And got four. Two times four minus four, that's four. So, I know A, B. Okay, that's very interesting. So, I know the calculus of this right triangle. Now, how about angle B, C, A? Angle B, C, A has, let's call it phi. So, what I know is that tangent of phi is equal to A, B over A, C, right? At the same time, now we're going to derivatives. What is this particular angle, tangent of this angle? Tangent is a derivative at point C, which is the same as derivative at point B because it's a straight line, right? Tangent is exactly the same. Tangent is also the same angle phi, right? So, it's equal to a derivative at point F zero. Which I know in this case it's equal to, what's the derivative? Derivative of this function is equal to two, right? Now, knowing a catatous and the tangent of this angle, I can find out A, C. A, C is equal to A, B divided by tangent phi, right? From here. A, C is equal to A, B divided by tangent phi, which is, I know A, B is four. I know the tangent is two, which is four divided by two, which is two. Now, knowing A, C, I can very easily determine the coordinate at point C. Coordinate at point C is equal to X coordinate of A minus this, which is four. That's coordinate of point A, X coordinate, minus the length of this segment, which is A, C, which is two, minus two, and it's equal to two. And two is a solution to our original equality. Now, what's basically, let's call this particular point X1. So what did I do? X1 is equal to what? It's equal to original coordinate of the point A, which is X0, minus AB, which is value of the function at my point X0. So this is X0 and this is X1. So it's value of the function at point X0 divided by tangent, which is a derivative at point X0. So that's basically what I did. These are my calculations. So first, I chose completely arbitrarily point X0, in this case is equal to four. Then I took the value of the function at that point, which is function of X0, that's AB. And to get the AC, I divided it by the tangent of this angle, which is the same as tangent of this angle, which is the same as derivative of my function. And I know the function, so I know it's derivative, it's two. In this case, constant, by the way. But in any case, it's derivative taken at this particular point. And if I subtract this segment length from the original coordinate of the point A, I will get coordinate of the point C. Now, what's very remarkable about this? This is everything which I know how to calculate for any X0, right? No matter what X0 I take, four or five, I mean, I can take five, for instance. What if I take five? If I take five, I will have five minus five divided by, no, that's not five. Five, that's ten, that's six, sorry, six. That's six. Divided by derivative, which is the same two. So five minus three, which is exactly the same. Once more, we got exactly the same two. So, regardless of where exactly I chose this point A, I can calculate this because everything here depends on the point A only. That's coordinate of the point A, that's the function at point A, and that's a derivative at point A. So, chosen any point A, I can find X1. And that's very, very important. It gives you an algorithm, basically, to solve this equation without basically solving it. What I have to do is I have to calculate certain things. I have to calculate the value of the function at any given point and the value of its derivative at that point. And then my calculations are from here. Now, obviously, nobody is doing this on linear functions. So let me just make a little bit more complicated example and we will see how it's applicable. Okay. My next function is 2 9s X square plus 2 9s X minus 4 9s. Here is the reason why I chose these rational numbers. It will be obvious from the graph. So let me just put one, two, three, four. One, two, three, four. Now, what's interesting about this is that my old line, which was this, is actually a tangential line to this particular quadratic polynomial. I specifically chose quadratic polynomial in such a way. Obviously, I made some calculations, so I chose it. So my tangential line would be exactly like this. So this, my old line, straight line, is a tangent for the parabola, which is the graph of this function. Okay? Now, for instance, I don't know how to solve quadratic equation. So I have to solve this f of x is equal to zero equation. And I have no idea how to solve quadratic equation. By the way, in case of cubic equation, if it's the third degree, it's really difficult to do it analytically. I mean, there is some formula, Cardano's formula, but nobody's using it anywhere right now. So everybody's doing some approximation, right? So that's exactly what I'm looking for. I'm looking for approximation. So what do I notice here? I notice that the tangential line to this parabola basically goes to the same direction from any randomly chosen point. In this case, I have chosen again my point x zero exactly equals to four as before. So both tangential line and the parabola are going towards left from this point. So I expect basically that when parabola is intersecting my x-axis, and that's the solution which I'm looking for, right? Where exactly f of x is equal to zero. That's the point x is equal to one, by the way. I mean, if you substitute one, it will be zero. So let's pretend we don't know this. But in any case, one is the solution. So my point is the tangential line, that's actually the structure of tangential line and that's how Newton was actually thinking along these particular lines of reasoning. It goes towards the same direction from the first arbitrarily chosen point towards the x-axis, which means that this particular point is closer to the solution than original. That's his idea. In many cases, that's actually true. But let's assume that this is the true statement. So we have come closer to the real solution than before. Which means what? Which means we can actually start repeating the same action. So I will find the perpendicular same thing as this and start tangential line again. And again I should get closer to my solution. So that was his original idea. So let's just try to calculate. So x zero is equal to four, x one is equal to x zero minus function at point x zero divided by tangential line divided by tangent of this angle. Now, what's the tangent of this angle? Well, that's actually a derivative at this point, right? Which is equal to what? Four minus function of x zero. If this is equal to four, it's equal to sixteen nines minus no, thirty-two, thirty-two nines. That's sixteen, thirty-two. Now this is eight minus eight nines and plus four nines, right? Four minus divided. Okay, let's put it this way. Because I have to divide it by the derivative at point x zero. This one, derivative is equal to, let's just put it down, f of x is equal to four, nine, x plus two nines. So at point x is equal to x zero, which is four, that's sixteen nines. So that should be, let me write it down this way. Sixteen nines plus two nines. Four minus. So that's how it will be, right? So f of x zero is this one and divided by, I think it's four thirty-six divided by eighteen. So that's actually two, which is equal to four minus two, which is equal to two. And that's absolutely the same, by the way, as we have completed the previous example, if you remember. And as we should, because I said that I have chosen this particular parabola in such a way that the tangential line is exactly as my first example. So obviously I've got two and that's how it's supposed to be. But now I would like to repeat the same thing. Assuming that by taking as before going up perpendicularly to the intersection with my parabola from this point, so my x one is equal to two. Okay, now x two, I will do exactly the same. And let's see what it will be equal to. So my f of x one, x one is equal to two, so it would be so x one two minus, so two is four, eight nines. This is two plus four nines, minus four nines. And here, if I have put two, I will have ten nines, right? Eight nines plus two nines. So this is eight over ten, right? This is eight over ten, four fifths. So it's two minus four fifths, which is what? Six fifths. Well, yes. Six fifths is a little bit greater than one. So this is my six fifths. And yes, indeed, I am closer to one. Now, if I will do exactly the same again, so starting from the point x is equal to six fifths, I put the perpendicular up to intersection with parabola, then the tangential line, and I will repeat exactly the same calculations. And I'm not going to do it right now because I did it before. And x3 is equal to 1.01176. As you see, we are within one hundredths of the solution. And as you understand, if you repeat this process as many times as you want, you get as closer to one as you want. This is exactly the Newton's method. This is exactly the key to the whole thing. This is how contemporary computers, for instance, can calculate the roots of certain equations, zero functions. So as you see, we are getting closer and closer. So my point is that the final formula for the Newton's method is the following. xn plus 1 is equal to xn minus function of xn divided by its derivative of xn. This is a recursive process. You start from any point x0 equal to whatever you want. Well, I will explain not exactly whatever you want. But for instance, you start somewhere, you don't know the solution. You start somewhere. And then you follow this process. And as you see, the right side depends only on the point which you have chosen. If it's x0, then everything depends on x0 and the function and the derivative at point x0. So there is no unknowns thing. And that's how you get x1. From x1, you get x2 from x2, x3, etc. Well, let me start first with some good things. So for instance, everything is good and you are approaching the solution as in this particular case. When should you stop? Well, actually, wherever you want. It depends on certain rules for precision which you would like actually to use. If your precision, for instance, is, okay, I will be satisfied if my result is within, let's say, one thousandths of zero. So you substitute this into your equation and see how far from zero you are. If you are within one thousandths of zero, you stop. Or you might say that I can stop if my next value differs from the previous value by, let's say, one thousandths. Or you might actually require both. Your value is less than certain value and the difference between successive x-coordinates. And if you substitute it into the function, you get no more than certain precision boundaries within zero. So it's basically up to you, but these are natural rules which everybody established for himself based on concrete application. Now, what's the bad news? Well, the bad news is that this method is not necessarily working in all the cases. And that's what I'm going to explain to you. Okay, let's say you have chosen a point arbitrarily and your tangential line, instead of hitting the x-axis, is actually parallel. What if you chose this particular x and your tangential line is parallel? Well, it means it's parallel to x-axis, which means there is no intersection and you can't really step from x-zero to x-one. There is no x-one. Well, what should you do in this particular case? Well, try to use another point as your initial starting point. So instead of x-zero equal to whatever it is in this particular case, by the way, where is the derivative is equal to zero in this particular case? Well, it's equal to zero when x is equal to minus one-half, right? So if your initial point x-zero is minus one-half, you will have this derivative equal to zero, which means you cannot divide for it because there is no x-one. So that's a problem, but it's a problem which is not such a big problem. Let's put it this way. Instead of minus one-half, you can choose some other value, let's say one or zero or something like this. Well, in case of one, you immediately hit the... By the way, if you hit one, then this particular calculations should really produce point x-one, which is also equal to one, right? Let's just check it just for fun. So if x-zero is equal to one, x-one is equal to one minus, and f of x-zero, you substitute one and you get zero, right? Divided by doesn't really matter what, so you will get exactly one. So from x-zero equal one, you get x-one is equal to one, which basically indicates that you, by accident, hit the solution. But in any case, this is not such a big problem when the tangential line is parallel to the x-axis and your derivative is equal to zero. You just choose another point. That's fun. Now, another problem, well, what if the function doesn't really have zero at all? I mean, you don't know really if this function intersects the x-axis or not. So you start looking for a solution and you will never find it, right? So if, for instance, the situation is, let me just put some graph as an example. So let's say you have something like this. Let's say it's a parabola, for instance. You start from here. That's your x-zero. You draw a tangential line. You get this. Okay? Now from this, this is your x-one. You go up. You draw a tangential line. And you go here. This is your x-two. From x-two, you go up and do some tangential line. Get x-three. So you will be probably dancing around certain things that you will never actually get to any solution. So your sequence x-zero, x-one, x-two, etc. will never be any closer to zero than certain value, basically, right? And more than that, at certain point, for instance, if you are very close to this and you draw a tangential line, your x-sevenths might be all the way up to the very large numbers. So my point is, if the function doesn't have zero, obviously the Newton's method is non-converging to any solution. Can you identify it? Well, probably you can. If you are calculating, for instance, the difference between consecutive values which you are calculating, and if you see that this is not actually going to zero but at some point goes to a completely different direction, it's increasing instead of decreasing. In the previous example, we saw that it's really decreasing because it goes to a certain limit. If it converges to a limit, then the difference between successive numbers, consecutive numbers is getting less and less, right? But in case there is no limit, it goes nowhere. So this is your case when the Newton's method fails and you can identify it by basically analyzing something like this. And another method, another case, excuse me, when it fails, let me just demonstrate it first graphically. Let's say you have this type of a graph. By the way, there is a very good example. It's f of x is equal to x times e to the power minus x. This is a graph. And if you're looking for this point, if you start somewhere here, this is your x zero, then eventually you will get whatever is necessary. Somehow. You go here, you go here. I mean, it will dance around this zero from left and right and you will get it. But if you start somewhere here, if this is your x zero, look what happens, et cetera, you go to infinity. So in some cases, when the point which you start from is really far away from the zero point, then it might actually lead you to a completely different direction. What's important is this hump. If it's closer to the real solution than this hump, then you will succeed. If it's further from the solution which you're looking for than this hump, you will not succeed. So it's very important to guess approximately where your solution might be and get the value of x zero as close to that supposedly to be found solution, as close as possible. Because only if it's very close, if it's before some kind of hump affects the behavior of the function, only in this particular case you can count on successful iteration and successful recursive process which gives you the solution eventually. Other than that, if you are starting a little bit further than hump like this, you will not succeed. Well, basically that's it. This is all I wanted you to know about the Newton's method of finding zeros of the function. So it's very important to make sure that the function does have zeros and there is no universal rule how to determine this. For instance, at certain moment you have the function positive and certain other value of argument you have the function negative. Then somewhere in between there is the zero function. There is a point where the function is equal to zero. That's one of the reasons, right? So if the function is, let's say, positive here and function is negative here, then somewhere it crosses the x-axis. We are supposed that the function is differentiable, obviously, and therefore it's continuous and that's why it's supposed to intersect the x-axis if it's positive at one value and negative in another value. And finally what's very important is to realize if the function has some kind of a hump like that, now how can this particular hump be, you know, classified, if you wish? Well, basically in this particular case the first derivative is positive and this thing is negative because here we are monotonically... it's a monotonic behavior, right, of the first derivative. And here it's... so now we can actually look at the second derivative, if you wish, because here the second derivative would be positive because the function, because the tangent would be... in this case it's negative, so it would be less negative, if you wish. In this case, in this case the first derivative... the second derivative is negative. But in any case, there are certain approximate consideration which gives you a picture like this and if you anticipate that this will be the case then you get better as close to a potential solution which you don't know. And I understand it's difficult in some cases but that's what you have to do and the process will not converge. Okay, that's it. Thank you very much. I do suggest you to go to the website and read all the notes for this lecture. They're a little bit more precise and more detailed maybe than whatever I was just talking about. And good luck. Thank you.