 Hi, we are discussing iterative methods for capturing isolated routes for a given non-linear equations. In this today, we will discuss fixed point iteration methods. Fixed point iteration methods is a class of iterative methods and it can be seen as a generalization of the certain iterative methods. In fact, you can see that Newton-Raphson method that we have discussed in the last class can be obtained as a particular case of the fixed point iteration method, but second method does not come under fixed point iteration methods. However, Jacobi and Gauss-Seidel methods can be viewed as fixed point iteration methods. In this lecture, let us try to understand the fixed point iteration methods. Mostly fixed point iteration methods are used as a good tool for theoretical studies, but not much used in the computations. However, some of the particular cases like Newton-Raphson method or Gauss-Seidel method and Jacobi method for linear systems, they are used in the practical situation. Let us try to first get the idea behind the fixed point iteration methods. The idea of fixed point iteration method is to first rewrite the given non-linear equation f of x equal to 0 in the form x equal to g of x. That is we have to find a suitable function g such that f of x equal to 0 is equal into x equal to g of x. That is a root of the non-linear equation f of x equal to 0 should be a fixed point of the function g. Now, the question is what is mean by a fixed point? Let us first see that suppose the graph of the function g is given like this. This is y equal to g of x in the x y coordinate. Now, what you do is you draw the straight line y is equal to x. The point of intersection of this line with the graph of the function g is what is called as a fixed point of the function g. Now, you have to find a function g such that the root of the equation f of x equal to 0. What is that? That is nothing but the point of intersection of the graph of the function y equal to f of x with the x axis. This is r. Now, you have to find a suitable g such that the root of the equation f of x equal to 0 is same as the fixed point of the function g. That is the first step of a fixed point iteration method. Well, how will you find such a g? That is not very difficult. In fact, you can find many such g for a given equation f of x equal to 0. Let us see an example. Let us take the equation x square minus x minus 2 equal to 0. Here, f of x is x square minus x minus 2. Now, this equation can be written as x is equal to x square minus 2. That is one way of rewriting this equation. In this case, our g is nothing but x square minus 2. You can see that a root of this equation is the fixed point of the function g. This is not the only way that you can get g. You can also get g by keeping x square on the left hand side and push the other terms on the right hand side and then take the square root. Well, you can take the plus or even you can take the minus also. In this case, g is given like this and you can in fact do it in many other ways. For instance, you can also write the given equation in this way where g is given like this and so on. Once you have chosen your g suitably, then you have to define your iterative sequence as x n plus 1 equal to g of x n. In this case, the function g is called the iterative function. If you recall, we have written the Jacobi method as x is equal to b x plus c. So, this is also coming in this form where the solution of this system is equivalent to the solution of this system also. That is solution x of the system A x equal to b is also the solution of the system x is equal to b x plus c. The same thing here. As we have seen in the last example, the iterative function for a given equation f of x equal to 0 is not unique. You can find many iterative functions. Let us give the algorithm now. As I input, we have to also give the iterative function g. Well, we need not give the function f. Now, we have to give the iterative function g and you have to give a initial guess x naught. Once you give, you will generate the iterative sequence using this formula. So, this is a very simple algorithm for setting up fixed point iteration method. Well, in the practical situations, we have certain iterative functions which are commonly used. One is that g of x is equal to x minus f of x divided by f dash of x. You should recall what is the method that you will get if you take your iterative function like this. Well, that is nothing but the Newton-Raphson method. Similarly, you can take the iterative function as I have shown here that is g of x is equal to b x plus c. In the case of linear system, we have studied this as the Jacobi method. Similarly, we also have Gauss-Seidel method which comes under this fixed point iteration method. Now, you can see that fixed point iteration method is a framework. It is not one single method like Jacobi method or Gauss-Seidel method or Newton-Raphson method for non-linear equations. It is not like that. It is a framework where you choose your own iterative function. That is how the fixed point iteration method is more like a general framework. There are certain particular choice of iteration function that leads to very efficient and fast methods. But in general, you can show that the fixed point iteration method will converge with at least order 1. That is, it has at least a linear order of convergence. But that does not mean it will only converge with order 1. It will converge with order at least 1. In particular, we have a method which is Newton-Raphson method that we have shown that it converges with order 2. But it still sits in fixed point iteration method. That is because we will show that fixed point iteration method is of order 1. That means at least 1. That is very important. I am just emphasizing again and again. You should not get confused. Why Newton-Raphson method which is a fixed point iteration method is order 2? That is the common question that people ask. That is why I am emphasizing again and again. Now, you can understand that in the fixed point iteration method, the crucial step is to choose the iteration function. Will we have freedom to choose our own way of defining the iteration? That gives us also the responsibility of choosing a right iterative function that is a good iterative function. But now, what is mean by a good iterative function? So, that is the question. Let us try to understand it. A good iterative function should have certain properties. What are they? The first one is that for a given x naught, the approximations x 1, x 2 and so on all should exist. That is the first thing. If they do not exist, then there is no meaning for saying that it is a method to calculate the root. Therefore, the sequence should exist. That is the first property that we will look for from our iterative function. The next one is the convergence. Once the sequence is obtained, then the next worry for us is whether that sequence is going to converge or not. So, we should choose our iterative function in such a way that the sequence should exist and then it should also converge. Now, once it converges, the next question is, is the limit of the sequence a fixed point of the iterative function g? That is our next question. Well, we should choose our iterative function in such a way that the sequence should exist and the sequence should converge and the limit should be a fixed point of the function g. Because remember, we have constructed g in such a way that the fixed point of g is precisely the root of our equation f of x equal to 0. Therefore, the limit of the sequence should be a fixed point of the function g. So, these are the three properties that we wish to have from our iterative function. Now, the next question is, how to choose such a iterative function for a given non-linear equation f of x equal to 0? Let us see one by one. Let us take the first property that is, we want our g in such a way that the sequence x n generated by that fixed point iteration method should be well defined. Let us illustrate this situation with a bad example. Let us take the equation x square minus x equal to 0. We can take our iterative function in many ways. One bad choice is to take our iterative function as g of x is equal to minus root x. Why? Because this equation can be written as x square equal to x. From there, you can get x equal to plus or minus root x. If I take g of x equal to minus root x, that will be obviously a bad choice. Because if you choose x naught as any positive number say 1, then g of x naught is equal to minus 1 and that is your x 1. And now, x 2 is g of x 1 and that is root minus root minus 1 which is not a real number. So, that is the problem. Therefore, even if you choose your initial guess x naught as positive, we will get into the trouble of generating a real sequence. Therefore, this choice is obviously a bad choice. Now, how to obtain such a iterative function? Well, what you have to do is that our iterative function g should be in such a way that g of x should also be in the domain in which g is defined. That is, you take g in such a way that it is defined say on an interval a, b. Then what happens? You take any x in a, b and find g of x. Then g of x should also be defined. Because for the next iteration, you are going to apply g on this one line. That is how the sequence goes. Therefore, x belongs to g of x should imply that g of x also belongs to a, b. Then only when you go to apply one more time g on this, that will be well defined. It means g should be a map from an interval a, b to itself a, b. Such a map is called the self map. We make our first assumption on the iterative function as g is a self map. That is the first assumption we have to make. That is, g is defined from some interval a, b to itself. That is the first assumption on our good iterative function. That precisely means this. Now, let us go to take the point number 3. Let us see what is the point number 3. We have achieved this by putting g is a self map. Next, we have to worry about the convergence. But I will just go to point 3 where I will assume that the sequence is converging. Then I will see what is the condition that I have to impose on the iterative function g. So, that the limit of the converging sequence is a fixed point of g. Why I am jumping to the third point? Because that is more easy for me to address. I will address this easy point and then I will come to the second point. So, let us first take the third point. What is the third point? The third point is that the sequence is converging. Then the limit is a fixed point. To have this, what is the assumption that I have to give on g? That is the question. Let us see. It is not very difficult to see that we have to impose that the function g is a continuous function. Why it is so? Well, suppose the sequence x n is converging say to x star. Then what you know is x star is equal to limit n tends to infinity x n. Therefore, you can write x n as g of x n minus 1. Remember that is how you are defining your sequence. x n is equal to g of x n minus 1. That is what I am writing here. Now, if g is a continuous function, then you can push this limit inside g. That is what is a intuitive definition of continuity. With that, we can see that limit n tends to infinity g of x n minus 1 is equal to g of limit n tends to infinity x n minus 1. Now, that is precisely g of x star. So, what you have seen is x star is equal to g of x star. That is precisely the definition of x star being a fixed point of the function g. Therefore, if you choose your g such that it is a continuous function defined on a interval a b to itself, then we have achieved two properties. That is your fixed point iteration sequence exists and if the sequence converges, then the limit will be a fixed point of your function g. With this, let us now move on to the next property that when will the sequence x n converge. This is little involved. We have to get a good picture about how this fixed point iteration works. Geometrically, let us see that in this I will give you two examples. Geometrically, where in one example the fixed point iteration converges and in another example the fixed point iteration diverges. Then you try to see what is the condition that we have to impose on g in order to get a convergent sequence. Let us take our first geometrical example where my function g is such that its graph is shown in this blue solid line. Now, what we are interested in? We are interested in capturing the fixed point of the function g. Let us denote the fixed point by xi and what is that? That is nothing but the point of intersection of the line y is equal to x with the graph of the function g. That is the fixed point of the function g. Now, let us see if we take this g and form our sequence x n using the fixed point iteration approach. Then how this sequence goes? Let us start with an x naught. Say, let me start the x naught at this point. From here, you can see that you have to first take the value of g at x naught and then how will you define x 1? If you recall x 1 is equal to x 2. 1 is equal to g of x naught. So, it means what? I am taking this point and seeing where this point intersects this line. This is nothing but x equal to y. So, this is y is equal to g of x naught. That is this point and this point is nothing but y is equal to x. That is what is this point. Therefore, this point is nothing but y is equal to g of x naught. That is what precisely I want to call as x 1. So, therefore, this is my x 1. If I start my x naught and define my x 1 as g of x naught, then my x 1 is precisely this point. I hope you understood what this iterative definition geometrically means. Once you get x 1, now you have to go to find x 2. How will you find x 2? Well, for that you have to take this point. You have to take this point which is the value of g at the point x 1 and that is given like this. Now, x 2 is defined as g of x 1. So, x 2 is now obtained by going and catching this point. So, from here I went up to here and from here I went up to here. Now, together I get x 2. What I have to do? I have to go from here to here and capture this point and that is my x 2. So, you see how our path is going. So, this is what is called the fixed point iteration path. It started from here, it went here and then it from here it went to this point and from there it went to this point. This is a geometrical interpretation. Whereas, analytically how you obtain this values x 1 is equal to g of x naught and once you get x 1, x 2 is obtained as g of x 1 and so on. So, once you get this, this is your x 2. Now, how will you get x 3? Well, you will come across this, catch this point catch this point and you should find corresponding x 3 for it. So, you just see where it intersects and that point is your x 3 here. Now, once you have x 3, now you have to go to find x 4. For that you will go like this and find this point that is your g x 3. So, that is how you get x 3 and from there you will go and trace where this line intersects x axis and you will see the corresponding x coordinate that is your x 4. Once you get this, similarly you will go here and then go here to get x 5. Once you get x 5, you will go here and here and then you will get x 6. You will see observe how the convergence is happening in this geometrical. So, it goes like this, like this, like this and how it is getting attracted towards the fixed point. See, there is a nice geometrical interpretation for this. You can see that it goes like this and it converges nicely to the fixed point xi. So, the fixed point xi is attracting this sequence towards it, that is what we are seeing. So, from here we go here and then here and then it goes on like this. So, this is one example where the sequence is getting attracted towards the fixed point. In other words, the fixed point iteration sequences converging to the fixed point. Now, let us take another example which is a bad example. Here I am taking the graph of the function g like this, which is shown in the blue solid line and what is the fixed point of this function? Well, that is the point of intersection of the graph of the function g with the line y equal to x and that is given like this. Let us denote it by xi as we did in this example. Now, let us see how the iterative sequence goes. If we take this g as the iteration function, I hope you understood the geometrical idea behind this sequence. Analytical is very simple, but geometry is something which you have to once think and see how I have constructed this fixed point path. This is called the path and you just have to put the arrows here. So, the arrows indicates how this fixed point iteration moves and goes towards the fixed point. So, it goes like this. Now, let us start with some x naught. Now, purposely I have taken it very close to the fixed point. Unlike in the previous example where I took the x naught pretty far from the fixed point, but then it was the nice story that it was getting attracted. But now I am purposely taking x naught close to xi. Now, it will go away from the fixed point. Let us see how the sequence goes. To build the sequence, we need to find x 1 and x 1 is given by g of x naught. g of x naught is this point and you have to see the point of intersection of this line with the, that is this horizontal line with the line y equal to x and that is taken as x 1. Once you have this, then you compute g of x 1 and that is given like this. From there, you approach to this point and then once you get this point, you know what is x 2 and that is given like this. Now, you see unlike in the previous case where we were going towards the fixed point, you can now see that our path is set in such a way that we are going away from the fixed point. So, we have just traced this path outside the inner box. That is what I meant by saying we are going away. Now, you see you get x 2. Once you get x 2, you can find g of x 2 and from g of x 2, you can get x 3 defined as precisely the value of g at x 2, that is precisely this point. x 3 is precisely g of x 2. This is how we define the fixed point iteration. Therefore, it comes to be this point. Once you get x 3, again you will go to find the value of g at the point x 3 and that is given by this and once you have this, you will trace back x 4. For that, you will first see where this horizontal line hits the line y equal to x and from there, you will get this point as x 4 and your path goes similarly. You can see that g of x 2 is equal to x that you went like this and then like this and then you are going away. As you go on computing the iteration, you are going away from the fixed point in this case. You see it goes like this. Like that it will keep on going away from the fixed point. Now, just observe what is the difference between this function and this function? What is the condition that you have to impose in order to get your fixed point iteration sequence as a convergent sequence? Well, it is all depends on what is the slope of the function g when you are very near to the fixed point. That is in this small neighborhood of the fixed point, you want your g to have smaller slope. Here, you see the slope here is little smaller when compared to here. g is growing more rapidly here when compared to how g grows in this example. Now, what is the precise condition that we have to put? Well, we understood that we have to put a condition on g dash, but what is that condition? You see this function that is this line has slope 1. Now, if the slope of the function g dash is something less than 1 that is what is happening in this case, then the sequence will converge. Otherwise, you have no guarantee, you may face a situation like this. So, that is the condition that you have to impose on g in order to get a convergent sequence out of your fixed point iteration method. So, let us finally, put our assumption 3 on the iterative function that the iterative function g should be differentiable on the interval where you have chosen g as a self map. Remember that is the first assumption. And now, the g should also be such that there exists a constant k which is strictly less than 1 this is very important such that mod g dash of x is less than or equal to k and that is less than 1. It means the slope of g should be less than 1, it means g should not grow more than 1. So, that is the idea because the other line that is this line is growing with slope 1 and therefore, this should not grow with slope more than 1 that is the idea and that is what is imposed here. And such a map is called contraction map. If a function g satisfies this condition with k less than 1 it means g is a contraction map. So, to summarize in order to get a good iterative function we need to choose the g. Remember you are given a equation f of x equal to 0. Now our job is to choose a suitable g such that root of this equation is equivalent to the fixed point of the function g that is the basic idea of the fixed point iteration method. Now there are many ways that we can choose such a g. We have seen what all the assumptions that we have to impose in order to get a good g. What is mean by good g? Good g should have that those three properties for that we need the following assumptions. One is g should be a self map, second is g should be a continuous function and finally, more important and more involved assumption is that g should be a contraction map. It means that g dash should exist and it should be a continuous function and that more g dash of x should be less than or equal to some constant and that constant should be less than 1. It means g dash should be bounded away from 1 that is more important. It should not grow either equal to 1 or more than 1 it should be something less than 1 it is growth that is what is called the contraction map. So, with all these assumptions we can show that the corresponding iterative sequence is a convergent sequence and it will converge with linear order of convergent. It means it will converge at least linearly that is what we can show which we will show in the next lecture. Thank you for your attention.