 Hi, we are discussing errors in polynomial interpolation. In this lecture, we will discuss an interesting phenomena called Runge phenomena. Runge phenomena is also connected to errors in polynomial interpolations. Then we will also see that there are two ways that we can minimize errors in polynomial interpolation. One way to minimize the errors in polynomial interpolation is to construct polynomial interpolations on unequally spaced nodes and another approach is to go for piecewise polynomial interpolations. Let us first discuss the Runge phenomena. Runge phenomena is all about certain oscillations near the boundaries in interpolating polynomials of some functions especially when we go to construct interpolating polynomials on equally spaced nodes. This phenomena was first observed by a German mathematician and physicist named called Runge. In fact, we have already observed Runge phenomena in the example that we have discussed in the last class. If you recall, we have discussed an example where we constructed interpolating polynomial for sine function. There, we observed that when we go on increasing the degree of the polynomial, we observed oscillations appearing near the boundaries of the interval of interest. We attributed this phenomena to arithmetic error in that example that is because in that particular example, mathematical error was well behaving because it was bounded by 1 by n plus 1 factorial. Therefore, as n tends to infinity, the upper bound of the mathematical error was tending to 0 and that makes the absolute value of the mathematical error also tends to 0. On the other hand, we have seen that arithmetic error was growing drastically especially when we construct the polynomial interpolation on an equally spaced nodes. Now, the question is, is such oscillations are always due to arithmetic error? In other words, if we make the calculation of this interpolating polynomial using infinite precision computation, do we always get a good approximation? That is the question. The answer is again no. Runge has given an example where you can observe that oscillations can occur even for polynomials of degree much less. Let us consider an example where f of x is given by 1 by 1 plus 25 x square. This is an example of a Runge function. In this figure, we are constructing and showing two interpolating polynomials of the function f of x. One is the quadratic polynomial which is shown in blue solid line. You can see that quadratic polynomial is approximating the Runge function rather in a poor way. This is rather understandable because we are giving the information about the function only at these three node points. Therefore, it is understandable for why the quadratic polynomial is not approximating the Runge function nicely. Next is let us take nine nodes and construct p 8 of x that is interpolating polynomial of degree 8. Now, we are giving the information about the function at nine different node points, but still you can see that p 8 of x which is shown in the red solid line is not approximating the Runge function that well. In fact, we can see that the polynomial is oscillating between the node points and this oscillation is very nicely seen towards the boundary of the interval which is also called edges. This is what is called the Runge phenomena. Let us go to increase the degree of the polynomial and see how the interpolating polynomial is behaving near the boundary. Let us take n is equal to 18 and thereby we are constructing an 18th degree polynomial interpolating the Runge function and it is shown in this blue solid line. You can see that the polynomial is oscillating widely near the boundary. This is typically what is called the Runge phenomena. In other words, we say that the Runge phenomena is clearly visible in the interpolating polynomial of degree 18 for this function. In fact, we can see that the total error is something near to 30 which is clearly not acceptable as a good approximation. Now, the question is is the Runge phenomena purely due to the amplification of the arithmetic error. This is a quite natural question that one can ask after seeing the last example in our previous lecture. But the fact is even if you construct the interpolating polynomial with infinite precision, we may still observe the Runge phenomena that is because in the Runge function the amplification of the error that is the oscillation near the boundaries is actually happening due to the mathematical error mainly. It may be happening due to arithmetic error also. But you see we are only considering 18th degree polynomial and we are constructing this polynomials on a 64 bit processor. Mostly 64 bits processors have the capacity to handle the rounding errors for such a small n. Therefore, this Runge phenomena may be coming more because of the mathematical error. Let us try to have a close look at the mathematical error. If you recall in one of our last lectures, we have derived an expression for the mathematical error involved in the polynomial interpolation and we have also derived an upper bound for the mathematical error. Let us take this upper bound and let us view this expression in two parts. One is b minus a to the power of n plus 1 divided by n plus 1 factorial. In our example, a is equal to minus 1 and b is equal to 1. Therefore, this is 2 to the power of n plus 1 divided by n plus 1 factorial. You can clearly see that this part of the error is a well behaved expression especially as you go on increasing the value of n. Therefore, if at all something is going wrong, it must be going from this part of the upper bound. Let us try to have a close look at how the n plus 1st order derivative of the function f behaves. Recall the infinite norm of any function is nothing but the maximum of the modulus of the function. In this small window, I have plotted the graph of the 19th order derivative of the Runge function. Here, you can see that Runge function is attaining its maximum somewhere near the midpoint of the interval and the maximum is something like 7 into 10 to the power of 11. So, it is quite a large value. Let us go to see how the 19th derivative of the function f looks like. Again, you can see that the 19th order derivative of the Runge function has its maximum very near to the midpoint and its value is something like 2.5 into 10 to the power of 30. So, these are quite big. In fact, you can precisely compute the upper bound of the 19th order derivative of the Runge function and it is approximately given by 10 to the power of 19. That gives us a severe warning that the mathematical error may be as large as 10 to the power of 19. It only gives a warning. It need not be as big as this because it is just an inequality. Nevertheless, there is a possibility that the mathematical error can be as big as 10 to the power of 19. Therefore, the key takeaway is that if you go on increasing the degree of the interpolating polynomial, at certain stage surely your arithmetic error will start increasing quite rapidly and spoils the accuracy of the interpolating polynomial, especially when you are working on the equally spaced nodes. Even if you are not taking n to be very large, there are certain functions like Runge functions. If their derivatives are growing rapidly somewhere near the midpoint of the interval, then again there is a severe warning that the mathematical error may grow drastically. These are the key takeaways. Such drastic increase in the error generally happens with equally spaced nodes. There are at least two ways that we can improve the approximation of interpolating polynomials. One is to constrict the interpolating polynomial with some appropriately chosen unequally spaced nodes. Another approach is to go for piecewise polynomial interpolations. Let us take up these two approaches one by one. Let us first see how we can constrict some unequally spaced nodes in order to improve the approximation of the interpolating polynomial. Observe that the error drastically increases near the boundaries of the interval, which is commonly called as edges. Therefore, one idea is to put more points near the boundaries of the interval when compared to the number of points that is nodes located away from the boundaries of the interval. One such set of nodes is called the Chebyshev nodes. Chebyshev nodes are given like this. You are given a n for that you have to generate n plus 1 nodes. If you recall equally spaced nodes are generated as x naught is equal to a, where the interval is a comma b and then x 1 is equal to a plus some h, where h is given by b minus a by n. Then x 2 is a plus 2 h, x 3 is equal to a plus 3 h and so on up to x n is equal to a plus n h, which is nothing but b. So, this is kind of formula that we use to generate equally spaced nodes. Now, what we are doing is you are given a n. Now, we are constructing an unequally spaced nodes using this formula. For each i, x i is given like this in order to specify that these nodes are generated with some value of n, we will also including this in our notation. We will use the notation as superscript bracket n. Remember this is not to indicate derivative. Generally, we use the same kind of notation to denote derivative of a function. Here we are just using it to mention that this set of nodes is constructed with the value n. How these are defined? For each i, you have cos 2 i plus 1 pi divided by 2 into the n that we have taken here plus 1. So, this is how the Shebyshev nodes are defined. For instance, if you take n is equal to 4, then you need to generate 5 node points and these are denoted by x naught to the power of bracket 4 and so on and they are given by cos pi by 10 that is first you are taking i is equal to 0 here. Therefore, it is cos pi divided by 10 because n is equal to 4. Similarly, x 1 4 is obtained by taking i is equal to 1 and that is given by cos 3 pi by 10 and so on. Let us see how these nodes are placed in the interval minus 1 to 1. In this figure, we are showing the Shebyshev nodes for n is equal to 4 in the red dots. Similarly, if you can construct the Shebyshev nodes for any given n, let us construct the Shebyshev nodes for n is equal to 18. In this case, we will get 19 node points and they are given like this. You can observe that the nodes are more clustered near the boundary whereas, they are sparsely placed when you come away from the boundary. This is the main idea of the Shebyshev nodes. What the formula for the Shebyshev nodes is doing is, you just take the unit circle and then the points are located equally spaced along this unit circle and then what you are doing is, you are projecting them onto the x axis. This is why you can see that the points which are near the boundary, they are getting more clustered here and similarly here whereas, if you go towards the center of the interval, you can see that these are equally spaced, but when you project them on the x axis, they have unequally spaced position. That is what the idea of Shebyshev nodes. Let us see how Shebyshev nodes are placed when n is equal to 32. You can write a small python code to generate these nodes. You can see again that the nodes are more clustered near the boundary and you can see for n equal to 64, very clearly the points are getting accumulated near the boundaries when compared to the points which are placed away from the boundary. They are little sparsely placed. Let us try to construct the interpolating polynomial now using Shebyshev nodes. Let us take n is equal to 4 and we have 5 corresponding Shebyshev nodes like how we constructed with equally spaced nodes. In a similar way, we can also construct interpolating polynomial either using Lagrange form or Newton's form. Remember in these two forms, we have never put any conditions on how these nodes have to be distributed in an interval. They can be distributed in any form you want. So, in particular if you have equally spaced nodes, they suffer from the Range phenomena. That is why we are looking for an alternate idea. We are going for the Shebyshev nodes. You can see that again n equal to 4 with Shebyshev nodes is showing some kind of Range phenomena near the boundary. Let us go to construct the interpolating polynomial of little higher degree, but now using Shebyshev nodes. We are now constructing 18th degree polynomial using Shebyshev nodes. We are now given 19 node points not equally spaced, but Shebyshev nodes. You can see that the Range phenomena is still observed, but not as bad as what we got with the equally spaced nodes. If you recall, we have shown p18 of x with equally spaced nodes. This is with equally spaced nodes. You can see such a bad oscillation near the boundaries. That is the Range phenomena is very much visible, whereas the same 18th degree polynomial now constructed with Shebyshev nodes has improved the approximation drastically. Let us go ahead and compute 30 second degree polynomial of the Range function now using Shebyshev nodes. You can see the power of clustering more points near the boundary when compared to the points distributed towards the center of the interval. You can see that the Range phenomena at least visually is almost suppressed. Of course, you can see a small error here, but this is due to some bug in the program. It is not the numerical error. Next you can see the polynomial of degree 64 constructed using Shebyshev nodes. You can see that the approximation is pretty good. From here, you can see the power of using unequally spaced nodes. In particular, it is the Shebyshev nodes. The main idea of Shebyshev nodes is to put more points near the boundaries of the interval and to balance that you can have less points away from the boundary. If you would have constructed the polynomial p 64 using equally spaced nodes, it would have gone much more worst. Now let us go to learn another approach to improve the approximation in polynomial interpolation. This is due to piecewise polynomial interpolations. We have seen that when we go for polynomial interpolation that is one polynomial on the entire interval, the approximation is not that satisfactory especially when we work with equally spaced nodes and also when you go to construct or interpolating polynomials with height degree. Now in certain functions, you cannot avoid giving more points especially when you are working with rapidly varying function something like the ringer function. You need to give more points in order to capture all the variations of the function. In that way, if you go for the polynomial interpolation, then it increases the degree of the polynomial and even the arithmetic error will start playing a vital role in amplifying the error. In such cases, what you can do is you can fix the degree of the polynomial, but you can apply the polynomial interpolation rather piecewise. That is the idea of piecewise polynomial interpolation. Let us just take an example and see this idea more clearly. Let us take 3 nodes x naught, x 1 and x 2 with x naught as a and x 2 as b. Therefore, you have x naught which is equal to a, then x 1 and then x 2 equal to b. Now, you can obtain a quadratic polynomial interpolating a given function in this interval. An alternate idea is not to go for one single quadratic polynomial interpolating this data set. Rather, you go for a linear interpolating polynomial in this interval. Let us call it as p 1 comma 1 of x and then you go for another linear polynomial interpolation in this interval. Let us call it as p 1 2 of x. So, the idea is not to go for quadratic polynomial rather go for two different linear interpolating polynomials. That is the idea of piecewise interpolation. Let us make this discussion more precise. What you do is suppose you are given a data set with nodes as x naught, x 1, x 2. Then consider the first sub interval x naught to x 1. You can write the linear interpolating polynomial either in the Newton's form or the Lagrange form. Here I am showing the Lagrange form of the linear interpolating polynomial. This is only for the interval x naught to x 1. Then you take the second part of the interval that is the second sub interval of the partition. Construct another linear interpolating polynomial. I have used a Lagrange form again here and then define the interpolating function. Let us denote it by s of x as p 1 plus 1 of x when x belongs to x naught to x 1 and p 1 2 of x when x belongs to x 1 to x 2. That is suppose your function looks like this. Now you want to construct an approximation with three node points. So, what you are doing is you are taking this interval and making the linear interpolation here and then take the second part of this interval and make another linear interpolation. This is what is called s of x. This is what is called the piecewise linear interpolating function. Once you understand this idea, you can also generalize it to piecewise quadratic interpolation, piecewise cubic interpolation and so on. Let us take an example. Let us consider again the same Ronge function. We have seen that one single polynomial interpolation with equally spaced nodes is not going to be a good idea to approximate functions like Ronge functions. Let us take a data set consisting of three nodes points and the corresponding function values on them. They can be equally spaced nodes and our interest is to construct a linear interpolating polynomial. Let me just put the Newton's form of interpolating polynomial here. There is no reason you can use either Lagrange or Newton. Since I have already shown the expression for Lagrange in the previous slide, now I have shown the Newton's interpolating polynomial formula here. You can use anything you want. If you have given some values to x naught, x 1 and x 2 and correspondingly f of x naught, f of x 1 and x of x 2 can be calculated from this function and thereby you can get an explicit expression for the piecewise linear interpolation. Since you have two piece of linear interpolations, we will use the notation s 2 of x here. Let me take the nodes as equally spaced nodes. So, x naught is minus 1, x 1 is 0 and x 2 is 1 and find the corresponding values of the function f at these nodes and you can now plug in these values into this expression. I leave it to you to do this calculation. You can see that the piecewise linear polynomial interpolating the function f of x at these node points is given like this. Let us see how this function looks like graphically. Here the black solid line is the Runge function and the blue solid line is the graph of the piecewise linear polynomial interpolating the Runge function f of x. Of course, the approximation is not that good. It is understandable again. Let us just take a point on the graph of the Runge function. Its corresponding value from s 2 is given like this. Let us go on increasing the number of nodes in the interval. Now, remember when we increase the number of nodes in the interval, we are not going to increase the degree of the polynomial. Rather, we will be increasing the number of pieces in the piecewise polynomial interpolation. Let us observe how these points are located on the graph of the corresponding functions. Let us take four pieces. It means we have five node points located here and the corresponding piecewise linear interpolation is shown in the blue color s 4 of x. You can see when compared to s 2, s 4 is approximating the values pretty well at least at the point where we have chosen to observe the approximation. You can see this is the total error involved in s 2. The total error involved in s 4 is this. Let us now go to increase the number of nodes in the interval. Now, I am taking s 8. It means I have taken nine nodes and therefore, there are eight pieces of linear interpolating polynomials and the total error is now decreased drastically and then I have gone to 16. You can see that your approximation is tending to the exact value. See this is what is meant by the approximation s 2 is tending to the exact value as you go on increasing the number of node points. If you recall, if you would have constructed p 16 of x, you can go back and see it had a very poor approximation. Shebyshev gave a better approximation and suppressed the Range phenomena significantly. Similarly, piecewise linear polynomial interpolation is also suppressing the Range phenomena significantly. However, it is not able to approximate the function in certain places especially at those places where the function has higher curvature. This is again understandable because we are doing piecewise linear interpolation. If you would have gone for piecewise quadratic interpolation, the approximation would have been little better than this. But let us go on with increasing the number of nodes in our data set. Now, we are having 33 node points. Thereby, we have 32 linear interpolation polynomials and we constructed s 32 with that. Well, you can see that s 32 is approximating the function pretty well and better in this region also when compared to s 16. Then I have just jumped to s 100 just to show that as you go on increasing n, equivalently you will be calculating p 100 of x, it would have gone significantly bad in its approximation. If you would have used the polynomial interpolation, of course here I am doing with equally space nodes. That also you should remember I am not constructing it with Shebyshev nodes. I am constructing this piecewise linear polynomial with equally space nodes. You can see that Range phenomena is not at all occurring and the s n of x is pretty converging to f of x at every point. That is what is interesting here. Let us try to state the convergence theorem for piecewise linear polynomial interpolation. For this, you need to assume that f is a c 2 function on the interval a comma b and let s n denotes the piecewise linear interpolating function f with nodes as x naught, x 1 up to x n. They can be equally space nodes. No problem as far as you are working with piecewise polynomial interpolation and the interpolation degree is not too high also. You should also take care of that. Here we are stating this theorem only for piecewise linear polynomial interpolation. In fact, the theorem is specifically choosing equally spaced nodes. Even then, you can say that the sequence of piecewise linear polynomial s n of x converges to f of x for all x in the interval a b as n tends to infinity. This is quite encouraging because one thing is you are working with equally spaced nodes and another one is even you take larger and larger values of n, you still can have convergence. That is what the theorem says. Let us quickly try to prove this theorem. The idea of proving this theorem is not something new. You have already seen it in one of the previous lectures. Take any n and take any point x in the interval a to b chosen arbitrarily. Now, once you choose this n, your partition is fixed. You have x naught x 1 up to x n. These are fixed equally spaced. Once you take a x, it will surely sit in one of the subinterms. Let us say x is sitting in x j minus 1 to x j for some j. Then, for that x, s n of x will be p 1 j of x. That is how s n of x is defined. Therefore, if you take the mathematical error involved in s n of x when compared to the original function, then it can be written as mod f of x minus p 1 j of x because your x belongs to this interval and in this interval s n of x is a linear polynomial. That is why this is equal to this. Once you have this, you know how this looks like. If you recall, we have derived an expression for the mathematical error involved in a interpolating polynomial of degree any n. Here, in particular n is equal to 1. Remember, it is in one piece the degree of the polynomial is 1. You should not get confused with the number of nodes attached to the degree of the polynomial. That is only in the case of polynomial interpolation. Now, we are working with piecewise polynomial interpolation. Whatever may be the value of n, your degree is not changed here. Therefore, you can use the mathematical error in the linear polynomial interpolation. If you recall, the expression is given like this where xi is an unknown real number lying between x j minus 1 and x j. Let us try to find an upper bound that is an estimate for this. Since we have assumed that f is a c 2 function, you can bound it by some positive number m. Therefore, you can freeze this xi by replacing f double dash of xi x by m. So, you can write f of x minus Sn of x modulus is less than or equal to. I am just replacing this here and therefore, it is m by 2 and then I am having this term as it is here. Now, we will not consider this as an estimate because it involves the variable x here. In order to eliminate this variable x, we will go to find the maximum of this function. Note that this is a quadratic polynomial with a modulus here and you can clearly see that it attains its maximum at the midpoint of the interval x j minus 1 x j and its value is given like this. Remember, we have a working with equally spaced nodes and therefore, this is equal to h square that is what I am writing here. Now, you can replace this here with again a less than or equal to sign and therefore, we finally got the estimate of your mathematical error involved in the piece wise linear polynomial interpolation and that estimate is given by m into h square by 8. Remember, this holds for any n and for any x in the interval a b. Therefore, in fact, you can say that this is nothing but f minus Sn infinity is less than or equal to m h square by 8. Now, you can see as n tends to infinity, what happens? Well, as n tends to infinity since h is equal to b minus a by n, you can see that h tends to 0 as n tends to infinity. That means, the right hand side goes to 0 as n tends to infinity. That means, the left hand side will also go to 0 rather uniformly and that proves that Sn converges to f and this completes the proof of the theorem. In fact, you can extend this idea of constructing piece wise linear interpolating polynomial to piece wise quadratic interpolating polynomial, cubic interpolating polynomial and so on. In fact, you can also extend the idea of proving convergence theorem for these cases also. I will just remark that if you are going for a quadratic polynomial interpolation, then you at least need 3 points in order to construct a quadratic polynomial. Why? Because to construct a quadratic interpolating polynomial, you need 3 points. Therefore, when you have a data set and you want to construct a piece wise quadratic polynomial function, you have to group the nodes in this way. That is, you need 3 points to construct one piece of the quadratic interpolating polynomial. Therefore, this will be P 2 1 of x and for the next piece, you have to take these nodes and construct P 2 2 of x. Remember, this will be common both for this as well as for this. This is also the case with linear polynomial. You can go and observe that and you will have the continuity of the piece wise polynomial because of this one. Otherwise, it will have a gap between 2 nodes. We should not have this. So, you should carefully choose the nodes to construct the piece wise interpolating polynomials. Similarly, for cubic polynomial, you have to group the nodes with 4 nodes in one piece and the boundary nodes should overlap for both the pieces on the either side of that point. So, this is how you can construct piece wise interpolating polynomials of any degree. With this, we will end this lecture and thank you for your attention.