 Hi, we are deriving some finite difference formulas for approximating derivative of a given function. In the last class, we have derived three primitive finite difference formulas for approximating the first derivative of a given function. Well, we started with the basic idea of getting this primitive formulas directly from the definition of the first derivative that we learnt in the calculus course. Then, we also obtained these formulas through the interpolating polynomial of degree 1 for the function f and then we also extended this idea of using the interpolating polynomials to get finite difference formulas for derivatives of higher order that is you can also get the finite difference formula for f dash f triple dash and so on and also you can get variety of finite difference formulas by choosing the degree of the interpolating polynomials and also the positions of the nodes involved in the interpolating polynomials. In today's class, we will have another way of deriving finite difference formulas. This is using the method of undetermined coefficients. This method is not something new to us. We have already seen this idea when we were deriving the quadrature formula for a given function. The idea is almost the similar. Let us see what is this idea. The idea behind the method of undetermined coefficients to derive certain finite difference formulas for a given function is the following. Suppose you want to derive a finite difference formula for the kth derivative of a function f at a point x involving the nodes x naught x 1 up to x n then what you do is well basically you look for the finite difference formula in this form that is w naught f of x naught plus w 1 f of x 1 and so on up to w n f of x n. Here you already know what is x naught x 1 up to x n because these are given to us and we have to only find w naught w 1 and so on till w n. Now how will you find? Well you can find this unknown quantities by imposing the condition that this finite difference formula is exact if the function f happens to be a polynomial of degree less than or equal to n. What is this n? Well that is the n that is chosen here that is the number of nodes. If you are given n plus 1 nodes then you have to find n plus 1 unknowns. This n plus 1 unknowns therefore you have to impose the condition that this finite difference formula should be exact for polynomials of degree less than or equal to n. Why? Because the space of all polynomials of degree less than or equal to n will have n plus 1 basis elements. You take the monomial basis and then you can form a linear system with solution as the vector w whose coordinates are w naught w 1 up to w n. This is something which we have already done in the quadrature formula. The same idea goes through here also. Let us see how to go ahead with it. It is well understood through an example. Therefore we will consider our nodes as x naught equal to x minus h, x 1 equal to x and x 2 as x plus h. That is we are taking n is equal to 2. If you recall this is what we call as the central difference nodes and therefore the resulting finite difference formula will be called as the central difference formula. We will try to derive the formula for f double dash of x. Well what you have to do is first write the general form of the finite difference formula by taking the central difference nodes and in order to get w naught w 1 and w 2 you have to impose the condition that this central difference formula is exact for polynomials of degree less than or equal to 2, which is equivalent to imposing the condition that this formula is exact for polynomials 1, x and x square. Why we are restricting to only this particular polynomials? Because they form a basis for the space of all polynomials of degree less than or equal to 2. That is why we are restricting ourselves only to these particular polynomials which are polynomials of degree less than equal to 2. Now by considering each element of this basis we will get one linear equation in w naught w 1 and w 2. Let us do this. Let us first consider the polynomial of degree 0. That is let us consider the case when f is equal to 1 for all x. In this case as per our condition this formula should give you the exact value for the f double dash of x. What is f double dash of x? If f happens to be the constant function well that is 0. Therefore, you have f double dash of x equal to 0 and what is the expression coming from the finite difference formula? Well you have to take f is equal to 1 right here and that leads to w naught plus w 1 plus w 2. Therefore, your first linear equation is given by this. Let us go for the second case that is let us take f of x equal to x. In this case again f double dash of x equal to 0 and the right hand side will be given by w naught into x minus h because f of x equal to x. Therefore, you will have w naught into x minus h plus w 1 into x plus w 2 into x plus h equal to 0 and that is your second equation. Now what you do is you can use the equation we have obtained in case one. What was that equation? It was precisely w naught plus w 1 plus w 2 equal to 0. Therefore, these terms will disappear right and you will be left out with w 2 minus w naught into h equal to 0 and we always assume that h is strictly greater than 0. Therefore, you will have w 2 minus w naught equal to 0 and that is the second equation. Let us take the third case where f of x equal to x square for all x in this case f double dash is equal to 2 right and the right hand side expression will reduce to this. Obviously, again you can use the expressions that is the equations obtained in case 1 and case 2 and then you can reduce this equation to this equation. So, we got three equations by imposing three conditions and we have three unknowns w naught, w 1 and w 2 and they are appearing linearly in this system. Therefore, it is a linear system of equations. You can solve this system to get the values for w naught, w 1 and w 2 and those are given by these expressions. Now what we have to do? Recall that we have assumed that our central difference formula will look like this. In this, these terms are known to us provided of course, we know f, x and h. Then these three terms are known to us and now we also know w naught, w 1 and w 2. You just substitute these values into this expression. You can see that the central difference formula for approximating the second derivative of a function f at a point x is given by this formula right. This is how we derive the central difference formula. If you derive the central difference formula using the interpolating polynomial, you can see that that will also lead to the same expression. What you have to do is you have to take x naught, x 1 and x 2 as given here. This is the central different nodes and then you write the Newton's form of interpolating polynomial using these nodes that will be a quadratic polynomial. Differentiate it twice and substitute x naught, x 1 and x 2 into it. Then also you will get the finite difference formula exactly the same as what you got using the method of undetermined coefficients. Once you understand this idea, you can also derive the finite difference formula for the second derivative using forward nodes, backward nodes. Similarly, you can do for third order derivative and so on. Just like how you can do with interpolating polynomials, you can also do with method of undetermined coefficients. You can obtain the finite difference formula of any order with any given nodes. Let us try to derive the mathematical error involved in the central difference formula that we have derived just now. For that again you have to use the Taylor expansion. Remember that the central difference formula involves f of x plus h. Therefore, you have to use Taylor expansion for this term and also it involves f of x minus h. Therefore, you have to use Taylor's formula for this term also. That is why we are first writing the Taylor's formula for f of x plus h as well as f of x minus h. Remember the central difference formula is given like this. Therefore, you put the Taylor formula for f of x plus h here. Similarly, the Taylor's formula for f of x minus h here and try to simplify that expression. If you substitute these two expansions, what you get is this expression for the central difference formula. You can see how you get it. You simply have to substitute this into this and then just group the similar terms. Then you will get this expression here. Remember I am not writing the Taylor's formula with the reminder. Why? Because when you substitute this into this expression, certain terms will get cancelled. Therefore, at the beginning at this stage you do not know what will remain as the reminder term that will come as the mathematical error finally in our expression. That is not very clear at present. Therefore, what I suggest you is you write the Taylor's series, substitute it into the expression. Whatever may be the central difference formula you do, what you first do is you write the Taylor's series and then substitute it into the finite difference formula and then group all the terms like this and see which are all the terms which are getting cancelled. In this case, if you see you got w naught, w 1 and w 2 in such a way that w naught plus w 1 plus w 2 equal to 0. This is how you got the linear system for w. If you recall we have just now seen that w naught, w 1 and w 2 satisfies a linear system in that the first equation is w naught plus w 1 plus w 2 equal to 0. In fact, w naught minus w 2 is also equal to 0 and this is equal to 2 by h square. If you recall that is what we obtained here. So, I am just using this in this expression and similarly this term is also going to 0 and what remains here is this term. So, this term onwards in this series will remain of course, certain terms may be 0, but the leading term remaining here is this term. Therefore, that gives us a clue that this term should be taken as the reminder and that will decide what is the order of accuracy of the central difference formula. Let us see that your linear system was this. Therefore, by using this you can see that the fourth order term will survive in this expression because this goes off, this goes off and this goes off. This is going to contribute for your f double dash. You have d 2 h f double dash has to come to the other side in order to make it mathematical error and that will be equal to now this term. So, this gives us a clear idea of what you have to take as the reminder term. This is very important while deriving the expression for the mathematical error. You do not decide something up purely because you may be writing the reminder term for this, but that is not correct because the term is very nicely getting off from your formula and what actually dominates your error is this term. Therefore, this has to be taken as the reminder term. You have to keep this in mind very carefully and write finally that the left hand side that is the central difference formula is equal to this term becomes 1 now. You have therefore f double dash of x all other terms vanished and you are left out with this term plus something, but all that can be clubbed and made as the reminder term here which involves unknowns xi 1 and xi 2 where xi 1 and xi 2 are some unknown numbers lying between the nodes x minus h and x plus h. Now, this is what finally we derived as the expression for the mathematical error. You see that this is the mathematical error and that is given by this. Again, you can use the intermediate value theorem. Recall we have seen such a problem in one of our tutorial session as I told in the last class. If you go to the lecture number 7, we discussed some tutorial problems in that lecture. In that you take the second problem you will get an idea of how to use the intermediate value theorem in order to reduce this expression to this expression. I leave it to you to see that by using that you can in fact write this expression as h square by 12 into the fourth derivative of f for some unknown xi lying between x minus h and x plus h. Finally, this is the mathematical error for the central difference formula for f double dash of x. Again, from here you can see what is the order of accuracy of the central difference formula for the second derivative of a function. The order of accuracy is 2 that is it is the second order formula. This is all about how to derive a finite difference formula and how to obtain the mathematical error for that formula and also how to see what is the order of accuracy of that finite difference formula in approximating a derivative of some order of a function. Now, let us see how arithmetic error can be analyzed in these formulas. In fact, the idea that I am going to introduce here for analyzing the arithmetic error involved in finite difference formulas can be used also to analyze the arithmetic error involved in quadrature formula. Let us see how to analyze arithmetic errors in finite difference formulas. As you know, finite difference formulas are useful in many practical applications. In particular, they are used in solving differential equations. Similarly, quadrature formulas are also used in practical applications. They are very nice. However, they suffer very seriously from the arithmetic errors just like how the polynomial interpolation suffer. Similarly, the quadrature formulas and finite difference formulas will also suffer significantly due to arithmetic errors. These are quite expected because these formulas are derived basically from the polynomial interpolation. Therefore, they have to also suffer. So, to analyze the arithmetic error involved in finite difference and quadrature formulas, first what you do is you write the exact value involved in the evaluation of the function at any point say x i is something like f i. This is the approximate value. Just imagine that your computer evaluated the value of the function f at the point x i and it gave this value to you. It obviously will involve certain rounding errors. With that, it is supposed to give you this value, but it gave you this value. What is the error involved in it? We will denote that error by epsilon i. So, since we are going to analyze the arithmetic error involved in the central difference formula with three central different nodes, I am just taking i to be 0, 1 and 2. That is, we are going to have x naught, x 1 and x 2 as the nodes. If you recall, if you take x naught equal to x minus h, x 1 equal to x and x 2 equal to x plus h, you will get x naught the central difference formula in this form for approximating the second derivative of the function f at a point x. In this, you are now involving an arithmetic error also. Now let us see how this formula will look like. If you use the approximate values instead of exact values of the function evaluated at the nodes, then the central difference formula will look like this. I am using the approximate values of the function at the nodes. Let us denote the corresponding value by d dash h of 2 f of x 1. Now our interest is to see what is the total error involved in it? What is the total error? Well, this is the exact value and this is the value given by your computer. Previously the mathematical error is the exact value minus the approximate value that does not involve any rounding error. Now it is the approximate value that involves the approximation of the operator that is the second order derivative operator as well as the approximation involving the rounding errors. So, that is the total error and that is given by f double dash minus d h 2 f of x and this is the mathematical error plus you can see that the arithmetic error now becomes like this. How will you get this? Simply what you have to do? You have to subtract d 2 h f of x 1 and add d 2 h f of x 1. So, that is what I am doing f double dash minus the exact operator is given like this. Then the exact operator minus the operator with rounding error will give you this expression because the exact value minus the approximate value is taken as epsilon. That is why you will get I will leave it to you to just derive this and see. So, this is nothing but d 2 h f of x 1 minus d 2 h f of x 1 dash that is what comes like this you can easily derive that. So, this step has to be done not only for the central difference if I ask you to derive the total error of any formula any quadrature formula or any finite difference formula what you have to do is the total error then you add and subtract the formula without any arithmetic error and then you will get mathematical error plus the arithmetic error here. Now, you know that the mathematical error expression for the central difference formula is given like this just now we have derived this expression for the mathematical error and now we have the arithmetic error just taken from here therefore this is the expression for the total error. Just to have a clear idea of the behavior of this expression let us just take the maximum of all this epsilon and let us denote it by epsilon infinity epsilon infinity is the maximum of the absolute values of epsilon not epsilon 1 and epsilon 2. Now, what you do is you take the absolute value of the total error and that is given by this expression after taking the modulus. Now, you use the triangle inequality to get this expression that is the absolute value of the total error is less than or equal to h square by 12 into mod f 4 of xi plus you again take the modulus in the numerator and apply the triangle inequality you will see that that is less than or equal to mod epsilon 2 plus 2 times mod epsilon 1 plus mod epsilon not and you just replace all these epsilon by the maximum value then you will see that that is less than or equal to 4 times epsilon infinity right then you of course, have a h square term in the denominator. So, that is what I am getting here therefore, when you take the modulus on both sides of the expression for the total error you can see that the modulus of total error is less than or equal to this quantity. So, we have derived an upper bound for the total error now let us try to understand how this upper bound looks like you can see that the upper bound involves a h square here which will make the right hand side to decrease as h decreases, but it may not happen for h tending to 0 because after certain value of h if you further decrease you can see that this term will start making the right hand side to increase drastically. So, that is what I am writing here the error bound will initially get smaller and smaller as you decrease h because of this term, but as you keep on decreasing the value of h then after certain value of h the second term will start dominating and it will take the right hand side to increase drastically. Therefore, there is an optimal h beyond which the right hand side will start increasing rapidly as h tends to 0. In fact, qualitatively you can see that the upper bound of the total error will look graphically like this is what I am trying to say that because of the first term you can see that as h decreases the upper bound will start decreasing, but this may not last for all h greater than 0 you can find a optimal h beyond which if you start decreasing h your upper bound will start increasing rapidly. So, this is what this expression tells you what I am doing here is that I am just plotting the graph of the constant times h square plus some other constant divided by h square you may even take c to be 1 and k to be 1 and its graph will something look like this that is qualitatively how this function as a function of h will behave. Let us take an example where our interest is to find f double dash of pi by 6 for the function f of x equal to cos x. If you use the function value which involves certain approximation then how the central difference formula will approximate the second derivative of the cos function that is the question to be precise we will consider that the approximation f i is obtained from f of x i using 6 significant digits. It means what if you recall from our errors chapter 6 significant digits means the relative error involved in f i when compared to f of x i it will be bounded by half into 10 to the power of minus 5 because you have 6 significant digits right. If you have n significant digits then it is minus n plus 1 that is why you have 5 here also you can use the fact that cos function is always bounded between minus 1 and 1 and from there you can see that the absolute error is less than or equal to 0.5 into 10 to the power of minus 5 right I am just using this here and getting this estimate for the absolute error. We will now use this information into the central difference formula evaluated with rounding error. Rounding error is involved only in the function value that is just for the sake of simplicity we are assuming that the rounding error is involved only in the function value but not in the H value. Then what happens is if you recall the total error has the upper bound as this in that now we will try to eliminate this because this is involving a unknown and also we will try to get an estimate for this epsilon infinity because we have assumed that the approximation in the numbers is just by 6 significant digits right. So, with that you can see that epsilon infinity can be taken as 0.5 into 10 to the power of minus 5 and just to eliminate this what I am doing is I am taking epsilon to be approximately pi by 6 why I took pi by 6 because that is the point at which I want to find the derivative of the function. You may use some other idea but I am just using this idea to eliminate this xi with that I can see that the total error is bounded by h square by 12 into cos pi by 6 this is a very crude approximation actually plus 4 times epsilon infinity is less than equal to this. So, I will put this in the place of epsilon infinity and that gives me precisely how this upper bound function will look like this is an estimate now for me let me denote this estimate by e of h it is a function of h. So, we have the upper bound for the total error and we are using the notation e of h for that now our interest is to find the optimal h what optimal h means optimal h means we want to find the value of this h that h such that when you decrease your parameter h greater than that point let us call this as h star when h is greater than h star as you decrease h in this region your total error will tend to decrease at least its upper bound will tend to decrease if you take h less than the optimal h h star then as you go on decreasing h your total errors upper bound will increase. So, we want to find this optimal h how will you find the optimal h well the graph of the function e of h will be qualitatively looking like how I have shown in the graph of the previous slide. Therefore, it is very clear that the optimal h can be obtained as the minimum of the function e of h right. So, let us denote this optimal h by h star and it can be obtained by finding the minimum of this function. So, that optimal h is such that the upper bound will increase drastically if you go on reducing h in the region 0 to h star that is the danger of you know go on decreasing the parameter h believing or thinking that you will get better and better approximation no it is not true if you are putting this formulas on a computer then there is a limit beyond which you cannot go on decreasing your h that is the moral of this story. Let us see what is this h star in this particular example to find that you have to find the derivative of this function e of h and see where that derivative vanishes right. This is a simple and well known technique introduced in the calculus course how to find maximum and minimum of a function right using that technique you can find this h star and in this example that h star is given by 0.129. So, remember if you take h less than 0.129 and from there if you keep on decreasing the parameter h then the upper bound of the total error will increase that is what we are seeing that does not mean that the total error should increase. But unfortunately it will also increase in this particular example what I am doing is I am taking f double dash of pi by 6 right. Remember f of x is taken to be cos x therefore f double dash of pi by 6 is given by this number up to some 10 digits I am taking and now let us see how this central difference formula will approximate the second derivative of the function f that is cos function when you compute these values on a computer. I am just taking h equal to 0.2 which is greater than the optimal error in this example will the value obtained for f double dash at pi by 6 using the central difference formula on the computer that is why I am putting a bar there because it will certainly involve the arithmetic error right and computer gave us this value which is pretty close to the exact value I mean this is also strictly speaking not the exact value but this is the value with better accuracy than this value that is what it means and when compared to this value this approximate value has the total error like this and the upper bound has the value this remember the upper bound expression is given like this. So, you can plug in h in this expression and get the upper bound value also at h equal to 0.2 and that is given like this. Now let me take h is equal to h star that is the optimal value of h right with that h your central difference formula gives this approximation the total error involved in it is this well you can see that from h equal to 0.2 to h equal to 0.129 the total error decreased as we expected also the value of the upper bound also decreased let us go ahead and choose a h less than the optimal error and see what happens now I am choosing h as 0.005 which is less than h star right you can see that the value of the central difference formula for the function f at the point pi by 6 is given like this the total error you can see that it increased from what you have got for h equal to 0.129 by reducing the value of h we got a bad approximation but theoretically when you take h tending to 0 you are supposed to get better and better approximation but numerically when you compute on a computer the situation is different that shows that computer is now started making significant error in this calculation coming through the rounding error right you can also see that the upper bound as drastically increased I have further gone ahead and took h equal to 0.001 well I am decreasing h therefore mathematically I should get better approximation for the second derivative using the central difference formula but my central difference formula gave this as the value and therefore what is the total error involved in it it is really almost 100 percent error it gave and the upper bound value is 20 right. So, that is the danger in dealing with the computation especially when you do on a computer you have to be very careful mathematically it may be that when you go on decreasing certain parameters you will get better and better approximation but it may not be the situation when you go to implement these methods on a computer one has to carefully understand the error analysis both from the mathematical point of view as well as from the computation point of view with this note let us conclude this lecture thank you for your attention.