 Hi, let us continue our discussion on arithmetic errors. So far we have learnt what is mean by floating point representation and floating point approximations. And we also learnt what is mean by loss of significance and then we have also seen through an example the danger in losing significant digits in our calculation. Further we have also seen that only subtracting two very close positive numbers will lead to loss of significance whereas, adding two positive numbers multiplication and division are relatively safe as far as the amplification of relative error is concerned. In this class, we will learn how to judge whether a function is good or bad when we go to evaluate it on a computer using a concept called condition numbers. Before going into this, let us continue our discussion and understand what is mean by total error involved in a calculation. Recall from our previous discussion that arithmetic using n digit rounding and chopping is done using the steps which leads to the computed value given by this. When we want to compute x circle dot y, we land up getting this value on a computer. This is what we have understood in the previous discussions. Now the question is what is the total error involved in this calculation? First thing is the error in f l of x and f l of y due to n digit rounding or chopping. And next is after calculating we further do a floating point approximation one more time. Therefore, there are two levels of approximations involved in this calculation and therefore, two levels of errors are involved in this calculation. Therefore, the total error is defined as the value that we want to find. This is ultimately what we are interested in and this is what our computer has calculated and given to us. The difference between these two is called the total error and that can be written as x circle y which is the true value. Now what we are doing is we are adding and subtracting this intermediate one. So, remember you took f l of x, f l of y and then first you calculated that and then you went to make the approximation. So, this value is what is added and subtracted here. Now you can see that the true value minus this value is the first level of error and further in the second level this value is further made an approximation here. Therefore, this is the second level of the error that is involved in it. Adding these two leads to the total error. In this the first part is called the propagation error and the second part of the error is called the floating point error that is this part. Let us have one more example to have a feeling of how this errors propagate from one step to another step. Let us take the integral 0 to 1 x to the power of n divided by x plus 5 into dx. I want to perform this integral for some n say for instance n is equal to 0, 1, 2 up to 20. There are two ways that you can do it. One is you can use this formula. One can easily derive this formula from this integral. I will leave it to you to derive that. So, what you can do is you start with n equal to 0. Clearly n equal to 0 will give you a very simple calculation and that can lead to the exact value of I 0. Once you have this then you can substitute I 0 here. Say you take n is equal to 1 you have I 1 which is equal to 1 minus 5 into I naught this one. I naught I naught is already known to us here. So, you can substitute that to get I 1. Once you have I 1 then to get I 2 you can use this formula again 1 by 2 minus 5 into I 1 and so on. You can go on like this. So, you start with the exact value of the integral and go on with the iteration and you can reach whatever n that you want to get right. So, that is the idea. There is also one more way of getting it. What you do is you start with some n greater than 20 and from there you decrease your n and reach n equal to 20. In that case you can also use this formula. This can also be derived from this integral. It is very simple. You can see how to derive that. And now what we do is we start with some n greater than 20. For instance I have started with n equal to 30. In that case we have the integral value which is I will not say it is exact, but it is very pretty good approximation. How I found well I used Simpson's rule to get it and I will start with this value and you can see how the iteration is defined. When you have n equal to 30 then this will be I 29 which is equal to 1 by 5 into 30 that is this first term minus 1 by 5 into I 30. We know already what is I 30. So, you plug in this here and you get the value of I 29. Once you have this you can go for I 28 that is equal to 1 by 5 into 29 minus 1 by 5 into I 29 and so on. I 29 again you know you can plug in that here and you can go on like this. So, in that way you can also reach I 20 and get the value of I 20 here. So, I have performed this calculation and I have shown the results in this table. I have used 6 digit rounding in this calculation. So, from the forward what I am doing is from I naught I have calculated I 1 and so on and similarly for the backward I started from somewhere I 30 somewhere and came up to I 1. So, from the backward direction and I am also showing the exact value. You can see I am just not showing it for 1, 2, 3 and so on. I am just jumping from 1 to 5, but to get I 5 I have to calculate I 2, I 3, I 4 and then only I will get I 5. I am only not displaying it here and then going forward I have I 10 remember for this I have to come forward and for this I have to go backward from the larger value of n to smaller value of n I am coming backward ok. So, you can see that the forward value and the exact value are pretty coinciding for some n and after 15 or 20 onwards there is a drastic difference between exact value and the computed value here. However, you can see that the backward iteration is maintaining the accuracy very well when compared to the exact solution that is what we observe here. You can see that at the 30th iteration I remember I have started with n equal to 30 for backward and I have gone upward and reached 1 and even when I reached 1 it was pretty good approximation whereas, for forward I started with I naught and went forward in n and reached I 30 you can see that the value that is computed using our forward formula is no way near to the exact value. Now, what went wrong in this forward iteration is the question. Let us go back to the formula and see what went wrong. Let us take this at every iteration you make a approximation here and that approximation approximated value is plugged in here and therefore, this I 1 is the value plus some error and that error is getting amplified 5 times at every step whereas, what is happening in the backward formula is the error that you committed in I 29 is when it is substituted here the error is getting divided by 5. So, that is why the propagation of error in the backward formula is rather very slow whereas, the propagation of error from one step to the other step in the forward formula is very fast because every time it is multiplied by 5 whereas, here it is divided by 5 that is the idea here. So, this shows how the propagation error can go from one step to the other step. Let us now take the next topic of evaluating a function on a computer. So, how will we judge whether evaluating a function on a computer is good or not it is based on a concept called condition number of a function. Let us try to understand what is mean by the condition number of a function. Let us take a function f from r to r and we want to find f of x, but instead of x we are only provided with an approximate value say x a therefore, we are only getting f of x a, but not f of x. So, let us consider this situation and see how the relative error will get into the function value when compared to the initial error in x a when compared to x. For that we will use the mean value theorem to write f of x minus f of x a this is the error in the function value right and that can be written as f dash of xi into x minus a for some unknown xi lying between x and x a. Now from here we want to get the relative error therefore, you divide this by f of x right and therefore, this side also we have to divide by f of x and that will give you relative error in f of x a when compared to f of x and that is equal to f dash of xi divided by f of x into x minus x a. Now what you do you also want to make this a relative error therefore, divide this also by x then you will have x multiplied here and divide here that gives you f dash of xi divided by f of x into x relative error in x a when compared to x right. So, this is what we are getting from here you can clearly see that the relative error in the function value is amplified by this factor of the relative error in x a when compared to x that is what we are understanding. Now instead of having this expression which is not very good for us because this xi is not known to us. So, what we will do is we will take xi approximately equal to x because in numerical analysis we need to have a quantified number rather than something which is not known to us here you can see that x and x a are pretty close to each other right that is what is the understanding of having x a is approximately equal to x and now xi is lying between these two numbers right. Therefore, it may be a good approximation to take either xi equal to this or this look we do not know what is x a because that is decided by the computer. When you plug in x the computer takes that x as an approximate value by chopping or rounding or whatever it does. Therefore, this number is not known to us whereas, this number is known to us because this is the point where we want to find the function value. Therefore, it is fair to take x approximately equal to xi and plug in that into your mean value expression this is only approximately equal to remember it is not exactly equal to and then based on that you get the expression like this. Now, you see this factor is something which is kind of known to us because all these arguments are known to us right and that is what is called the condition number. Let us define the condition number of a function you are given a c 1 function condition number of a c 1 function that is continuously differentiable function at a point c is given by mod f dash of c divided by f of c into c because this number will give you an idea of how much the initial error is amplified and got into the function value. So, therefore, this number plays a important role in getting a feeling of how good or bad a function is when you go to evaluate it on a computer. So, this is what we call as well conditioned or ill conditioned the process of evaluating a continuously differentiable function f at a point c remember this is point wise analysis you give me a point c. Now, I want to know whether the process of evaluating that function at the point c is good or bad. If the condition number is small then we say that the process of evaluating the function at that point is well conditioned otherwise it is ill conditioned. Now, this is rather a very vague definition because it does not quantify what is mean by small or big well this is something which you really cannot quantify in practical applications because it depends on two factors one is in what application you are working with and second thing is what level of error that you can really afford to tolerate. Suppose, you are working with a very powerful computer it can handle more error whereas, if you have a very small computer may be even a small error can propagate very fast right. Similarly, it also depends on what kind of applications you are working with for instance we have seen that in missile path finding it is we cannot afford to have even a very small error like 0.1 percent or something like that whereas, if you are working with some applications like in finance and all even 30 40 percent error may be good enough. Therefore, you really cannot quantify this smallness in practical situations. However, from the academic point of view whenever this number is less than 1 we always say that it is well conditioned on the other hand if this number is tending to infinity as c tends to some number say x naught then we say that evaluating that function in a neighborhood of x naught is going to be very bad ok. These are something which you can surely tell right because if it is less than 1 it means something like what we did in the backward iteration in the previous integration there the error is getting reduced because of the 1 by 5 factor. Now, the similar thing will happen here whereas, if the condition number is going to infinity it means it is going to amplify the error in the function value drastically when you go to compute the function value very near to the bad point x naught. So, that is the idea let us see some examples consider the function f of x equal to root x if dash is given by this and from there you can immediately find the condition number of the function f of x equal to root x. And that gives half and that gives as a feeling that evaluating the square root of a number is going to be very nice on a computer it is not going to be dangerous at all ok because the error in the function value is going to be approximately half of the error that you have committed in the argument right. So, that sounds good therefore, we will declare that finding square root on a computer is well conditioned. Let us take another example f of x equal to 10 divided by 1 minus x square and here you can see that the condition number is given by this and from here you can see that as x tends to 1 the condition number is blowing up to infinity and that shows that evaluating this function near either 1 or minus 1 is going to be ill conditioned definitely right. Let us take another example and this is an interesting example look this function is not involving one single operation something like plus or minus or square root these are one single operation. Here you can see that the function evaluation includes more than one operation right it has a addition it has a square root and it has subtraction also. Let us try to see what is the condition number of this function with little effort you can see that the condition number of this function is less than or equal to 1 by 2 that gives us a feeling that evaluating this function on a computer is going to be very nice and therefore, it sounds as if it is well conditioned. But actually this is not true because we have seen an example in our previous class that a similar function has amplified the error drastically because of the loss of significance you can see that if x is very large then this number and this number are very close to each other and therefore, you tend to get loss of significance in the process of evaluating this function. Therefore, that understanding shows that this function is not going to be well behaved for large values of x. However, the condition number says that it is very good right. So, there is a conflict between these two ideas the actual thing is that when you go to study the behavior of a function which involves many arithmetic operations then just seeing the condition number of the function is not enough you need to look for the stability criteria. Now, what is mean by stability let us define that suppose that there are n steps to evaluate a function f of x at a point x equal to c just like in the previous example we have there are how many steps you take x and then you add with one that is one step and then you find the square root of that that is another step and then you find square root of x that is another step and then you subtract this. So, 1 2 3 4 steps are involved in evaluating this function right in this case just finding well condition of this function is not a good idea to judge whether the function is good or not what you have to do is you have to look for the well condition or ill condition of each of the steps that is what the stability idea says. Suppose, there are n steps to evaluate a function f of x at a point c then the total process of evaluating this function that is finding the value of this function involves four steps that is what we meant by saying total process of evaluating this function is said to be unstable if at least one of these steps is ill condition at least one of this steps. If all the steps are well condition then only we will say that the process of evaluating this function is stable. Let us take this example and see whether this is stable or unstable in the previous example we have seen that it is a well conditioned function as far as the condition number of the function is concerned. Now, we will split this function into each step and see what is the condition number of each of these steps. How will you split? You want to find the value of that function at some point x naught the first step is you will take that x naught and add it with one that is this that is the first step once you do that you call it as x 1 and then take the square root of x 1 that will give you the second step let us call this as x 2 and then you go to the second term take x naught and plug in here and get the square root let us call it as x 3 that is step 3 and then once you have x 2 and x 3 then you subtract both of them you get x 4. Now, you have to find condition number for each of this by considering each step as a individual function that is let us take f 1 of x equal to x plus 1 this is one function find the condition number of this function then take f 2 of x is equal to root x that involves these two steps find the condition number of this function and then take f 3 of x equal to either you can write x minus x 3 or x 2 minus x 3 one of them you can fix and vary the other one that is enough let me take x minus x 3 you can also take x 2 minus x it does not matter for you as long as seeing the condition number of the function this step alone. Now, from your previous experience you can see that this is a well conditioned function this is also well conditioned you can easily judge that this is not going to be well conditioned especially when x tends to x 3 that is whenever x goes very close to x 3 which is precisely going to happen when x is very large when x is very large this term and this term are going to be pretty close to each other therefore, when x tends to x 3 this will be very bad why it is so let us try to see that f 3 dash of x divided by f 3 of x into x that is the condition number that is going to be 1 divided by x minus x 3 into x and obviously, this tends to infinity as x tends to x 3 therefore, this step is a very bad step it is ill conditioned and that will make the process of evaluating this function as unstable ok. Individually if you see the condition number of this function that is misleading it is well conditioned however, the process of evaluating this is unstable therefore, this is very dangerous to evaluate on a computer because of this step this is the idea as we have seen in the previous example there is a scope to rewrite this function in a different form mathematically equivalent and that form can give us a better result ok. What is that form that is nothing, but 1 by root x plus 1 plus root x. So, this is actually equal to root x plus 1 minus root x right, but if you find the value using this expression this is bad, but the same if you use this expression this is very good. Why you can again split this function into each individual steps like what we did in the previous example you can see that each of the steps are good ok. Even you have one at one step dividing 1 by some x that is also good we have seen there is no subtraction involved in this. The basic idea in deciding whether evaluating some expression is good or not is to see whether it involves subtracting two positive numbers which are close to each other. That is the overall message that we are taking from this analysis with this our chapter on arithmetic error is finished. Thank you for your attention.