 Today, we are going to consider some of the problems for the Newton's method. So, we look at the f x is equal to 0, we want to find the root. So, approximation to this root is obtained by Newton's method. In Newton's method, we have proved that there is quadratic convergence. In fact, that is the advantage of Newton's method. So, we will look at a specific example and in that we will show that the quadratic convergence is achieved. Then we will look at one more example for finding approximation to square root of a function using Newton's method. After that, we will consider some problems for solution of system of linear equations. So, we will calculate condition number in terms of maximum magnification and minimum magnification of the coefficient matrix A. Then we will look at the residual obtained estimates for the residual and also we will consider the condition number, how it depends on the scaling. So, our first example is we want to find root of this function f x is equal to e raise to minus a x minus x, where on a our condition is that 0 should be less than a less than or equal to 1. We will show that this function is going to have a unique root in the interval 0 to infinity, then the x 0 is the initial approximation in the Newton's method. So, if this initial approximation you choose to be bigger than 0, then we will show that modulus of c minus x n plus 1 is less than or equal to 1 by 2 modulus of c minus x n square. c is the exact root x n plus 1 is the n plus first iterate in the Newton's method. So, this is the error in the n plus first iterate c minus x n square that is going to be the error in the nth iterate and error in the n plus first iterate is less than or equal to 1 by 2 times error in the nth iterate its square. So, this will illustrate the quadratic convergence of Newton's method. So, the first thing we are going to show that this function has a unique root in the interval 0 to infinity. So, let us look at the Newton's iterate. It is by definition x n plus 1 is equal to x n minus f x n upon f dash x n, n is equal to 0, 1, 2 and so on. So, I am recalling how the error in the n plus first iterate and error in the nth iterate they are related f of c is equal to 0. Write down the Taylor series expansion for f of c. So, that is going to be equal to f of x n plus c minus x n f dash x n plus c minus x n square by 2 f double dash of d n. This d n is going to lie between c and x n. Then what we do is we divide by f dash x n, take this term on the other side. So, that gives you x n minus f x n upon f dash x n minus c. This will be equal to f double dash d n. These two you are dividing by f dash x n and then c minus x n square. So, now this is nothing but x n plus 1. So, the left hand side is x n plus 1 minus c. I am taking modulus. So, it is modulus of c minus x n plus 1. This is equal to modulus of f double dash d n upon 2 times f dash x n and multiplied by c minus x n square. In this particular example, we want to show that modulus of c minus x n plus 1 is less than or equal to 1 by 2 into modulus of c minus x n square. So, that means I need to show that this coefficient of c minus x n square is going to be less than or equal to half. So, here is proof of unique root in interval 0 to infinity. Our function is f x is equal to e raise to minus a x minus x. So, f dash x is going to be e raise to minus a x minus 1. x n plus 1 is equal to x n minus f x n. So, that is e raise to minus a x n minus x n divided by f dash x n. So, it is minus a e raise to minus a x n minus 1. So, denominator you have got negative sign. Here this is negative sign. So, this will become plus. Then x n and this x n it is going to get cancelled. So, you are left with a e raise to minus a x n multiplied by x n and then from here plus e raise to minus a x n divided by a e raise to minus a x n plus 1. So, we are starting with x 0 to be bigger than 0. Now, look at x 1. Our a is lying between 0 and 1. So, this number is bigger than 0. Exponential is always bigger than 0. x 0 is greater than 0. Similarly, the denominator will be bigger than 0. So, that will mean that x 1 is bigger than 0. You continue and then you get x n to be bigger than 0. In fact, I am not showing that f has a unique root. I will tell you how it follows. Look at f dash x. f dash x is going to be less than 0. If f dash x is less than 0, that will mean that f is going to be a strictly decreasing function. When you consider f of 0, f of 0 is going to be equal to f of 0 to 1. So, our f x is e raise to minus a x minus x. f dash x is this is e f dash x is minus a e raise to minus a x minus 1. a is bigger than 0. So, f dash x will be less than 0. So, f dash x will be less than 0 on 0 to infinity. This implies f to be strictly decreasing. When you consider f of 0, it is going to be equal to 1. So, it is bigger than 0 and as x tends to infinity, f of x will tend to minus infinity. If f of x is tending to minus infinity, then there will exist some m such that x bigger than or equal to m will imply that f of x is say less than minus 1. f x is tending to minus infinity as x tends to infinity. So, by definition, this means that for some m, whenever x is bigger than or equal to m, f x is going to be less than minus 1. Now, you look at interval 0 to m. Your function f is continuous. At f of 0, it is 1. At m, it is going to be f of m will be less than minus 1. So, by intermediate value theorem for the continuous function, there has to be some c in the interval 0 to m such that. So, we will have c. So, there will exist c in the interval 0 to m such that f of c is equal to 0. So, that means we have proved existence of 0 of f and now strictly decreasing implies f has a greater than or equal to 0. So, this was the proof for showing that our function f has a unique 0 in the interval 0 to infinity. Now, we are looking at x 0 to be bigger than 0. If the initial approximation is bigger than 0, we calculated formula for x n plus 1 in terms of x n and using that formula, we saw that x 1 has to be bigger than 0 and then use the same argument. x 1 bigger than 0 will imply x 2 bigger than 0 and in general, your x n's they are going to be bigger than 0. Then, we look at c minus x n plus 1 and then c minus x n. So, we obtain x n plus 1 to be this much. Then, f x is e raise to minus a x minus x, f dash x is minus a e raise to minus a x minus 1. The second derivative is going to be a square e raise to minus a x. So, when you consider modulus of f double dash d n divided by 2 minus a x minus a x minus 2 times f dash x n, this is going to be equal to a square e raise to minus a d n divided by 2 times a e raise to minus a x n plus 1 and now, this is going to be less than or equal to half. Why? Because our 0 is less than a less than or equal to 1. So, a square will be less than or equal to 1. e raise to minus a d n, a is bigger than 0, d n is going to lie between our point c and x n. So, that is why e raise to minus a d n also will be less than 1 divided by 2 a e raise to minus a x n plus 1. So, this number is going to be something bigger than 0. So, I can dominate 1 upon a e raise to minus a x n plus 1 by 1 minus a x n plus 1. So, that is why e raise to minus a d n also will be less than or equal to 1. So, numerator is less than or equal to 1, 1 upon this factor is going to be less than or equal to 1 and then you have got 2. So, you get this to be less than or equal to half and that gives you modulus of c minus x n plus 1 less than or equal to half c minus x n square. So, this illustrates the quadratic convergence of the equation. Now, why was it necessary to obtain this expression? Using this expression we show that all x n's they are bigger than 0. Our c is in the interval 0 to infinity. So, we have got c is in the interval 0 to infinity, x n is in the interval 0 to infinity, and d n it will lie between c and x n, x n can be to the left of c or to the right of c. So, d n also will be in the interval 0 to infinity and then using that you get e raise to minus a d n to be less than or equal to 1. So, in order to obtain this estimate we need to look at what is x n plus 1 and then we have proved the quadratic convergence. So, now in our next example what we are going to do is when we want to find the square root of a positive real number then we want to apply Newton's method to that to find an approximation to the square root. So, our function f x is going to be equal to x square minus a, where a is bigger than 0. So, we want to find the root of this. So, x square minus a is equal to 0 and let us decide that we want to find the positive square root. So, to this function we will apply Newton's method then that will give you a formula for finding x n. Now, remember the Newton's method may not converge always when it converges it is going to converge quadratically. We have got our formula x n plus 1, x n plus 1 is equal to x n minus f x n divided by f dash x n. If this sequence you are defining a sequence x 0 is your initial approximation and then you are defining x n. If these x n converge to a point then that point or that number is definitely going to be 0 of a function. So, let me repeat the Newton's method defines a sequence of real numbers. If that sequence is convergent then the limit is going to be 0 or a root of a function. If there is convergence then the convergence is going to be quadratic provided that the your 0 is a simple 0, but there may not be a convergence. We have seen some sufficient conditions for which guarantee that there is a convergence in Newton's method. So, we are going to look at Newton's method for finding root of x square minus a is equal to 0. We will show that the sequence or the Newton's iterates they converge. So, this is our next problem. So, we start with a to be bigger than 0, function f x is x square minus a, x n's are the iterates in Newton's method. About the initial approximation our condition is x 0 should be bigger than 0 and x 0 should not be equal to root a. We want to show that x n's they are strictly decreasing x n's converge to root of a and root a minus x n plus 1. So, that is going to be the error in the n plus first iterate. This is equal to minus 1 upon 2 x n multiplied by root a minus x n square. So, here is error in n plus first iterate. Here is error in nth iterate and then it is square. So, once I show this then that will mean that we have got quadratic convergence. We will keep a track as to where we need x 0 to be bigger than 0 and x 0 to be not equal to root a. If x 0 is equal to root a then x 1 also will be equal to root a and the sequence generated by Newton's method is going to be a constant sequence. So, here that means you have found the root f x is equal to x square minus a. Its derivative will be given by 2 x n plus 1 is equal to x n minus f x n divided by f dash x n. This is equal to x n minus f x n will be x n square minus a divided by f dash x n which is 2 x n. So, to start with when I look at x 1 I am going to have x 0 in the denominator. So, that is why I need x 0 to be bigger than 0. x 0 we say it should not be equal to root a because if it is equal to root a then what will happen is x 1. So, x 0 square will be equal to a. So, this term will be go away and you will have x 1 is equal to x 0. So, you get a constant sequence. So, this is equal to 1 by 2 x n plus a by x n. You have 2 x n square minus x n square divided by 2 x n. So, that gives you 1 upon 2 x n and then this a upon 2 x n is written here. So, x n plus 1 is equal to 1 by 2 into x n plus a by x n. A is bigger than 0. So, that will mean that x n they are going to be bigger than 0. x 0 bigger than 0 implies x 1 bigger than 0 that will imply x 2 bigger than 0 and so on. So, this is our first step x n plus 1 is 1 by 2 x n plus a by x n. Then let us look at x n plus 1 square minus a. So, square both the sides and subtract a square of this is 1 by 4 x n square plus a square by x n square plus 2 a minus a. So, this will be equal to 1 by 4. Now, here you have got 2 a by 4. So, that is going to be a by 2 and then this is minus a. So, you will have minus a by 2 and that gives you x n minus a by x n square. So, this will be equal to nothing but x n square minus a divided by 2 x n whole square and that gives you x n to be bigger than root a for n bigger than or equal to 1. So, we start with x 0 to be bigger than 0. Then your x 1 it is bigger than root a and this condition will be satisfied by x 2, x 3 and so on. So, your first x 0 you have got 0, you have got root a. So, your x 0 which you choose it may be to the left of root a or to the right of root a, but after the first iterate x 1, x 2, x 3 they are all going to lie to the right of root a. Now, this becomes important because we are going to show. So, we have got now x n to be bigger than root a. Now, we will show that they are strictly decreasing. So, if you have got a monotonically decreasing sequence which is bounded below, then such sequence is always convergent. That is a property of real sequences that monotonically increasing sequence which is bounded above that is convergent or monotonically decreasing sequence which is bounded below that is going to be convergent. So, we have got x n to be bigger than root a for n is equal to 1, 2 and so on. So, we have got a sequence which is bounded below and now let us show that it is a decreasing sequence. So, x n plus 1 is 1 by 2 x n plus a by x n we obtain this expression look at x n plus 1 minus x n. So, we are subtracting minus x n from both the sides. So, what you will have will be 1 upon 2 a by x n here it is 1 by 2 x n minus x n minus x n minus x n minus x n so that is going to be minus 1 by 2 x n. So, this is equal to 1 by 2 a minus x n square and then there should be divided by x n. So, we have got x n plus 1 minus x n is equal to 1 upon 2 x n a minus x n square. Just now we have proved that x n is bigger than root a for n is equal to 1, 2 and so on. So, this will mean that x n square will be bigger than a and that implies a minus x n square is going to be less than 0. So, we have got x n plus 1 minus x n minus x n plus 1 minus x n minus x n minus x n minus to be less than 0 because this a minus x n square is less than 0 and our x n they are going to be bigger than root a and hence bigger than 0. So, this implies that x n is a decreasing sequence. So, we have got a decreasing sequence which is bounded below and hence it will converge. Now, just strictly decreasing bounded below it tells us that it is convergent. Now, why it should converge to root a because if the sequence generated in the Newton's method if it is convergent it has to converge to root of your function f roots of our function are plus root a and minus root a. The sequence which is generated it is going to be sequence which is bigger than 0. So, if it is convergent it has to converge to root a it cannot converge to minus root of a. So, thus we have proved that the sequence generated in the Newton's method it is converging to root of a and now let us show the quadratic convergence. So, x n plus 1 is 1 by 2 x n plus a by x n it is converging to root of a. So, let me look at root a minus x n plus 1 that will be root a minus 1 by 2 x n plus a by x n. Now, this is going to be equal to minus 1 upon 2 x n. So, I am taking 2 x n as the denominator. So, I will have here x n square then you will have 2 root a x n because there is minus sign here it will be minus 2 root a x n plus 2 x n plus 2 root a x n is taken out. So, it will be a which is equal to minus 1 upon 2 x n and root a minus x n whole square this is e n plus 1 minus 1 upon 2 x n e n square. So, we have e n plus 1 is equal to minus 1 upon 2 x n e n square. So, e n plus 1 upon e n square this is going to be equal to let me take the modulus this will be 1 upon 2 x n this will converge to 1 upon 2 x n. So, 1 upon 2 root a because x n is tending to root a. So, this will mean that 1 upon 2 root a that is going to be our asymptotic error constant and if I recall the earlier notation then p is equal to 2. So, that is quadratic convergence limit as a intends to infinity modulus of e n plus 1 by mod e n square that is equal to 1 upon 2 root a and that is the asymptotic error constant and we have got quadratic convergence. We want to consider some of the problems which are related to system of linear equations. So, when we talked about the condition number we had obtained a lower bound for the condition number which was condition number of a is bigger than or equal to norm c j by norm c i where c j is j th column c i is i th column. So, using this estimate we had said that if the columns are not balanced if they are out of order then your matrix is illiterate and your matrix is ill conditioned. But, one says for the columns it is true for the rows also. So, the result about that condition number of a is bigger than or equal to norm c j by norm c i I had left that as an exercise. So, that is what now we will do we will define what is known as minimum magnification of a relate the minimum magnification of a to norm of inverse and then we will obtain condition number as ratio of maximum magnification divided by minimum magnification and then I will recall one of the example which we had considered of a 2 by 2 matrix. So, that matrix was ill conditioned. So, we showed that you change the right hand side slightly and perturbation in the solution is too big. So, here is the problem a is an invertible matrix minimum magnification of a is minimum magnification of a is minimum of norm a x by norm x x not equal to 0. If instead of minimum we have maximum here that is our definition of norm a. So, we want to show that minimum magnification of a is nothing but 1 upon norm a inverse the proof is straight forward we start with minimum magnification of a this is our definition. So, this will be same as minimum x not equal to 0 vector 1 upon norm x divided by norm a x. So, this becomes equal to 1 upon maximum x not equal to 0 norm x divided by norm a x when we have got a to be invertible matrix then x not equal to 0 vector implies a x not equal to 0 vector. So, proof of this result is where a x is equal to 0 vector it will imply that a inverse a x is also 0 vector I am applying a inverse and this will mean that x is equal to 0 vector contradiction. And if a x is not equal to 0 vector then x cannot be 0 this is for any matrix that if you have got x is equal to 0 vector a x is equal to 0 vector a x is always 0 whatever is the matrix whether it is invertible or not. So, for a invertible matrix we have got this if and only if condition. So, now look at the minimum magnification of a. So, here we are taking maximum over x norm x upon norm a x let me put x is equal to y. So, y is equal to a x then when I this will be same as 1 upon maximum y not equal to 0 for x we have got a inverse y divided by norm y and that is nothing but 1 upon norm a inverse. Now, once we show this the condition number is norm a into norm a inverse norm a is maximum norm a x by norm x x not equal to 0 vector and then this norm a inverse which is in the numerator I write as 1 upon norm a inverse in the denominator. So, this is maximum magnification of a just now we showed that 1 upon norm a inverse is minimum magnification of a. So, the condition number is going to be equal to ratio of maximum magnification of a divided by minimum magnification of a. So, now let me recall the 2 by 2 examples which we had considered for this matrix it is infinity norm is 1999 infinity norm that means row some norm take the absolute value of the entries look at the first row look at the second row take the sum whichever is the maximum a of vector 1 1 is going to be 1999 99 divided by this. So, you have norm x infinity is 1 1 norm a x infinity is going to be maximum of these 2 numbers. So, you have for this particular vector norm a x divided by norm x is equal to norm a infinity. So, this 1 1 is going to be the direction of maximum magnification for this matrix. Now, let us calculate or let us find out the direction of minimum magnification. So, the direction of the minimum magnification is going to be direction of maximum magnification for a inverse. So, a inverse is this matrix it is infinity norm is again the same as before a inverse of minus 1 1 is going to be this vector 1997 minus 1999. So, minus 1 1 is the direction of maximum magnification for a inverse this relation you can write as a of this vector is equal to minus 1 1. So, this vector defines direction of minimum magnification by a the direction of maximum magnification by a. So, this vector defines direction of maximum magnification was 1 1 and minimum magnification will be decided by this vector. Let us go back to our system of linear equations. So, we have got a x is equal to b we obtain an approximate solution x cap or it is our computed solution. There is some error because you are using computer. So, we have got finite precision or you may be using some indirect method such as Jacobi method or Gauss-Seidel method. So, what we want to do is. So, residual is something which you are going to calculate like I find a computed solution x cap I will like to know whether this x cap is near to the exact solution exact solution we do not know. So, what we cannot calculate norm of x minus x cap or we cannot calculate the relative error what we can calculate is a x cap minus b. If x cap where exact solution is x cap then a x cap will be equal to b and the residual will be 0 otherwise it will be something non-zero. So, what one wants to know is if the residual is small then whether I can say that my computed solution is near to the exact solution we have got computed solution x cap I calculate a x cap minus b. So, suppose this number is small its norm is small in that case whether one can say that x cap is near to x. So, that is the question now that once again it is going to depend on the condition number. If your condition number is small then the residual small will mean that x cap is near x. If the condition number is big then it may not be the k. So, let us prove the corresponding result. So, a is invertible matrix x cap is approximate solution r is the residual b minus a x cap and then this is going to be the relative error in the computed solution. This will be less than or equal to norm a into norm a inverse the condition number into norm r. So, what it means is if this number is a big number then even though this is small it can happen that the residual it can happen the relative error in the computed solution can be big. If this is something reasonable then the small residual implies the nearness of the computed solution to the exact solution. So, we have a x is equal to b r is equal to b minus a x cap. So, that will mean that a x minus a x cap is equal to r just from these two equation a is invertible matrix. So, x minus x cap is equal to a inverse r norm of x minus x cap will be less than or equal to norm a inverse into norm r. Now, norm b from here will be less than or equal to norm a into norm x. So, we get 1 upon norm x to be less than or equal to norm a into norm b. So, in the earlier estimate what I had written there should be norm r by norm b. Now that is something logical here you are taking relative error. So, here you should not look at the absolute error absolute error will be norm r norm of b minus a x cap. So, here also you should look at the relative error. So, that is norm r by norm b. So, if this relative error is small and if the condition number is not too big it will imply that this relative error will be small and then other way inequality is similar. So, we have got a x minus a x cap is equal to r. So, do not apply a inverse from here you can say that norm r is less than or equal to norm a into norm of x minus x cap a x is equal to b. So, you have got x is equal to a inverse b. So, norm x is less than or equal to norm a inverse into norm b. So, from combining these two you will get 1 upon norm a into norm a inverse norm r into norm b to be less than or equal to this relative error. Now, we want to consider an example about scaling when you multiply a matrix by a non-zero number. You multiply a matrix by a non-zero number it does not change the condition number the condition number remains the same if I multiply the matrix by a non-zero number, but if I multiply say one of the row by a non-zero number then the condition number is going to be affected. So, we are going to look at a 2 by 2 example and in that we will try to determine the non-zero scalar alpha by which if I multiply the first row then the condition number is minimized. So, that is the scaling that the matrix is given to you. So, you look at its rows and columns and try to see that they are not say out of proportion. If you multiply a row by a non-zero constant you do not change the system it is the same system as the original system, but new system may have a better condition number. If you multiply a column of the coefficient matrix by a non-zero scalar then your solution gets changed it is a new system, but there is a relation between the earlier solution and the solution which you will obtain in the new system and that is if you multiply the j th column by a non-zero constant the corresponding solution will change only in the j th component. So, that we have seen. So, we are going to look at a 2 by 2 matrix or a 2 by 2 system. So, we have this is a matrix what I want to do is determine alpha. So, that the condition number of a alpha with respect to the infinity norm is minimized. Now, here is the 2 by 2 matrix so, I can calculate its inverse infinity norm is specified. So, what we are going to do is look at a alpha look at its norm a alpha infinity calculate a alpha inverse calculate its infinity norm and then we will have to consider 2 cases and in both the cases we will see that we get the same alpha. So, we are trying to find alpha which minimizes the condition number of our coefficient matrix. So, we have this is our matrix a alpha when you look at the infinity norm of the matrix that is rho sum north you have to take the modulus. So, first rho its sum will be 0.2 into mod alpha the second rho its sum is 3.5 and norm a alpha infinity will be maximum of these 2 numbers you look at a alpha inverse the inverse of this matrix a alpha is given by this our alpha is going to be not 0. So, this is the inverse now we will calculate its infinity norm. So, norm a alpha inverse will be the first rho gives us 2.5 plus 0.1 mod alpha divided by 0.3. So, this is the inverse of the norm a alpha and the second rho gives you 1.0 plus 0.1 mod alpha divided by 0.15 alpha. So, the maximum of the 2 number is going to be the first number which you can simplify and get 50 upon 3 mod alpha plus 2 by 3. So, we have got norm a alpha to be maximum of the 2 numbers norm a alpha inverse is given by this quantity case 1 will be 0.2 mod alpha will be less than or equal to 3.5 that means mod alpha will be less than or equal to 17.5 when I take the maximum of the 2 now the maximum is 3.5. So, it will be 3.5 times this. So, this is going to have minimum to be equal to 17.5. So, you will have minimum of norm a alpha norm a alpha inverse to be 17.5 and in the second case also you are going to get exactly the same value. So, now in the next lecture we are going to start new topic and that is the approximate solution of initial value problem. So, thank you.