 In numerical analysis, we are mainly interested in implementation and analysis of numerical algorithms for finding an approximate solution to a mathematical problem. The solution in general is approximate because we use computer and when you are using computer you have got only finite precision and you have only finite number of steps, you perform only finite number of steps. Also the method which you are using that itself may be iterative method or even if it is a direct method, then because of the finite precision there is going to be an approximate solution. So, when one looks at the sources of error which are in the numerical analysis or in numerical algorithms, then they can be because of the uncertainty in the data or because it will be rounding of errors that is because of the finite precision or there can be discretization error. So, we are interested in this analysis and implementation of numerical algorithms and main idea is using a computer when finds an approximate solution and these are the sources of data, sources of error. So, let us look at first the rounding of error. You have a number, let us look at real number, it has this representation to the base beta, the base beta is either 2 or 10 or 16. So, these are the generally values of beta which are used on computer. So, real number x has this it is of this form where as a normalization one assumes that d 1 is not equal to 0. It is always possible to do such a thing for a non-zero number because you can adjust the exponent. Now, floating point representation of this is as I said we have only finite number of digits available on computer. So, this is the basis of the data. So, you look at value of d n plus 1 when the base is beta all these digits they are going to lie between 0 to beta they are going to be strictly less than beta. So, if d n plus 1 if it is less than beta by 2 then one writes this as point d 1 d 2 d n into beta raise to e or if d n plus 1 lies between beta by 2 and beta then one increases the value of d n to d n plus 1. So, this is when you are using rounding of. Now, it is easy to see that x is our number floating point of x that is the number of floating point representation that is going to be less than 1 by 2 into beta raise to e minus n. Now, this is absolute error the more relevant error is to consider relative error that is modulus of x minus floating point of x divided by mod x this will lie between or this will be less than or equal to 1 by 2 into beta raise to minus n plus 1 this same thing one can represent as floating point of x is equal to x into 1 plus delta and modulus of delta is going to be less than or equal to 1 by 2 into beta raise to minus n plus 1. So, this is there is a error x was our real number when you represent it on computer you are going to consider floating point representation and then you introduce a error that relative error is less than or equal to 1 by 2 beta raise to 1 minus n, but there is a error and this error you are going to perform various operations. So, it will keep accumulating now instead of rounding off on some computers what is done is chopping off. So, the chopping is this is our number. So, irrespective of value of d n plus 1 d n plus 1 will be such that 0 less than or equal to d n plus 1 less than beta. So, you consider floating point of x to be just forget this d n plus 1 onwards. So, it is plus or minus that is the sign point d 1 d 2 d n beta raise to e and in this case the relative error is less than or equal to beta raise to 1 minus n when it is rounded off it was 1 by 2 beta raise to 1 minus n. So, they are similar. So, in this case floating point of x is going to be x into 1 plus delta this delta is going to be always negative in this case and it will be bigger than minus beta raise to 1 minus n and less than or equal to 0. So, this is about the finite precision that on a computer you can represent a number by using only finite number of digits. Another source of error that is the discretization error when one defines a mathematical year quantity. For example, when one considers integral a to be f x d x if f is a continuous then one defines it as limit of Riemann sums. So, when one considers limit of Riemann sums it is limit as n tends to infinity where n is number of intervals sub intervals of our interval a b. Now, mathematically this definition is fine, but when I want to calculate integral a to b f x d x using a computer then I can have only finite number of steps at my disposal and then that will introduce discretization error. So, when you replace infinite process by a finite process which you have to do when you are using computer that introduces the discretization error. So, you have f to be a continuous function say defined on interval 0 to 1 all continuous functions are integrable you subdivide the interval 0 to 1 into n equal parts then integral 0 to 1 f x d x is limit n tending to infinity summation i goes from 1 to n f of i by n into 1 by n value of our function f at nodes 1 by n 2 by n and 1 multiplied by length of the sub interval that is 1 by n. Now, when you are not using computer then for some special functions one can evaluate this limit, but what we want is given a given function our computer should tell us what is integral 0 to 1 f x d. So, this limit will be approximately equal to summation i going from 1 to n f of i by n into 1 by n 1 chooses n to be big enough. So, that will depend on how big accuracy or how accurate the solution you want. So, this is going to be the discretization error. So, these are the two main sources of error you have rounding of or truncation error and you have discretization error. Now, a numerical algorithm that is nothing, but it is a complete and unambiguous set of procedures which gives an approximate solution to a mathematical problem. Now, there are various mathematical problems one problem was finding integral a to b f x d x or it can be finding the derivative value or it can be solution of a differential equation and so on. So, numerical algorithm it tells us that how I should proceed. So, I have a set of instructions and that gives us a method for finding an approximate solution. Now, once you choose an algorithm what you have to do is you have to decide the accuracy how much accuracy you want. Then you have to decide the number of iterations because generally your method is going to be a iterative method or the finite number of steps. Then some bounding of the error like there is going to be some error. So, whether for a given problem I can know beforehand that my error modulus of error is going to be less than or equal to something or also we have to keep track of rounding of errors that you are performing each number is going to be represented only using finite number of digits and then you do various operations. So, the operations they are going to be addition, multiplication, subtraction and division. These are going to be the four operations which you will be doing. So, then the rounding of error that will go on adding. So, one has to keep track of that. Then also in case the method is not converging then one has to keep some way out that you perform these many number of iterations if it does not work then think of something else. So, here are the main points that you decide the accuracy which is needed. Then magnitude of round off errors and discretization errors you have to find an upper bound. The number of iterates you fix then check on accuracy and then what to do if there is no convergence. So, algorithm will be a set of procedures and then these are the things one decides or these are the things which need to be taken care of. Now, there can be various methods to solve a problem. So, now criteria for a good method that is going to be number of computations. So, how many additions or subtractions and how many multiplications or division one needs to do. When one calculates the number of computations addition and subtraction they are considered on par and multiplication or division they are considered on par. Our method it should be applicable to a large class of problems. It cannot be specific just to a very narrow class of problem. So, applicability then how fast there is going to be convergence. Like if you have two methods then you consider the speed of convergence of the two methods. Of course, the faster method will be preferred, but then generally there is a tradeoff that if the method converges fast then you are doing more work. So, one has to look at the balance then about the error management that you have got round off error, you have got discretization error. So, then for a particular method whether one can have some idea about the error and then the fifth point is stability. So, about the stability I will say little later. So, these are the criteria for a good method. Now, in this course the topics which we are going to consider they are going to be first we will look at polynomial and piecewise polynomial interpolation. The topics which follow they are going to depend on this polynomial interpolation. So, we will first consider polynomial approximation. Then the idea is you are going to replace a function a continuous function by a polynomial. And the polynomials one can integrate one can differentiate using computer. So, if you have a function f if you want to find its integral replace it by a nice function such as polynomial and that will give rise to numerical integration. Similar idea will be used for numerical differentiation. Then we will also be considering approximate solution of system of linear equations and associated eigenvalue problem. Next we will consider the root finding method for a nonlinear equation. And the last topic will be the solution of boundary value problems and initial value problem. So, here is polynomial and piecewise polynomial interpolation the first topic based on that you will have numerical integration and differentiation. Then we will consider the direct methods and iterative methods for solution of system of linear equations. And for eigenvalue problem they are going to be necessarily iterative methods. We will have approximate methods for finding the 0 of a function and there again we will be using polynomial interpolation. And lastly the differential equations. So, in that we will be considering initial value problem and boundary value problem. The lectures will be based on mainly material from the following two reference books. So, the first one is book by Konte and Karl Dabur. So, it is elementary numerical analysis and their emphasis is on algorithms it is available in the international editions. And a book by K.E. Atkinson introduction to numerical analysis which is published by John Wiley in 1989. So, now we are going to look at some mathematical preliminaries. See now there are many packages available for finding solution of various problems. The reason why one studies numerical analysis is that you should be able to write your own algorithms or even if you are using a standard package you should be aware of its limitation that you are going to use computer. So, you are going to get approximate solution. So, how reliable that approximate solution is? You should be able to write a adapt an algorithm which is specific to your problem that an algorithm may be available for a general class of problem. But for your special problem may be you can make some changes and you can have a better method. So, that is why one is interested in analysis of various algorithms. Now, this analysis is done by using of course results from mathematics. So, I am going to review some of the basic results from mathematics. What is assumed is that you have familiarity with properties of real valued functions with real variables or complex valued functions and a bit of basics of linear algebra. So, the first result which I want to recall is about convergence of a sequence. So, we are going to consider a sequence of real numbers or complex numbers and I am just going to recall the definition. So, we have got x n it is a sequence of either real numbers or complex numbers then definition x n converges to x as n tends to infinity for x belonging to r or c if for every epsilon greater than 0. There exists a natural number n such that whenever n is bigger than or equal to capital N it implies that modulus of x n is greater than or equal to capital N minus x is less than epsilon. So, we have suppose x is a real number and you are dealing with sequence of real numbers then what we are saying is look at the interval x minus epsilon to x plus epsilon. So, after a certain stage all the elements x n they should be in this interval. If your x is complex then you are looking at a disc with center x and radius epsilon. So, your x n after a certain stage they should lie in this disc with center x and radius epsilon. So, that is the definition of convergence of sequence of real numbers. Now, next thing I want to do is I want to consider continuous function defined on interval a b closed interval a b taking real values and recall some of the basic properties of continuous function. So, our function f is defined on interval a b closed and bounded taking real numbers real values and it is a continuous function. In that case f is going to be bounded a range of f is going to be interval small m to capital M. So, our m is less than or equal to f x less than or equal to capital M for all x belonging to interval a b. Now, if I take alpha to be any number between capital M to small m then there exists c belonging to a b such that f of c is equal to alpha. In fact, when I write range of f is equal to closed interval m to capital M, it precisely means that any number which lies between small m and capital M that is going to be in the range. So, alpha lying in this interval is equal to f of c this is known as intermediate value theorem small m this is known as absolute minimum and capital M is known as absolute maximum. So, this is about continuous functions. Now, suppose your function f is differentiable then the basic theorems for differentiable functions they are Rolls theorem, Cauchy's mean value theorem and Taylor's theorem. So, the Rolls theorem is if you have f to be continuous on closed interval a b I am assuming f to be real valued differentiable on open interval a b and f of a is equal to f of b. If these three conditions are satisfied then the conclusion of the theorem is then there exists c belonging to a b such that f dash c is equal to 0. So, graphically what it means is suppose f of a is equal to f of b is equal to f of b. I have got a function defined on interval a b such that it is continuous on closed interval a b differentiable on open interval a b and f a is equal to f b. So, suppose the function graph of the function is something like this then there exists c belonging to a b such that f dash c is 0 f dash c is 0 that will mean that at point c the tangent is horizontal so here you have tangent to be horizontal. We are saying then there exists c there can be more than such c for example, here at this point also the tangent is going to be horizontal if I look at some different function say a function like this then I am going to have a unique c such that f dash c is equal to 0. So, it depends on your function, but what we know is there will be definitely one c such that f dash c is equal to 0. So, this is Rohl's theorem and now this Cauchy's mean value theorem it is very important and we will be using it several times during the course. So, for the Cauchy's theorem the conditions are again it should be continuous on closed interval a b differentiable on open interval a b then when I look at the quotient f b minus f a upon b minus a that will be equal to f dash c for some c in the interval a to b and later on we will see the applications or where one needs this Cauchy's theorem. So, we have Cauchy's mean value theorem. So, you have f to be continuous on closed interval a b differentiable on open interval a b then there exists a differentiable c belonging to open interval a b such that f b minus f a upon b minus a is equal to f dash c. If f a is equal to f b as in Rohl's theorem then f dash c will be 0 that was the conclusion of Rohl's theorem. So, now we are going to look at Taylor's theorem. So, mean value theorem will be a special case of Taylor's theorem. In mean value theorem we assume f to be once differentiable. Now, suppose f is n plus 1 times differentiable then we have got the Taylor's theorem and the statement of the Taylor's theorem is f from a b to R and c is a point in interval a b. Our assumption is function f its derivative f dash and its nth derivative they are all continuous. So, the first n derivatives they are continuous and f n is differentiable on open interval a b. So, if you put n is equal to 0 then that means f continuous and continuous on closed interval a b and differentiable on open interval a b which was the assumption in mean value theorem. If this happens then for all x belonging to a b one can write f of x is equal to f of c plus f dash c into x minus c plus nth derivative of f at c upon n factorial x minus c raise to n plus n plus first derivative evaluated at some point d x. This is going to be a point in the open interval a b divided by n plus 1 factorial into x minus c raise to n plus 1. Our point c is fixed it can be any point in the interval a b including the end points a and b. So, look at the nth derivative of f dash expression in blue c is fixed. So, it is f of c plus f dash c into x minus c plus f n c upon n factorial x minus c raise to n x can vary over interval a b. So, this is going to be a polynomial in x whereas, the expression in the red that is the remainder in the interval a b. So, here this f n plus 1 and this d point d it depends on x it also depends on c, but c is our fixed. So, f n plus 1 d x into x minus c raise to n plus 1 n plus 1 factorial. So, the dependence of an x is here as well as here. So, you are writing f x as a polynomial and then there is going to be some remainder. So, this is a very basic theorem and that is the Taylor's theorem. Now, another rule it is about the derivative of a composite function. So, suppose your function f is defined on interval a b and the range is contained in interval alpha to beta. The function g is defined on alpha to beta and taking real values. So, I can talk of g composed with f. If your function f is differentiable at c where c is some point in the derivative of f is some point in the interval a b and if g is differentiable at image of c by f, then the g composed with f its derivative at c will be given by g dash of f c into f dash of c. So, these were the results about the function defined on closed interval a b and the result is that we will be assuming generally the function to be continuous. Now, the next thing I want to review it is about the fundamental theorem of algebra. So, the theorem is you look at a polynomial p x which is a 0 plus a 1 x plus a n x raise to n. X can be either real or complex. The coefficients a 0, a 1, a n these also can be real or complex numbers. Then let us assume that a n is not equal to 0. That means, we have a polynomial of degree n. Then the fundamental theorem of algebra it tells that p x has at least 1 0. Now, even if a 0, a 1, a n they are real, still what we can guarantee is that it has a root with that root it may be complex. So, even for real polynomial the root can be complex. So, this is the fundamental theorem of algebra. Now, from this theorem it is easy to deduce that a polynomial of degree n it is going to have exactly n roots. Here we are saying that it has at least 1 0 and now we want to say that it has got exactly n 0. Now, those 0s they can be repeated. So, let me define what is a simple 0 and what is a multiple 0. So, the definition is you have a function f. So, if f of c is equal to 0 and f dash c not equal to 0. Then we say that f of f has a simple 0 at c. On the other hand if you have got f of c is equal to f dash c is equal to f m minus 1 c is equal to 0 and f m c is not equal to 0. Then we say that f has a 0 of multiplicity m at c. For example, look at sin x f x f x is equal to sin x f of 0 will be 0 f dash x is going to be equal to 0. So, f of x is equal to 0 f dash x is equal to cos x. So, f dash at 0 is going to be equal to 1. So, that means f has a simple 0 at 0. On the other hand if I look at f x is equal to x sin x. Then f of 0 is equal to f dash of 0 is equal to 0 and f double dash 0 is not equal to 0. So, this function f will have a double 0. So, now look at p n to be a polynomial of degree n. That means it is of the form a 0 plus a 1 x plus a n x raise to n a n not equal to 0. Then by the fundamental theorem of algebra it has 1 0 or it has a root. So, p n z 1 is equal to 0 for some z 1 belonging to c. Now, you can factorize. So, p n x is equal to x minus z 1 into q n minus 1 x. Now, this q n minus 1 it will be a polynomial of degree n minus 1. p n is a polynomial of degree n. We are writing it as x minus z 1 into something. So, that something has to be a polynomial of degree n minus 1. Now, q n minus 1 will have a 0. So, you apply fundamental theorem of algebra to q n minus 1 and then the 0 of q n minus 1 it can be either z 1 or it can be something n. So, using this argument you get p n x to be equal to alpha into x minus z 1 raise to m 1 into x minus z k raise to m k where the powers m 1 plus m 2 plus m k they will add up to n and alpha is going to be a constant. So, this is a factorization theorem and this tells you that p n has exactly n 0 provided you count z 1 as repeated m 1 times, z k as repeated m k times. So, z 1 z to z k are distinct 0s, but when you add the multiplicity that is going to be equal to n. So, once again this is a basic result which we are going to need later on. So, now we looked at some properties of continuous functions continuous and differentiable then fundamental theorem of algebra which tells us about the roots of a polynomial and now I want to recall the notation and basic things or basic properties about matrices. So, you were matrix A and let us consider the matrix with the entries either as real numbers or complex numbers. So, A is m by n matrix that means it is an array with m roots. So, you have got a 1 1, a 1 2 up to a 1 n then a 2 1, a 2 2, a 2 n and a m 1, a m 2, a m n. So, here a i j they belong to either real numbers or complex numbers or complex numbers. So, i denotes the row index, j denotes the column index. So, this is a m by n matrix if you have a vector which is say x 1, x 2 up to x n. So, it is a column vector m by n matrix and then this is our x then 1 defines A into x. So, this A into x it is going to be a m by 1 vector and the matrix into vector multiplication is the first entry A x 1 the component of A x its first component is given by a 1 1 x 1 plus a 1 n x n. In general, A x its i th component will be given by you have to look at i th row. So, it is a i 1 x 1 plus a i n x n and this we will be writing in a compact notation as summation j going from 1 to n A i j x j. So, we have defined a matrix into vector multiplication then we can consider the two matrices and their multiplication. If you have got two matrices of the same size m by n then you can add the addition is done by component wise you can multiply a matrix by a scalar. So, that is you multiply each entry by that scalar alpha if you are looking at matrices. So, one matrix of the order m by n another of n by p. So, the first one has m rows and n columns second one has n rows and p columns then you can multiply these two matrices. So, you have say A is equal to a i j B is equal to b i j both of the sides are m by n then addition is A plus B it will be a matrix C which is again of the same size m by n and then c i j is nothing but a i j plus b i j at the corresponding entries. If your alpha is either a real number or a complex number then alpha times A is going to be alpha times A i j you multiply each entry by alpha. So, if you have got two matrices of the same size you can add them or you can multiply by a scalar. Now, let us look at the multiplication of two matrices. So, if you have A to B a m by n matrix and B to B a n by p matrix. So, which is given by B i j then C which is C i j it is a multiplication of matrix A into matrix B where the size of C is given going to be given by m rows and p columns and the entries C i j they are given by m rows and p columns. So, this is given by summation a i k b k j k going from 1 to n a i k this k refers to the column of a and this k refers to the row of b. So, that is why you need A to B m by n and B to B a n by n if you are taking square matrices then there will be no problem if you have got A and B to be a n by n matrix then you can talk of A into B you can also talk of B into A. If you have got a general matrix you can talk of A into B, but not of B into A. So, if you have got say A B C to be square matrices. So, A B and C these are the square matrices. So, all of them of the same size n by n then I can consider A B into C this is going to be same as A into B C. So, this is known as associative property that so long as you respect the order you are going to get the same result here you are first multiplying A and B and then multiplying by C later on here you are multiplying B and C first and then pre multiplying by A then the result is going to be the same. If you look at A into B this is in general not equal to B into A. So, it is not commutative. So, this is a very important point that matrix multiplication is not commutative it is associative but it is not commutative. So, these were some of the basic results which we will be using throughout the course. Now, there will be some more results but what I am going to do is I am going to prove or I am going to quote when we need those. So, now what we want to do is suppose you are given a function f we want to approximate it by a nice function look at polynomial. So, the polynomial is of that form a 0 plus a 1 x plus a n x raise to n for some n. So, if I want to represent this polynomial in a computer what I will need to do is store the coefficients a 0 a 1 n that contains all the information about my polynomial. This is not the case with a general function. Suppose I am looking at a function defined on interval a b then I need to tell what value of f is at each point x belonging to interval a b there are infinitely many points. On the other hand the for a polynomial I store a 0 a 1 a n if I want to evaluate it at some point I know that it is I have to look at a 0 plus a 1 multiplied by x plus a 2 multiplied by x square plus a n multiplied by x raise to n. So, for storing the polynomials they are very useful in computer also those are infinitely many times differentiable and the derivative again it is a polynomial it is going to be a polynomial of 1 degree less. So, for the derivative once again I can use computer if I want to integrate then I will get again a polynomial with 1 degree higher and if you are looking at indefinite integral there will be some constant of integration. So, on a computer what you can do is addition subtraction multiplication. So, using these 3 numbers what you can have is you can represent a polynomial you can have also division then if you use division then what you can represent is the rational functions that means the quotient of 2 polynomials and you have one more thing available on computer that is the comparison. So, that will give us possibility of piecewise polynomial or piecewise rational function, but the polynomials they are smooth functions they are infinitely many times differentiable any continuous function can be approximated by a sequence of polynomials. So, that is why we will be considering approximation of a continuous function by polynomial. Now, what we are going to do is we are going to look at say various ways of approximating a function by a polynomial and one of that is going to be interpolating polynomial. So, this we will do in our next lecture. Thank you.