 Hello. So, let us continue with these orthogonal systems of polynomials and we will round up this capsule with these discussions. So, last time we were discussing some interesting exercises on the Shebyshev polynomials Tnx which is basically cos of n cos inverse of x. The next exercise we are going to discuss is a very amusing problem in elementary calculus. So, what are the problems say that if you take the cos theta as a independent variable the dd cos theta applied n minus 1 times to sin to the power 2n minus 1 theta. That has a nice closed expression minus 1 to the power n 1 into 3 into 5 into 2n minus 1 upon n sin n theta. This formula goes back to Jacobi in 1836 and you can look at Watson's monumental treatise on the theory of Bessel's functions to understand its immense use in the theory of special functions. So, the idea is to put T equal to cos theta and look at what happens when you differentiate 1 minus T square to the power n minus half n minus 1 times. This expression on the left hand side will simply become ddt to the power n minus 1 1 minus T square to the power n minus half. This satisfies the Shebyshev's OD. So, LHS is basically the Shebyshev's ODs and so, the LHS is basically going to be f of T equal to cn sin n cos inverse of T and then you have to determine the cn. So, some hints have been given to you on that. So, you may work on that. This is more an amusing piece of exercise. Now, let us say a few words about orthogonal polynomials in general. We see certain pattern emerging in our discussions. The Legendre polynomials, the Shebyshev's polynomials, the Hermite polynomials, the Laguerre polynomials, they are the four important classical orthogonal systems of polynomials. They all have certain features in common that the zeros are real, they are distinct and they lie in the interval of orthogonality. What is the interval of orthogonality for the Legendre polynomial and Shebyshev's polynomials? It is the interval minus 1 1. So, all the zeros lie in the open interval minus 1 1 and the zeros are real and distinct. For the Hermite polynomials, the interval of orthogonality is the real line and the measure is e to the power minus x squared dx. The measure for the Legendre polynomials is simply the Lebesgue measure dx. For the Shebyshev's polynomial, the interval of orthogonality is the same as that of the Legendre, but the measure is different. For the Shebyshev's polynomial, the measure is dx upon root of 1 minus x squared. For the Laguerre polynomial, the interval of orthogonality is the open interval 0 infinity and the measure is e to the power minus x dx. So, all of them have a three-term recursion formula. For all of them, the zeros of fn and fn plus 1 interlace. We approved this for some of the orthogonal systems, not for all of them. And there is an analog of the Roderick's formula for all of them. And all of them arise when you apply the Gram-Schmidt's process to 1 x x squared, etc. with respect to the appropriate measure. So, is there some general theory behind it? So, you could take a general orthogonal systems of polynomial with respect to a general measure. What is the situation? They will have a three-term recursion formula. The zeros will interlace and the zeros will be real distinct and in the fundamental interval of orthogonality and so on. We are talking about real polynomials with real measures. For a general discussion, I would recommend you the delightful little book of J. Todd, Introduction to the Constructive Theory of Functions, Berkhausen 1963. It is an excellent place to begin a study of approximation theory. The Shebyshev's polynomials are extremely important when you study best approximations in the SUP norm. When you study best approximations in L2 norms, the Legendre polynomials will be the correct ones to look into. And the book by Ion Sneddon, Special Functions of Mathematical Physics and Chemistry, that also contains an instructive list of problems on Legendre polynomials, particularly the problems on page 96 to 105. So, with these general references, I will go to the next item in this chapter, Abstract Fourier Analysis in Hilbert spaces. Again, now we are starting with an abstract Hilbert space H. So far, we have been looking at examples. We have been looking at the Bergman space. We have been looking at L2 of minus 11. We have been looking at L2 of the real line. We have been looking at the Hermite functions and so on. Those are concrete examples. Now we should go to the abstract theory further. We should push the abstract theory a bit further and we start with the Hilbert space and we take a orthonormal basis B, which means that B consists of pairwise orthonormal vectors and linear span of B is dense in the whole space. We are not assuming here that B is countable. Hilbert spaces with uncountable orthonormal basis are huge objects, but these do appear in some applications in Fourier analysis and differential equations, such as the theory of almost periodic functions. For example, a very nice reference to this is the book by Rhys-Norge Functional Analysis, Dewar Reprint 1990. But let us look into it in this generality. So for most spaces that we shall be dealing with, the orthonormal basis will be countable. Now suppose if V is an arbitrary vector in the Hilbert space H, then you take inner product of V with V alpha. V alpha is the elements of my orthonormal basis and the scalar x alpha is the alpha-th Fourier coefficient of V with respect to the orthonormal basis B. Now we would like to say that the vector V is a infinite sum summation x alpha V alpha alpha belongs to lambda. But what is the interpretation of this series? As it stands, the number of terms is not even countable. So how do we make sense of uncountable sums? Now first a lemma. Suppose for example we have got a family of non-negative real numbers, a alpha. And we are looking at summation a alpha, alpha belongs to lambda, 7.20. How do we make sense out of an uncountable sum of non-negative real numbers? The way to do that would be take the set of all finite sums a alpha 1 plus a alpha 2 plus a alpha n and you take the supremum of all these finite sums. So pick of any finite set of numbers, arbitrary finite subset of numbers from a given basket a alpha 1, a alpha 2, a alpha n, add them up. And the supremum of all these real numbers that we get, if it is finite we would call it the sum of this series summation a alpha, alpha belongs to lambda. But now we have a lemma. Suppose this supremum is finite. So I take all these finite sums and I take the supremum and I get a finite real number. Then what happens is that all but countably many of the summands must be zero. In fact if this, if you get a finite expression for here, then all the a alphas except countably many must vanish. That is a very important and interesting conclusion. Suppose not. Suppose that for each n, suppose that for each n we let E n to be all those a alphas which exceed one upon n. So form this set E n. We are assuming what? We are assuming that uncountably many of these a alphas are non-zero. In particular what can you say about this set E n? This E n we know must be finite because if any, even if one of the E n's is infinite, then I could take finite subsets of these elements in E n, add them up and I get one upon n plus one upon n plus one upon n, etc. And that will keep growing and the supremum cannot possibly be finite. So supremum is finite. These E n's must be finite. The union of all these sets E n. What are the union of all these sets E n's? This is going to be precisely the non-zero members a alpha because if a member a alpha is non-zero, it must exceed one upon n for some n and that a alpha must be an appropriate E n. So union of all these E n's is going to be precisely the non-zero elements but each E n is finite and the collection of E n's is countable. So we conclude that only countably many of the elements a alpha can be non-zero or all but countably many of them must be zero. Now we come to Hilbert spaces. We prove the following theorem. Suppose we got a Hilbert space H and we got an element V in H and x alpha are the Fourier coefficients of V with respect to an orthonormal basis B. Remember that the basis is orthonormal. That's important. Then what happens is summation mod x alpha squared alpha belongs to lambda is finite. As a consequence, all but countably many of the x alphas must vanish. So the series x alpha V alpha belongs to lambda is effectively series with countably many terms because all but countably many of the x coefficients are zero. And series converges in H in the usual sense and the sum is exactly V. So let's prove the theorem. So let alpha 1 alpha 2 alpha n be a finite set of indices in lambda. And what I do is a subtract of V minus x alpha 1 V alpha 1 plus x alpha 2 V alpha 2 plus x alpha n V alpha n. Take the dot product of this difference with V alpha 1. What is going to happen? V dot product with V alpha 1 that is going to be x alpha 1. Here I am going to get x alpha 1 into V alpha 1 dot product with V alpha 1. But that is one. The x alpha 1 cancels out. But V alpha 1 dot product with V alpha 2 is zero. V alpha 1 dot product with V alpha n is zero. So what do we get? This vector, this difference here is orthogonal to V alpha 1. Similarly, it is going to be orthogonal to V alpha 2. It is going to be orthogonal to V alpha n. And so it is orthogonal to the linear combination. But now apply Pythagoras's theorem. So this difference is orthogonal to this. So by Pythagoras's theorem, the norm square of this plus norm square of this is the norm square of the sum namely norm V square. What I do is that I knock off this term. I knock off this term and get norm of x alpha 1 V alpha 1 plus dot dot dot x alpha n V alpha n squared less than or equal to norm V square. But expand this and you will use orthonormality and you will get that mod x alpha 1 squared plus mod x alpha 2 squared plus da da da plus mod x alpha n squared will be less than or equal to mod V squared. So if you take a finite set of indices alpha 1 alpha to alpha n and I form these finite sums, all these finite sums all have a fixed upper bound norm V squared. So the supremum of all these things as I take all possible finite indices will be finite. In other words, we conclude that all but countably many Fourier coefficients must vanish. And so the sum is effectively an countable summation. And to prove that the series converges in H, we simply observe that the series of real numbers converges and these are series of positive real numbers. And so the partial sums will form a Cauchy sequence. And so now we show that the partial sums of the original sequence is also Cauchy. And that will follow quickly because when you take the norm of x alpha n V alpha n plus da da da plus x alpha n plus k V alpha n plus k squared, I simply get mod x alpha n squared plus da da da plus x alpha n plus k squared. Since the right hand side is a Cauchy sequence, we conclude that the partial sums of 7.22 also is a Cauchy sequence and we are in the Hilbert space and the job is done. Now we have to show that the series converges to V. Here we use completeness. This is where the completeness is going to be used for the last part. So let epsilon greater than 0 be arbitrary. Then there are scalars y 1, y 2, y alpha n such that y alpha n V alpha n plus da da da plus y alpha n V alpha n approximates V in norm by less than epsilon by root 2. And this n can be assumed to be as large as we want by throwing more terms with 0 coefficients and I can take this n to be terribly large. Now select n in such a way that mod x alpha n plus 1 squared plus mod x alpha n plus 2 squared plus da da da plus mod x alpha n plus 3 squared is less than epsilon squared by 2. Then what happens is that we observe that this vector V minus the first capital N terms is orthogonal to all these individual V alpha g's as before and so this difference will be orthogonal to a linear combination of these namely x alpha 1 minus y alpha P alpha 1 plus da da da x alpha n minus y alpha n V alpha n. So we got that this now we will apply the Pythagoras theorem. We will apply Pythagoras theorem we will see the norm of V minus y alpha 1 V alpha 1 plus da da da plus y alpha n V alpha n squared which is V minus x alpha 1 V alpha 1 plus da da da x alpha n V alpha n squared plus this other expression. Now this particular thing is already less than epsilon squared. So this expression here must be less than epsilon squared. If little n is larger than capital N then again I apply Pythagoras theorem and I get norm of V minus x alpha 1 V alpha 1 plus da da da x alpha n V alpha n the whole squared is equal to norm V minus the thing up to capital N V minus x alpha 1 V alpha 1 plus da da da x alpha capital N V alpha capital N squared plus this little tail that is left over but this is less than epsilon squared by 2 and this is less than epsilon squared by 2. The whole thing is less than epsilon squared and the proof is complete. And so the theorem has been completely established that is theorem 79. So if you have a complete orthogonal system even in a Hilbert space which is not separable. So the basis may not be countable and yet even you take a vector V and you take the Fourier coefficients x alpha the summation mod x alpha squared will converge that means that all but countably many Fourier coefficients are 0 and the Fourier series will converge to the given vector V. Now let us look at separable Hilbert space. Recall that a metric space is separable when the metric space has a countable dense sets and a separable Hilbert space is a Hilbert space with a countable dense set D1 after all a Hilbert space is also a metric space. So Hilbert space is separable if it is separable as a metric space. So pick a countable dense set D1. Now D1 may not be linearly independent. It could jolly will be linearly dependent but you know in a vector space if it happens to be linearly dependent set D1 happens to be linearly dependent then I can throw away some of the vectors and I can get a smaller set which is linearly independent call it D2 but linear span of D2 is the same as a linear span of D1 that can always be arranged not a problem. Obviously D1 which is contained in linear span of D1 but so D1 is contained in the linear span of D2 but since D1 is dense linear span of D2 is also dense that is a very trivial observation. Now what we do is we subject D2 to Gram-Schmidt's process remember D2 is linearly independent. So when you take a linearly independent set and apply the Gram-Schmidt you are not going to get a 0 vector along the way. So it is going to be a orthonormal set of vectors. So we apply the Gram-Schmidt and we get say D3. D3 is orthonormal linear span of D3 by definition will be equal to the linear span of D2 but linear span of D2 is dense in H. So linear span of D3 is also dense in H. So the separable Hilbert space has a countable orthonormal basis why is it countable because D1 was countable so D2 was countable because D2 was countable D3 is also countable. A separable Hilbert space admits a countable orthonormal basis. Classic examples of separable Hilbert spaces are L2 of the closed intervals AB that is a separable Hilbert space. How do I know that this Hilbert space is separable? Good question. Well C of AB closed interval AB is dense in L2 of the closed interval AB Luzin's theorem. But C of closed interval AB has a countable dense set namely polynomials with rational coefficients. Here we are using the Warster's approximation theorem but Warster's approximation theorem concerns approximation in sup norm. So if you can have a sequence of polynomials with rational coefficients converging to F in sup norm it will certainly converge in L2 norm as well. So these polynomials with rational coefficients will serve as a countable dense set in L2. Why is L2 of the real line separable? There are several reasons you can give. One simple answer is the Hermite functions is a countable orthonormal system which is complete that is if its linear span is dense. So there is one reason why L2 of r is separable. You could try to give a direct explanation why L2 of r is separable without any knowledge of Hermite functions. That is an exercise for you. How does one produce orthonormal basis for Hilbert spaces naturally? We want a natural method for finding orthonormal basis. It is useful to recall the case of a finite dimensional Hilbert space. After all a finite dimensional Hilbert space over the real numbers is Rn and over the complex numbers it is Cn. So let us work with the real scalars. So the usual dot product. The standard basis of course provides an orthonormal basis but may not be suitable for analyzing a specific problem. For example, let us take the problem. Identify the quadric 2xy plus 2yz plus 2zx equal to 1 in R3. Is it an ellipsoid? Is it a hyperboloid of 1 sheet? Is it a hyperboloid of 2 sheets? One way to go about it is to write this equation of the quadric as X transpose Ax equal to 1 where A is a 3 by 3 real symmetric matrix. What is this 3 by 3 real symmetric matrix? It is displayed in the slide 01110110. And you can appeal to the spectral theorem which you already proved by the way in this course saying that it has a orthonormal basis of eigenvectors V1, V2, V3. So V1, V2, V3 are unit vectors and they are mutually orthogonal and they will be written as columns and as string them together I get a matrix P which is a orthogonal matrix. P inverse AP is a diagonal matrix D. In other words, the appropriate orthonormal basis in this particular problem is not standard basis even E2, E3 but rather it is V1, V2, V3 namely the eigen basis of A. And now you write X equal to Py and the the quadric transforms to Y transpose dy equal to 1 but this new equation is simply lambda 1 X1 squared plus lambda 2 X2 squared plus lambda 3 X3 squared equal to 1. We clearly see the role played by the spectral theorem. So now we want to use the spectral theorem to produce interesting orthonormal basis for our standard Hilbert spaces. So if you take L2 of the real line, if you take L2 of R instead of some general real symmetric matrix what would you do? What are the natural operator that we have studied on L2 of the real line in the context of Fourier analysis? It is a Fourier transform. The Fourier transform is a very nice example of a operator on L2. It is a unitary operator. It is a normal operator. When is a matrix said to be normal? A matrix is said to be normal if A A star equal to A star A. A self adjoint matrix is normal. A unitary matrix is normal. The spectral theorem is true for normal operators. So here the Fourier transform is a normal operator and the Hermite functions are the eigen functions of the Fourier transform and they provided you the complete orthogonal system. So it is the generalization of this result from classical Fourier analysis that we want to be looking at. We will stop this capsule here and we shall continue this next time. Thank you very much.