 So, yesterday we have seen how in view of this inner product spaces and this best approximation of a vector, we can summarize the story in so far as solving for the system of equations given by Ax is equal to b is concerned particularly when this a is either a real or a complex matrix. So, that story has been completed. Just before we wind up with this aspect of inner products, maybe I will just take an example of an application if you call it that something that you are all probably familiar with. So, it is the case of Fourier series. So, does anybody recall what sort of signals or waveforms come under this consideration when we talk about Fourier series periodic waveforms right. So, what does this Fourier series do? What do you think it is? So, if I have a signal like so right, what is the Fourier series allow me to do? Any arbitrary sine and cosine? What sort of frequencies? So, what is the frequency of this waveform? This is not a sinusoid or a cosinusoid, but it is a periodic waveform. What is the period of this? So, what is the frequency? The angular frequency 2 pi by t right. So, 2 pi by t. So, what kind of sinusoids and cosinusoids make their appearance when you write this as a Fourier series integral multiples of this frequency yeah. So, have you ever given it a thought as to what is being done? Because in theory this Fourier series is an infinite series is it not? You just figure out that there is a formula subject to this Dirichlet's conditions and all that you have to check and then you have to calculate the coefficients and there is a formula for that calculation. That is probably the way you have seen it. Some integrations and stuff that you have to do right. So, now consider for a moment that instead of of course, it is practically not possible for you to have an infinite sum go into something. Suppose you truncated this infinite series after a certain number of terms, what kind of a waveform would you end up with? So, maybe something such as I do not know I am just taking a guess of something sort of this maybe, but of course not exactly equal for equaling it in exactitude you would need all those infinite terms. Now, the point is if I take more and more number of terms is it true that I will get closer and closer to the real waveform? Why when I say closer that brings us to this idea of what do you even mean by two signals being closer to one another? What is the difference of two signals that you know which when becoming become small allows us to infer that the two signals are closed close enough to one another? In this case very intuitively speaking if you can look at this area do not you think you should be trying to minimize this area the smaller is the area the closer is your approximation in terms of those sinusoids to this actual waveform suppose you say you been told that you are allowed only up to a certain number of terms say 20 terms that is what your processor can handle the point is why should we choose Fourier series to approximate the periodic waveform why not something else if you thought about it what is so desirable or attractive over this Fourier series exactly it turns out that it will give you the indeed the best possible approximation. So, in other words if you take all the terms in the Fourier series you will get exactly that periodic waveform. So, all periodic waveforms if you take the class of all periodic waveforms they actually form an infinite dimensional vector space. So, any periodic signal that you look at comes from an infinite dimensional vector space, but for very practical reasons you cannot deal with objects in infinite dimensional vector space in your practical calculations and computations. So, you can only handle a certain number of finite number of terms right if you want to do that you want to truncate the series at a point the number of terms that you can handle. If you want to do it in a best manner in a most efficient and optimal manner so as to minimize the error between the signal you are approximating yeah then it is it turns out that Fourier series is the best, but for that we have to of course give it some structure of an inner product and norm and things such as those. So, here is what I put to you that if you have a signal f of t which you want to approximate right. So, f t is a periodic signal right so periodic with period t yeah now let us go ahead and define the inner product in the space of these periodic signals as 0 to t f g bar dt what is g bar complex conjugate right. So, if that is how I define my inner product right so f and g are two periodic waveforms of signals with period t then this is how I define the inner product and thereby endowing this vector space with an inner product structure it now becomes an inner product space right. Now once I have this inner product I can of course bring in all these ideas of best approximation and what have you. So, what is the best approximation over a finite dimensional subspace of a infinite dimensional vector space recall the idea that we developed look at this finite dimensional subspace since it is finite dimensional it is span by a finite basis let us choose an orthonormal basis if you have an orthonormal basis for this subspace then all that you need to do is take any signal or take any vector from this infinite dimensional vector space take its inner products with these individual elements of that orthonormal basis yeah and those will be the scaling factors and those times the basis vectors themselves that is the best approximation right. So, the only thing that you need to basically verify is that this indeed satisfies the properties of an inner product which I leave to you as an exercise to do right anyway I would advise that you after this you know whatever we studying here you just go back to your undergraduate preliminary undergraduate textbooks where you first encountered Fourier series and reread that portion to see the you know see this in a new light. But anyway the point is that we have to now establish that what sort of a basis are we looking for and the claim is that this basis as it stands in the terms of this Fourier series those elements are indeed going to form an orthogonal basis what do I mean by that so in case of a Fourier series let us just say that you choose these functions to be e to the i where i is of course the square root of minus 1 okay i is equal to square root minus 1 in this case 2 pi by t k okay right. Now let us choose two different values of k and pass it through this test and see what happens so if you take e to the so if you take your f to be e to the i 2 pi k 1 t sorry there is a t missing very very important k 1 t upon big t and g to be e to the i 2 pi k 2 t upon t then what is the inner product f and g so of course if you take the conjugation of this you just have to replace i with minus i the imaginary parts sign gets flipped right so this then turns out to be yeah 0 to t e to the i 2 pi upon t into k 1 minus k 2 times t dt what is this going to be conica delta in terms of k 1 and k 2 so unless k 1 is equal to k 2 this is 0 right you can verify this so this is equal to 0 unless k 1 is equal to k 2 which means that indeed these kinds of functions for different different values of k if you choose them they turn out to be orthogonal to one another in the sense of this inner product that we have defined over the space of periodic signals right which means what should we do given any periodic signal if you want to approximate it up to a first say l number of terms what should we do we should take inner products naturally with the first l terms in this particular basis the basis that contains terms such as e to the i 2 pi k t upon big t where k ranges from 0 1 2 3 4 so on till l minus 1 right so when you are truncating a Fourier series at a point what you are doing is you are actually getting a best possible approximation thereof right so this if you allow it to expand up to l minus 1 you actually got the orthogonal basis in the space of periodic signals which contain harmonics up to the l minus first multiple of the frequency frequency of course determined by big t so every time you truncate a Fourier series that means this is the good thing about a Fourier series once you have obtained the generic term for a Fourier series if someone tells you I want the Fourier series up to the 10th term you just go ahead and evaluate c 0 c 1 c 2 till c 10 and there is a general formula for that if someone says I want till 20th term you just go ahead from c 0 to c 19 and those are just the numbers that you need right once you scale your waveforms corresponding waveforms by those numbers you are done what I mean by that is now if you want your f hat which is an approximation of the best possible approximation of f up to let us say the l th term you can just go ahead and say that it is going to be nothing but well the norm of the norm of so I will first have to evaluate of course the norm of this object which is already here if you just plug in k 1 is equal to k 2 then it is basically the norm of this fellow what is the norm of this fellow its square root of t is it not because this is the if you take k 1 is equal to k 2 that is the inner product of a member of this basis set with itself and that is the square of the norm that turns out to be equal to t so let me also complete this and equal to sorry equal to t for k 1 is equal to k 2 so if it is equal to t for k 1 is equal to k 2 that means the norm is square root of t yeah so I can just scale it up and down suitably by instead of this I am going to just write square root of t and what is above this is going to be just f the original function living in the infinite dimensional vector space it is inner product with e to the minus sorry not minus i 2 pi k 1 capital t into small t times so this is of course the component of f along the kth member of that basis but this must be multiplied along the basis it is a scaling along the basis and the basis has to be normalized as well yeah so this is like the norm of this so you are taking the ab cos theta is the dot product and they are dividing by b so you get a cos theta along the direction of b right so what you have here next is e to the i 2 pi k t 1 capital t and another square root of t the sum from k is equal to 0 up to l or rather l minus 1 if it is l terms that you want then it is 0 to l minus 1 so in a nutshell this is nothing but the inner product of f with e to the i 2 pi k t upon capital t divided by t k going from 0 to l minus 1 times e to the i 2 pi k t on capital t so that is the best possible approximation of any periodic waveform up to that many terms chosen by l yeah this clear so that is an application of the idea of so that is why the idea of Fourier series is so appealing yeah because it automatically gives you an orthogonal basis to expand and that is why when you take these inner products you are actually doing a best possible approximation at any point you truncate you could not have done any better by choosing that many terms right the space span by those first l members of that basis that is the best approximation there of right yes inner product so so if you just think about it if you now try to find out the difference between like you know that region you remember I have now erased it that region that I have sketched between them so what do you think it is going to be how are you going to find out that region because you do not want to penalize positive and a negative area differently both are errors so you have to square it somehow so you are interested in the magnitude of the whole thing so why not just square the difference and if you want to do that then this is going to be your idea of a norm and it so happens that this norm is induced by an inner product like this so this is my go to choice for the inner product yes yes of course when when the inner product is defined in this fashion which automatically means that your norm so there yeah you are right so when the norm is of a different nature when our idea of what it means by two signals being separated from one another yeah you could you could some people could come up with a norm that it says that I want to minimize the maximum difference between two signals at any point that is also a different norm but that norm is not necessarily induced by some inner product that becomes a difficult problem yeah the elegance of this of this format of this structure is that it fits in because this norm is induced exactly by an inner product and it also fits in with our notion of what it means to get closer so of course somebody might say that I want to design for the worst case so at any point what is the maximum what is the difference between the signal that I am approximating and my approximating signal if I want to minimize the maximum difference at any point that is also one yeah so there are different different types of norms that you can come up with depending on the space that you have in mind but the norm that comes from the inner product automatically gives us this idea of inner product and the orthogonality and thereby allowing us to extend any basis in a finite dimensional subspace to an orthogonal basis and the Fourier series readily gives us one such orthogonal basis thereby making our job of approximation simpler I mean you can find any linearly independent set that is itself a basis once you have a linearly independent set depending on the inner product that you define you can just apply the Gram-Schmidt orthonormalize you see that Gram-Schmidt is not for just Euclidean spaces right it's generally the moment you have an inner product you can go ahead and apply Gram-Schmidt to any basis set for a finite dimensional vector space and get an orthogonal set of orthogonal basis for that same finite dimensional vector space right so in fact you might see often in some exercise problem that will give you a basis for a polynomials for a for a vector space of polynomials of say a degree some 10 or 5 or whatever that accompanied by some inner product might lead you to conclude that this basis is independent but not orthogonal nonetheless and we might say apply Gram-Schmidt to this subject to that inner product and you have to orthogonalize the basis and then you can get a best approximation readily yeah any other questions