 A warm welcome to the 39th lecture on the subject of advanced digital signal processing, specifically multirate processing and wavelets. In the previous couple of lectures, we have looked at some very interesting expositions by students who looked at applications of wavelets and multirate digital signal processing. One of the successes and one of the joys in conveying a subject is to see how well it is reflected in the understanding of the student and more so if the student could actually challenge the teacher by giving an alternate proof or by rectifying an inaccurate statement and by providing a more accurate statement with a more accurate exposition. So among the audience of the course has been other than the students who presented some excellent application presentations in the previous couple of lectures. One student who pointed out an inaccuracy in the way in which we dealt with a very fundamental principle but you know perhaps not as accurately as it should have been. So he pointed out that although it was correct that a function cannot be compactly supported both in time and frequency, he also pointed out that the reasons for this had not been accurately explained and I was very happy to see that there was this alternate proof which he brought out which I shall now ask him to present in this lecture, the 39th. So I present to you Soham Bosu one of the students one of the audience of the course who then looked carefully at this issue of trying to be compactly supported in both domains time and frequency and came out with a much better explanation of the inability to have compactness in both the domains. I have asked Soham to present himself both the theme which he is going to talk about the concepts that underlie the proof and the proof itself. So here we have Soham Bosu to follow. Hello everyone, today we will be looking at a particular proof of a theorem based on the uncertainty principle in Fourier transform. So here is what we are going to look at proof that a non-zero function cannot be both time and band limited at the same time. Basically by this we mean that a function cannot have a compact support both in its time domain and its Fourier domain. So this is already a well known theorem some proofs of this exist, we will have overview of the existing proofs and also the new proof that I have found. So here is what we mean by time limitedness and band limited or compactness in general. So suppose we have a function f which we draw like this by the support of the function we mean the interval outside which the function is 0. For example if I take a rectangular pulse like this the support of the function is this say it is a to b and the support is just the interval a to b. On the other hand if I take the Gaussian function the support is a whole real line. So function is say to be limited or say to have a compact support when this support is finite in length for example for the rectangular pulse. So let us first look into the general background on which we will be working. So we will be considering Fourier transform which is one way of looking into a signal from its frequency domain by frequency we mean sine waves, sine waves define frequency. So over here we will be considering translates of sine waves and their linear combinations. So if we do it in the complex domain we can bring out exponentials because we can split up sine functions like this sine omega can be written as e to the power i omega minus e to the power minus i omega by 2. So if I want to write the complex exponential e to the power i theta I can write it as i sine theta plus cos theta but again cos theta is a translate of sine theta which is pi by 2 minus theta. So basically like all other transforms the general idea of Fourier transform is to project a function on to the space of e to the power i omega t or e to the power minus i omega t. The mathematical formula is given like this for function f the Fourier transform if it exists is given by f of s equal to f t into e to the power minus 2 pi i s t integrated over d t over the whole of the real line. The function f is taken over the whole of the real line the domain of the function. For the inversion of this formula we need some conditions on f s. So if the inversion holds for generally when a function f belongs to l 1 or l 2 r then the inversion formula holds and it is just given by a similar formula there is just a change in the sign in the dependent variable of the exponential part. So now we have come across l 1 and l 2 or in general we also consider the spaces l p for functions. So let us define that spaces first by a space l p if we say that a function belongs to l p r what we mean by that is the p th power of the modulus of the function when integrated over the whole real line say this is t is finite. For example the function which we took rectangular function so it goes from minus a by 2 to a by 2 this belongs to l p basically all l p's for p real numbers. On the other hand the function sinc x by sinc x we mean sin pi x by pi x. So if we draw it goes at x equal to 0 the limit goes to 1. So it goes something like this but this function does not belong to l 1 because if we take the modulus of this the areas become positive here the areas cancel out a bit and the final integral becomes pi by 2. On the other hand here this parts become positive and if we fit in triangles into this we can see that the area of the triangles diverge. So the area of modulus sinc x is also going to diverge. So in this new proof we will be extending we will be first considering functions which belong to classes l 1 and hence we will be extending the class of functions to general l p p from 1 to infinity. So there is a very general inequality which states that when a function belongs to l p p between 0 to p between 1 to infinity it also belongs to l 1. So mathematically if belonging to l p p belonging to 1 to infinity implies that if belonging to l 1. So for the time limited function this can very easily be proved with holders inequality. Holders inequality states that for two functions f and g this integral is less than equal to so here are the conditions for holders inequality to hold. The main point is 1 by p plus 1 by q has to be equal to 1 and of course p and q both positive from this we can get the range of p and q. So if we take p to be less than 1 then q becomes negative. So p can only go from p and q both can only go from 1 to infinity equal to infinity is also allowed if I allow 1 on one side. So here is the proof of this theorem this another inequality called Young's inequality which states the following fact given any two real numbers positive real numbers a and b a b is less than a to the power p by p plus b to the power q by q. This thing is very easy to prove if we use the generalized a m g m inequality of course the same conditions hold over here 1 by p plus 1 by q equal to 1 or otherwise p by q is equal to p q. So for the proof of this we will be using the generalized a m g m inequality. Here is the generalized inequality given two numbers a b and alpha 1 alpha 2 are the weights of a and b we have this result a and b are positive real numbers. So over here let us try to fit in the numbers which we have been given. Let us take alpha 1 to be q and a to be a to the power p alpha 2 to be p and b to b to be b to the power q below if we write p 1 p plus q we can as well write p q instead of this since we already know that 1 by p plus 1 by q is equal to 1. Similarly, on the other side we get p q and the p with root of a to the power p whole to the power q and b to the power q whole to the power p taking real value real values of p and q and as they are not 0 we can cancel this and what we get is a to the power p by p plus b to the power q by q on the left hand side. On the right hand side we get the p q with root of a b to the power p q this is greater than equal to. So finally, we have got the Young's inequality. Now say there are two functions f and g such that f belongs to l p and g belongs to l q with the same conditions 1 by p plus 1 by q equal to 1. So, from Young's inequality let us define redefine f as f is equal to f by the p th norm of f the p th norm is this over the whole of d t g is also redefined the same way g by q th norm of g. Now let us put in this f and g now point wise as numbers f t and g t in the Young's inequality. So, f t to the power p by p g t to the power q by q is greater than equal to f t g t f t and g t can be complex all functions also. So, we take numbers by defining the module. So, if we integrate this over the whole of the real line we basically get out whole less inequality because on one hand we are going to get 1 because is the weighted p th norm of f t already we get 1 by p on one side plus 1 by q on the left hand side which equals 1 and on the right hand side we have got mod f t into mod g t the integral where f t and g t are weighted. Now if you bring out the weights and multiply with 1 we get that the p th norm of f into the q th norm of g is greater than equal to the integral of the modulus. So, this basically proves whole less inequality how are we going to use this in our proof. So, say a function f is time limited can we show that if it belongs to l p class it belongs to l 1 class also yes we can by choosing a suitable g. So, say the function that we are interested in is non 0 on this particular interval c only compactly supported in c and 0 everywhere else. So, we can write down only on c because everywhere else the function is 0 the whole less inequality. Now we can choose g in such a manner that g is 1 only on the interval c and 0 everywhere else in that case on the LHS only mod f remains on the RHS there is the p th norm to the power 1 by p of f and g has been replaced by 1 on the compacts support c. So, it is just 1 into d t to the power 1 by q which will basically give me out the measure of c to the power 1 by q. So, on the right hand side it is a norm of f p which we already know is finite and the measure of c is also finite as f has been assumed to be time limited. On the LHS as this is less this has necessary f should necessarily belong to L 1. Next we will move on to some tools that we will be using for this whole thing. The tools are a particular Fourier transforms and in linear algebra the Vandermont matrix. So, we will be considering the Fourier transform of the rectangular poles it is defined as to have a value 1 between a by 2 to minus a by 2 and 0 everywhere else. So, this is a real symmetric function. In Fourier analysis we know that a real symmetric function gives a real symmetric functions otherwise what we can do is in the Fourier integral for a symmetric function f t we can just take the cos 2 pi omega t part because the sin omega t will give us a 0. So, let us see what comes out over here. Now the interval changes from minus a by 2 to a by 2 because all the rest is 0 and the way we define the sinc functions this is just sinc a of omega and this structure of the sinc function the factor a is only inside the sin part is very crucial for our proof. Next the Vandermont matrix this matrix is defined as c j's are the different values for different columns and their corresponding powers in different rows are i minus 1. So, when it is written down the matrix becomes like this a 1 and square up to say a n a 1 to the power n on the other side if it is a n then it goes as a n square and stops at n to the power n. So, whenever we have a linear equation a x equal to y vectors we are always looking at invertibility of a so that when we know y we can solve x like this. Now Vandermont matrix have some very nice properties for which it can be shown to be invertible if we just know the entries. So, here it is how it works. So, a 1 to the power a 1 to the power n and in between let us replace any particular column by x. So, it goes as x x square till x to the power n. Now, let us take the determinant of the matrix. So, we can take the determinant here already let us call it d x. This will be a polynomial in x with the maximum power x to the power n. If we put x equal to a 1 if we put x equal to a 1 these two columns are going to be the same and the determinant will come out to be 0. Similarly, for all other a n's except say we have put this x at the j th column except a j for all other a's this determinant is coming out to be 0. So, the polynomial d x will be of the form some constant which is minus 1 to the power some power into pi x minus a l's goes from 1 to n, but not equal to j and the rest part. This rest part can be found out. Now, let us replace x by a j. So, it is a j minus a l. Similarly, in this matrix we can replace other columns by x also and do the same operation. So, for each and every column there will be a product like this, but in the determinant the highest power of a 1 that we can have is a 1 to the power n. With that limitation what finally comes out by combining all these products is a i minus a j say i less than j. So, that we do not repeat and i is not equal to j ever in that case it is just 0. Now, matrix m is invertible whenever the determinant mod m is not equal to 0. So, over here whenever my entries in the first row a 1 up to a n are mutually distinct we get that the determinant is not equal to 0. So, it is always invertible in that case. Next we will move on to our theorem that a function cannot be compactly supported both in its time and its Fourier domain. So, the function only non-zero functions we will be considering because in our proof we will be showing that the only function which is allowed to have this property is identically 0 function. So, let us first consider some very smooth functions which have compact support. For example, the function e to the power in between minus 1 to 1 and 0 otherwise. So, it can be shown the function has a structure like this e to the power x false of when x goes near 1 this part goes near infinity and because of the minus sign c x tends towards 0. Not only that if we take the derivative of c s c x what we get is the minus parts cancelled out. Now, as x goes near 1 this thing goes near infinity as before and the exponential part falls off. On the other hand the lower part of this thing goes towards infinity, but exponential falls off faster than any polynomial that can be made. So, overall this thing tends towards 0 if we take the limit and this holds for all derivatives not only c prime x it holds for all c p x p belong to natural number. From this it can be shown that the function is very smooth. So, minus 1 and 1 this function and all its derivatives tend towards 0. So, can we have a very well behaved function like this also for which the Fourier transform is compactly supported. So, there are various proofs which use various sort of things for example, analyticity, holonomicity and all those things. So, I will first state one proof and give a brief overview of it. When a function is band limited and say the function f belongs to l 1 then as f s can be written like this we can have a bound on the modulus of f s. Now, this is 1, but this is the l 1 norm of f 1. The Fourier transform of the derivative of f f dash t if it exists is going to be i omega f into e to the power minus 2 pi i omega t g t. Now, we have assumed f to be time limited. So, instead of the whole real line the only interval we can handle now we will be handling now is the compactly supported interval c over here also because omega is in c only t sorry t is in c only when you take the modulus again we will get a bound for f dash t. We can show that all these derivatives f dash t hold at all points when f is both time limited and band limited. So, over here f was band limited. So, for only some interval omega 1 to omega 2 it would have been non 0 f omega and for the rest it would have been 0. So, if that holds we can write the function f t by the Maclaurin expansion as f 0 plus f t into f t 0 and so on. Now, due to the assumptions on f all this number exists. So, this is basically a power series expansion and this holds for all t. In that case by the identity theorem if this power series is 0 on any open interval it is going to be 0 everywhere on the real line. For compactly supported f of course, the power series is 0 outside the compact support c. In that case only the 0 function f equal to 0 is allowed because the function reduces to 0 by the identity theorem. This theorem states that a function f which is holonomic which has a power series everywhere is defined completely by any its values on any open interval. Now, the whole motivation of this proof is new proof is that all this previous proofs that are there using holonomic city or celebrated Parley theorem this require complex analysis and requires master in complex analysis. You have to know quite some bit of complex analysis to know this basics. On the other hand we have many other ways of looking into signals. By unlimited signals of course, we have come across the Shannon 9 quest reconstruction formula or the Shannon we take a cardinal series whatever it is called the reconstruction formula is a way to look into this theorem with that. So, in Shannon's revolutionary paper in signal processing he gave this formula which is called the Shannon 9 quest formula. What is so revolutionary about it? So, this formula holds when f is band limited this f h k is the samples. So, say there is a function f like this can you have an operation for measuring the function f t where to take the value of f t at each and every point. Instead of that if there is some limitation on f over here the limitation is in the Fourier domain it is compactly supported. So, can we remove the redundancy if there is some. So, indeed there is some redundancy in the Fourier domain because it is band limited we can take only the samples at regular intervals. So, h is a sampling time it goes over all case and from that we can get back our f t we will show graphically how this works. So, say a function f t is compactly supported in its Fourier domain. We will be using distribution theory for this function for this proof, but in the actual proof I have used just real analysis no distribution theory, but for the sake of brevity we will be showing here with distribution theory which equally holds. So, if you take a train of delta functions by delta functions we mean say a function delta function is defined like this the integral over the whole real line is 1, but the function is 0 at all points except t equal to 0. So, this is not actually a function in the normal sense it is a distribution. For Dirac delta train on the other hand side we also get another Dirac delta train a series of pulses like this. So, if this is t the difference between this is 1 by t. Now we know that multiplication in the Fourier domain implies convolution in our real set the time domain and vice versa. So, over here we multiply in the time domain with delta functions and this gives a sampling it just picks up the values at those particular points and everywhere else it is 0. So, you get some weight delta functions a i a i plus 2 something like that. On the other side capital f is getting convolved with delta function, but there is another property of delta function that it is convolution with any function gives the same function back. So, over here this whole capital f will be repeated along the delta train at each and every delta it will be repeated and the whole thing will be summed up final outcome is something like this. Now, can we reconstruct the signal from this say we had the repetition repeated lobes like this. If we did ideal lobes filtering with the rectangular pulse over here on the other hand side on this side we would have got our actual f s back now Fourier transform is a 1 to 1 operation. So, on the other side the corresponding operation should give us back f only. So, over here we can already see what is going to happen the threshold conditions say the repetitions of f is overlapping like this we cannot do the ideal lobes filtering. So, the condition for ideal reconstruction is that the capital f lobes have to be well separated and this is the threshold condition where they exactly touch say this is band limited by s minus s to s. In that case our repetition rate has to be good enough so that they become well separated. If we make the delta trains closer in the time domain in the frequency domain they are going further apart. So, threshold condition this is 2 s and 1 by t. So, 1 by t should be greater than 2 s greater than equal to 2 s. So, t should be get less than equal to 1 by 2 s and when this equality is there that is the shun nyquist threshold condition. Now, let us consider functions which are time limited and band limited. So, after multiplying with the delta train we get lobes like this and sampling over here. For f t which is completely supported in the time domain we get only a finite number of samples which are non-zero everywhere else it is 0. On the Fourier domain if we take t to be sufficiently small or h which will be using over here on the other side they are well separated with 0s in between them whole intervals which are 0s. Now, we have freedom on the reconstruction we can multiply this whole repetition with a rectangular function which goes from this point to this point. Because this is just 1 it is going to give us f s till this rectangular function remains within this range. So, we will have some other freedom let us call the rectangular function it goes to sigma prime and sigma is the band limit. So, we will define a parameter m which is 2 h sigma 2 h sigma prime. On the other hand in the time domain by corresponding operations we get the series to be the cardinal series or the shun nyquist series to be f h k sinc of here the m comes in and this is a factor of m in the form. Now, f is time limited. So, k will not go from minus infinity to infinity it will stop somewhere say at n. So, now this is a finite series. Now, what I told before the property of sinc function this sinc becomes sin pi m sin sin sin sin sin sin sin sin sin t minus h k by h due to this structure of the sinc function this 2 m's are going to cancel out which is very necessary for our proof. Now, what was the condition on m? We did not find out exactly, but here was sigma here was the repetition. So, sigma prime had to be between this was say 1 by t. So, if we take sigma prime lies between 1 by 2 t to sigma then this exactly holds. We can rewrite that with m whenever m lies between 1 and 2 h sigma the reconstruction formula holds. If the reconstruction formula holds the series cardinal series converges to f t. Now, k goes from some finite number n minus n to n. Now, when this condition is satisfied the m for all real m like this this holds exactly in that case we can take this cardinal series to be a function of m and t and if we fix t f t becomes a number t minus h k by h becomes become numbers. So, we have a freedom of m which we can manipulate. Let us take the first derivative of c m t where t is fixed. So, we would not write t anymore. So, c prime m t minus h k by h comes out it gets cancelled pi also gets cancelled and we are left with f h k cos m pi t minus h k by h k going from minus n to m. Now, f t was already fixed. So, when we take the derivative with respect to m it is just 0. So, c prime m is 0 and here is this finite series. If we write it graphically. So, this is a plot of c m t with respect to m. So, when m lies between 2 h sigma and 1 this is going to give us f t. We do not know what is going out going on outside, but over here we are sure that this is a constant function. So, we can take as many derivatives as we want of c m t with respect to m. So, let us take second order derivatives of c prime m second order derivatives with respect to m for this cos it will just bring out pi minus t k h by h to the power 2 i. So, say this is the ith second order derivative 2 i plus 1 and all of them are going to be identically 0 because c m is a constant function when m is like this. We can write this whole thing down as a matrix equation. So, say how many samples did we have? It is minus n 0 and n. So, 2 n plus 1 we take 2 n plus 1 equations like this doing the derivatives. So, the Vandermont structure Vandermont will come later on. So, we get a structure like this and c k. So, what we are going to get that V c is equal to 0 writing down all the 2 n plus 1 equations. Now, this V has a Vandermont sort of structure. It is going to be invertible as long as t minus h k square k i square is equal to t minus h k to k j square. This thing does not hold and this thing only holds for a finite number of values of t i not equal to j. Now, we can work on the whole of the real line other than this values of t. Let us call this set of t as b. For all other t we get that c is equal to V inverse into the 0 vector. So, it is absolutely 0. So, c is 0, c k is 0 for all k. In that case either f h k is 0 or the cos part is 0 or both of them. Now, the cos part can be 0, cos part can be only when m pi minus this thing, this whole thing equals odd multiple of pi by 2. So, only for some finite countable number of values of m this can hold, but in our proof we have a whole range of m. So, this is not going to hold for all m if we take any other m. This will give that for all the other m f h k is equal to 0. Now, it can be showed that for a time limited and a band limited function, the functions are very smooth. So, all the derivatives exist they are bounded on both the sides and all these things. So, if these things happen our cardinal series C m, our cardinal series is going to converge exactly not only point point wise, but exactly it is going to converge to f t. So, if you have f h k is equal to 0 all these things are equal to 0. So, if it converges to f t on the other side I am left with only f t equal to 0 nothing else, but we are missing the set b t belonging to b, but we already know that f is continuous. So, if only for a finite number of points it is non-zero and for the rest of it is 0 that cannot hold it has to be 0 on the whole of the real line. So, here it is we have finally, got that only the function f equal to 0 is allowed to have this property. We have had here a very interesting exposition of the proof of the inability to have simultaneous compact support in time and frequency by Soham. I must appreciate the effort by this young man to look again at the concept carefully to identify a number of basic lemmas inequalities and equalities that underwrite the proof and to present it in such a lucid manner. Not withstanding the fact that there have been small glossing over of points during the proof which we can condone the idea has come out beautifully and he has been able to put across the ideas in a very accurate and a very beautiful manner. So, even though at places we might have found small inaccuracies in writing which you know one can condone I thought I should mention that because when the audience listens to it it might be found that there are some places where symbols have been missed and so on or you know written with a little bit of overloading of the symbol and so on, but that happens I think we can condone that, but otherwise I think the proof has been brought out beautifully and I think we should appreciate this young man for the beautiful proof and the beautiful way in which he explained the idea and it is satisfying in two ways. It is satisfying because it tells us technically how worthwhile it is to pursue the subject of at least trying to be reasonably compact or you know going towards a reasonable support in one domain when the other is compactly supported. We know that there are wavelets that are compactly supported. Now, we also know from the discussion from Soham that in the time in the other domain they cannot be in the frequency domain they cannot be compactly supported, but the challenge is how close you can go to constraining them in the other domain. So, the challenge becomes evergreen once again that is the technical reason. The personal reason is that here is a young man who has looked at the concepts of the course carefully and put in his own thought his own creativity to augment the ideas of the course. I encourage all students who at some time of the other listen to these lectures to take an example from these students who have enthusiastically participated and participate as enthusiastically. Thank you.