 A warm welcome to the 19th lecture on the subject of wavelets and multirate digital signal processing. In this lecture, we shall go further on the uncertainty product or the time bandwidth product as we called it in the previous lecture to proceed to evaluate and to bound it to put certain fundamental limits on it and then to see how close we can get to those limits. Therefore, I have titled the lecture today as evaluating and bounding sigma t squared sigma omega squared. To put the discussion in perspective, let us recall very briefly what we did in the previous lecture. You will recall that we had talked about these quantities sigma t squared sigma omega squared. We had also noticed that this product sigma t squared sigma omega squared is a very strong invariant. It is invariant to translation, it is invariant to modulation both in time, translation in time, modulation in time. It is invariant to multiplication of the dependent variable by a constant or scaling the dependent variable or in other words multiplying the function by a constant. Most interesting of all is that it is invariant to a scaling of the independent variable that is striking. So, you see ultimately what is left is just the shape. The time bandwidth product is a direct function of the shape. The shape can then get compressed or expanded, it can get scaled in the vertical direction, it can be shifted, it can be modulated by an e raised to power j omega t kind of term rotating complex number or a phasor. All these do not affect the time bandwidth product sigma t squared sigma omega squared as we called it. Now, our objective today is to ask a very important question which lies at the heart as I said of wavelets and multirate digital signal processing namely how small can we make this product? Let us put down the question first. Essentially how small can the time bandwidth product be? Once again we need to emphasize a couple of points. Why are we talking about the time bandwidth product instead of just the time variance or the frequency variance? We must understand that making the time variance small or the frequency variance small is not a difficult job at all. In fact, one can do it by scaling the independent variable. If you compress a function in time, the time variance is compressed by the square of that factor. If you compress a function in frequency, the frequency variance is compressed by the square of that factor. So, compressing in time or frequency separately is not a difficult job at all. In fact, the problem is that when you compress in one domain, you are expanding in the other domain by the same factor. There is a cancellation effect as far as the time bandwidth product goes. So, compressing or expanding a function does not change its time bandwidth product, but it definitely changes the time variance and the frequency variance individually. This is what we saw in the previous lecture. In that sense, physically what it means is that nothing stops you from narrowing down as much as you desire in one of the domains. If you desire to focus on a small region of time, you can do it on as small region of time as you desire. If you focus on a certain region of frequency, you can make that region of frequency on which you focus as small as you desire. There is no problem. So, focusing in one domain is not a problem. You can do it as much as you desire. The problem is focusing in both the domains together. Any tool that we use, by tool I mean a function used for analysis. Any tool that we use has a time bandwidth product essentially based on its shape. So, when you use the same shape, you are bound by that number. No matter how much you squeeze or expand, you are not going to be able to focus in both the domains simultaneously. In fact, by focusing in one domain, you are going to compromise on your ability to focus in the other. This is the general tussle as we said between time and frequency. Now, the next natural question to ask is, how small can we make this tussle? If sigma t squared, sigma omega squared could have been 0 for example, could have been. It would be wonderful. Nothing needed to be done. You could make any of them as small as you desire and the other one as small as you desire too. So, you could focus in two domains simultaneously. But as I said, nature does not allow this and it is a fundamental property of nature that it does not allow this. What we are now going to answer is, what is the lower bound on this product? Meaning, to what extent no matter what tools you have, can you really focus in both the domains together? So, what is the ideal towards which we must strive and why is it that we have a fundamental limitation to work with which always poses a challenge before us? Anyway, to answer that question, let us recall some of the expressions that we had derived in the previous lecture. We had looked at an expression for the time bandwidth product based on entirely time domain quantities and let us put that expression down clearly once again. We had shown that the time bandwidth product without loss of generality can be written ultimately as follows. It is the L2 norm of Txt the whole squared times the L2 norm of the derivative again the whole squared divided by the L2 norm of x squared multiplied by the same. So, in other words, you could write the L2 norm of x raised to the power of 4 if you like by combining these two terms. Anyway, I am just writing them separately to emphasize that this is associated with this and this with this notionally. Now, of course, I must also again mention the slight abuse of notation here. When we write T times x t, we are referring to the whole function as an object and not to a specific value of t the same holds here. Anyway, this was just to recall what we did yesterday and remember we had done this after doing a little bit of preparation on the function so to speak. Having noted that translation in time and in frequency has no effect on the time bandwidth product, we had said we could shift the function in time so that it is centered in time. Subsequently, we can shift it in frequency without affecting the time function so that it is centered in frequency. In all this, the time bandwidth product does not change. So, here we have a function which has been assumed to be centered in time and frequency and now we are working with that function and this is without any loss of generality. Anyway, coming back then to the calculation of these quantities, let us once again give a vectorial interpretation to this. Now, you see look at the numerator. The numerator is a product of two squared norms. Let us focus our attention on the numerator. So, numerator here is the norm of T x t the whole squared multiplied by the norm of d x t d t the whole squared in L 2 R. Now, let us take recourse to a very fundamental principle that we know in vector analysis. Treat T x t and d x t d t as vectors as we often do generalized vectors. So, let us call them v 1 vector and v 2 vector. Now, if we recall a basic principle of inner products, the inner product of the product of the product of v 1 with v 2 is essentially of the form the magnitude of v 1 times the magnitude of v 2 times the cosine of the angle between v 1 and v 2. Theta is the angle between v 1 and v 2 and therefore, it is obvious in very low dimensional spaces which can be used and of course, be taken to higher dimensional spaces also that if you consider the magnitude squared of the dot products. Let me use the dot product notation. It is of course, the product of the magnitude squared of v 1 and v 2 multiplied by cos squared theta and cos squared theta is always less than 1 in fact, between 0 and 1. Here of course, we are talking about theta real and that is the interpretation which we always have even if we are talking about complex functions. The angle here is assumed to be real. You see what it means therefore, is that the modulus of the dot product squared is always less than or equal to the modulus of v 1 squared times the modulus of v 2 squared. This is a very simple, but a very important principle. In fact, this principle can be generalized to these generalized vectors that we are talking about functions viewed as vectors. In fact, this is a very important property or a very important theorem in functional analysis. It is often called the Cauchy-Schwarz inequality. It says and let me write that down in formal language. It says the inner product, the magnitude squared of the inner product of two functions. So, let us assume there are two functions f 1 and f 2 in l 2 r. The magnitude squared of the dot product of f 1 and f 2 is less than equal to the product of the squared norms in l 2 r of f 1 and f 2, the Cauchy-Schwarz inequality as it is often known. Now, of course, this vectorial interpretation makes this property obvious, but one can also prove this formally in functional analysis without taking recourse to this visualization in the language of vectors. However, since it is not our objective to review this basic concept from functional analysis, I mean to give a detailed proof, we shall give it this vectorial interpretation and be satisfied. So, we will take the Cauchy-Schwarz inequality as true and proceed from there. Now, please remember that we are assuming in this process that these functions belong to l 2 r. So, let us go back to our uncertainty time bandwidth product. Consider them x t, so that t x t belongs to l 2 r and d x t d t belongs to l 2 r. What if they do not? For example, we took the very simple case of a rectangular pulse a while ago. We saw that if we consider this function, not true for this function, because d x t d t does not belong to l 2 r here. Why so? d x t d t has two impulses. d x t d t would have an appearance like this and impulse there and a downward impulse at the end of the pulse. This is what d x t d t would look like here and impulses are not square integrable. Impulses do not have finite energy. I shall just spend a minute in justifying this, because we have so far been informally saying it. We have also proved this indirectly by looking at the frequency variance in this case, but we also must understand the direct proof here. After all, what is an impulse? An impulse is the limiting case of this, an object which lies on a width of delta with a height of 1 by delta. This is capital delta t if you please and an impulse or a unit impulse to be more precise is essentially something like a limit as delta tends to 0 of delta delta t. Now when you take the square of this, so delta delta t squared it looks like this and now if you take the integral of delta delta t squared d t over all t, then it diverges when you take the limit diverges. In fact, this integral is essentially 1 by delta. 1 by delta squared into delta. So, 1 by delta and this is divergent. This limit does not exist. So, therefore, the impulse is not square integrable. We must make a note of this. This is an important conclusion. Infinite energy, it contains infinite energy and that is manifested also in calculating the frequency variance for this rectangular pulse. Anyway, that a part. We have now agreed that if that is the case, anyway that lower bound does not arise, where these quantities diverge in spite of x belonging to L 2 R. Remember, since x belongs to L 2 R, in the beginning we had this denominator. So, you know you had this denominator right in the beginning. Norm x 2 x in L 2 R to the power 4 effectively and this is guaranteed to be finite because we have confined ourselves to x 2 into L 2 R. So, if this is in L 2 R, the denominator is finite and positive obviously otherwise it does not make sense. I mean you are not going to take a trivial function. So, for a non-trivial function, this is strictly positive and finite and then if any one of these is infinite, there is no question of finding a lower bound. It anywhere you know is the worst possible case that you can have. So, it is no harm then that we have in considering finite quantities in the numerator. So, with that little remark about our restriction, let us take them to be finite and proceed and use the Cauchy Schwarz inequality. So, using Cauchy Schwarz inequality, what do we have? Now, we will use it the other way. What we have in the numerator is the norm squared in L 2 R of t x t times the norm squared of d x t d t in L 2 R and from Cauchy Schwarz inequality, we go further. This must be greater than or equal to the magnitude of the dot product squared. Let us write this down. This dot product is essentially the following integral. Remember, we need not confine ourselves to real functions. We should not because we are allowing a modulation in time. So, we must allow complex functions here. We have of course, centered the functions. That is a different issue. We need to center them. That we have done. Now, let us essentially look at this integral a little more carefully. Now, we are talking about this as an entire complex number. This whole thing is a complex number from here to here. If you take the magnitude squared of a complex number, so let that complex number be z. It is obvious that the modulus squared of z is greater than the modulus or greater than or equal to in general, the modulus squared of the real part of z. We will use that property here. Of course, the same thing holds good for the imaginary part too, but we are interested in the real part here. In particular, we have the numerator of the time bandwidth product is greater than or equal to modulus real part of this integral. The complex conjugate is above this entire thing. This is what we have here. Now, a remark about this part, this complex conjugate. We are taking the derivative of a possibly complex function x of t with respect to the real variable t. So, the complex conjugate of dx t dt is also the derivative of the complex conjugate of x t. What I am saying in effect is that because t is a real variable, dx t dt bar is also dx bar t dt and I shall employ that in this expression first. So, this quantity now is equal to the modulus of the real part of the following. The real part operates on the whole integral. Now, we again look at the real part. You see, the real part is operating on an integral with respect to t. t is a real variable. So, the element of integration is real. This is a real function. This is possibly a complex function. So, the real part can be taken right inside the integral and brought to operate on this only. The rest of it does not require you to say real part explicitly. So, this expression is equal to integral from minus to plus infinity t times the real part of x t dx bar t dt dt the whole squared. Now, we take note of this. How do we calculate the real part of a complex function complex number in general? By adding the complex number and its conjugate and dividing by 2. So, essentially what we are saying is real part of x t dx bar t dt is half x t dx bar t dt is half x t dx bar t dt plus the complex conjugate of this. And what is that complex conjugate? It is x bar t dx t dt. Simple. And now, we can see a product rule has been employed here. Essentially what we have here is the derivative of x t into x bar t. That is an important observation. So, here this is equal to half d dt of x t x bar t. And x t x bar t is indeed the modulus of x t the whole squared. A very beautiful observation. And now, we will put back that observation into the time bandwidth product. So, therefore, the numerator of the time bandwidth product is thus greater than or equal to modulus of half integral from minus to plus infinity t times d dt mod x t squared d t. And this is the whole squared. Let us take a minute to reflect on this. How do we evaluate this integral? Well, that is easy. We can evaluate this integral by parts. And to evaluate by parts, we must first make the integral indefinite. So, let us simply consider the indefinite integral corresponding to this. Clearly, this is this term minus this term. And now, we can substitute the limits. So, we can put back the limits of minus to plus infinity here. And put this from minus to plus infinity first. And then this from minus to plus infinity as well. Now, let us focus our attention on each of these terms individually. Let us take the first term t times mod x t the whole squared. Evaluate this integral by parts. Now, let us evaluate it at plus infinity. And then from its subtract, evaluate it at minus infinity. So, let us focus our attention on t mod x t the whole squared. Evaluate it at the two limits and subtract it. Now, we have agreed that t squared mod x t squared d t from minus to plus infinity is finite. We have agreed on that. We said if that is not true, anyway we have an infinite bound. There is no question of lower bounding there. It is the worst case that we can deal with. So, if this is finite, obviously you see if the integral must be finite, the function must decay towards 0 at the ends. If the function does not decay to 0 asymptotically as you go towards plus infinity and as you go towards minus infinity, you can see that there is going to be an infinite range over which the function has a finite positive value which would make the integral diverge. So, in order that this integral converge t squared mod x t squared d t, the function t squared mod x t squared must decay as t tends to plus infinity and minus infinity. Let us make that observation. In order that this integral converge, t squared mod x t squared tends to 0 as t tends to plus infinity and t tends to minus infinity. Now, it is also true that for t greater than 1 and of course as t tends to infinity, t is definitely going to be greater than 1. So, for t greater than 1 and certainly so and I am talking about t squared actually or mod t. So, certainly so as mod t tends to infinity, t tends to plus infinity or minus infinity. You must have t squared greater than t. What this means is that t squared mod x t squared tending to 0 as t tends to plus or minus infinity guarantees t mod x t squared t tends to minus infinity t squared tending to 0 as t tends to plus or minus infinity and therefore, that first term has vanished. So, we are left only with the second term and the second term is very familiar to us. In fact, this is minus the norm of x in l to r the whole squared. So, simple and therefore, we have a very beautiful conclusion here. We are saying the numerator is always greater than or equal to modulus half into minus the norm of x in l to r the whole squared which is one fourth times this times this and therefore, overall the time bandwidth product is greater than equal to one fourth norm x in l to r to the power 4 divided by the norm x in l to r to the power 4 which is one fourth a very very fundamental conclusion. The time bandwidth product can never be less than one fourth 0.25. So, 0.25 is the very lowest value of the time bandwidth product that you can get and what we concluded here has nothing to do with the tools available at a particular time with the technology available at a particular time or with the machines and the political situation or whatever. It is fundamental to signal processing. In fact, so fundamentally is this result of P F derived in different manifestations it is seen in different subjects. What we call the uncertainty limit in physics is just another version of this. People talk about the inability to locate position and momentum simultaneously. Actually it is just another version of this. I shall just give a small hint as to how you might connect this idea of time bandwidth product to the concept of position momentum uncertainty and to do that actually we will go back to the expression for the time bandwidth product that we had derived. You see, you notice that we have shown that essentially the time bandwidth product relates to a time variance relating to the norm of t x t the whole square and you are assuming the object is centered. So, now think of x t as descriptive of an object. Remember that we have talked about a one dimensional mass. Now, here we are talking about x t then as a kind of mass distribution of the object or rather if not mod it is if not x t it is really mod x t square which is like a mass distribution of that object. So, division the denominator part is essentially a normalization make the object unit mass if you like, but here in the time variance the numerator is indicative of the uncertainty in position. What is the variance of the object as far as it is spread around the center is concerned that is what the time variance tells you. Now, if you look at the other term there the frequency variance. So, time variance involves this frequency variance involves this you know involving this the derivative of x t with respect to t. Now, x t or mod x t square as I said is indicative of in some sense the presence of that mass or that body on t. So, d x t d t is indicative of the movement of that body the change with respect to time. So, here in some sense the change it is indicative of a change. So, in a broad sense you can see the connection between position and momentum uncertainty in where it is an uncertainty in how it changes. Let me not dwell too much further on this to interpret precisely how this is the uncertainty principle in physics. I think we should leave it to a physicist a person who specializes in that, but this is what I have just given you is an intuitive indication that is all. However, it is just to bring out the various meanings that this time bandwidth product or this uncertainty limit has it has meanings in different subjects. Anyway coming back to a subject with which we are dealing this is the whole basis of the subject of wavelets of that matter time frequency methods. What it tells us is that no matter what we do we are not going to get any function which has finite energy and which can be confined beyond a certain point in the two domains time and frequency simultaneously that is bad news. What was worse is that if you looked at the hard case it is not confined at all that time bandwidth product was infinity. So, now we naturally ask the next question is there a function which gives us this 0.25 or is it something that we should never seek that is not very difficult to answer. In fact that can be answered again by using a vectorial interpretation. When does the numerator become equal in the Cauchy Schwarz inequality to the expression that we derived the optimal function in the sense of time bandwidth product means the Cauchy Schwarz inequality becomes an equality. Now, when would that become an equality it would become an equality if the cos square theta term is 1 that is the cos square theta term is 1 and what you mean by the cos square theta term being 1 what really is theta theta is an angle between these two so called vectors t x t and d x t d t. If you want the angle to be such that cos square is 1 these two vectors must be collinear they must be along the same line so to speak. So, therefore, in the language of functions we need the vectors so called vectors t x t and d x t d t to be collinear 1 dimensional what you mean by them being collinear they must be linearly dependent and what you mean by them being linearly dependent any of them must be a multiple of the other. So, in other words d x t d t is some constant let us call that constant gamma 0 times t x t the solution of this equation would give us the so called optimal function. So, we solve this how do we solve it by a simple change of the or this redistribution of the derivative. So, essentially what we are saying is d x by x is equal to gamma not t d t and if we integrate we get gamma 0 t square by 2 or log natural x well you must have a constant here a constant of integration and this is of course log natural x. So, in other words we have log natural x is of the form gamma 0 t square by 2 plus c 0 let us raise both sides using the natural base e. So, e raise the power l n x is e raise the power this and therefore, we have e raise the power l n x is just x. So, x of t is some constant e raise the power c 0 this is a constant let us call this constant c 0 till day times e raise the power gamma 0 t square by 2. So, let us write that down again and let us make a remark on gamma 0 x t is of the form c 0 till day times e raise the power gamma 0 t square by 2. You want x t to belong to l 2 r and therefore, you want mod x t square which is essentially mod c 0 till day square times mod e raise the power gamma 0 t square by 2 to be integrable. Now, this is possible if you have a constant only if gamma 0 has a negative real part. If gamma 0 has a positive real part this is going to be a Gauss so called Gaussian that grows in time it is not going to be square integrable and therefore, we must choose gamma 0 with a negative real part and we can choose any one example of that. So, in other words one function which reaches the lower bound which is optimal in this sense is with gamma 0 equal to minus 1 and there we have we can also choose of course, c 0 till day to be 1 because it does not matter if you scale a function the time band with product is unaffected and therefore, one optimal function in the sense of time band with product that is with time band with product equal to 0.25 is the Gaussian a very interesting conclusion. The Gaussian is optimal the Gaussian seems to arise in many situations it has a reason in this having noted this we shall conclude today's lecture and proceed in the next lecture to delve further into this issue of how close we can get to the optimal with other functions. Thank you.