 A very warm welcome to the 16th lecture on the subject of wavelets and multirate digital signal processing. Let us put in perspective what we are going to do in today's lecture building up from what we did in the previous one. In the previous lecture, we had looked at the Fourier transform of the scaling function or the so called father wavelet phi t and the wavelet function or the so called mother wavelet psi t in the harm multi resolution analysis. What I shall do is to begin with a description of where we wish to go from here. You see, we had made some observations about the nature of the magnitude of the Fourier transform of phi t and psi t. We also noted that when we multiply a given function x t by a translate of phi t, the magnitudes of the Fourier transforms of x and phi are getting multiplied. And we saw that the nature of the Fourier transform of phi or for that matter even that of psi was such that it emphasized some band of the Fourier transform of the underlying function x which was being studied. Now, what we intend to do today is to idealize from there. What is the ideal situation to which we strive? And therefore, I have put down the central theme in the lecture today to be ideal time frequency behavior. What is the ideal towards which we are trying to move? Let me put before you once again the nature of the Fourier transforms of phi and psi. By nature, I am essentially going to refer to the magnitude. The phase though important in general is not of prime importance at the moment because it is the magnitude which makes a selection of band. So, let us put down the nature of the magnitude. So, form of magnitude of Fourier transform for phi it had an appearance like this. So, this was 0 frequency and this was 2 pi here and all multiples of 2 pi subsequently and so on. This is mod phi cap omega. You see when I say form what I imply is that I am not going to consider any constants of phase. Constants would only scale this up or down and the phase would not affect the magnitude of course. So, let us also look at that of psi. The form of the magnitude of the Fourier transform of psi looked something like this. Sin square omega by 4 divided by omega by 4 and we had made an attempt to sketch this last time. We first sketched sin omega by 4 by omega by 4. So, we said this was the form of mod sin omega by 4 divided by omega by 4 the solid line here and I also drew a dotted line to indicate this time let us use a dot dash line to make a distinction from this margin or this axis. So, let us use a dot dash line to denote the magnitude of the other term sin omega by 4. So, that would have a peak at 2 pi. This is the form. Now, this solid line is multiplied by this dot dash line here. So, of course, you must visualize this dot dash line being replicated on the negative side and as you know for a real function the Fourier transform is magnitude symmetric. So, it is enough for me to study the positive side of omega and the negative side would be a mirror image. So, let me expand this part here focus on the positive a good motto in general. You see this sin omega by 4 by omega by 4 this one has a monotonically decreasing character from 0 to 4 pi. This one has a monotonically increasing character between 0 and 2 pi and then monotonically decreasing. Now, it is very clear that from this point onwards the product of these two is only going to decrease. So, you cannot possibly have a value of this product this of course, being mod sin omega by 4. You cannot have a magnitude of the product of this dot dash line with this solid line greater in the segment between 2 pi and 4 pi than it is at 2 pi. So, in other words after 2 pi between 2 pi and 4 pi this product is only going to decrease and therefore, I can get a feel. You see it is clear that the product is 0 at omega equal to 0 whatever is after 2 pi is going to be less than what is at 2 pi. So, somewhere in between it is going to achieve a maximum and then continue to drop. So, we get a feel of this we must get a finer feel of this and we had the last time. In fact, let me also make one more remark. You see if you look at the region between 4 pi and 8 pi thus this situation is a little simple. There is a kind of tendency to a maximum somewhere in between in both of these functions and then a drop. So, that similar pattern would be replicated in the product a maximum somewhere in between well not quite at 6 pi please remember this is not quite symmetric you must remember that although this is this is not quite symmetric. So, the maximum will be somewhere other than 6 pi, but that is not of so much of concern there will be a maximum somewhere near 6 pi and it would drop off on both sides. So, in total this is what the product would look like. I will mark this maximum it is a little difficult to calculate, but this is the nature this is the form and now let us take the trouble to draw them together again only on the positive side of the frequency axis. The form of the Fourier transform of phi and of psi let us focus only between 0 and 4 pi. So, phi looks something like this and psi looks something like this and we had made a remark on what phi does and what psi does phi in effect emphasizes those frequencies lying around 0 frequency and psi emphasizes those frequencies lying around its maximum in the band between 0 and 4 pi and deemphasizes frequencies on either side. So, in fact, if you look at psi it deemphasizes frequencies around 0 and then after that band. So, it emphasizes the band of frequencies it is clear that psi has a band pass character it is a band pass function. A band pass function is 1 which emphasizes frequencies around some sense. So, called center frequency where the response is a maximum and deemphasizes frequencies both around 0 and around infinity. So, to speak. So, there is a finite band of frequencies 1 band which that function emphasizes loosely speaking this psi omega here emphasizes those frequencies lying around its maximum here and of course, phi emphasizes frequencies around 0. We also made one more remark on the distinction between phi and psi. You see we noted that when we contract or expand. So, when we go up or down the ladder what are we doing in the Fourier domain when we go up the ladder we are expanding in frequency because we are contracting in time when we go down the ladder we are expanding in time and therefore, we are contracting in frequency. So, let us look at this figure once again as we go down the ladder we are contracting in frequency. So, we are emphasizing smaller and smaller bands around 0 and again since we are contracting this as well we are emphasizing frequencies around a smaller and smaller center frequency. In fact, it is very easy to see that this center frequency the point where there is a maximum in the magnitude of psi decreases geometrically or logarithmically as we go down the ladder in the higher multi resolution analysis and the width of this band also decreases geometrically or logarithmically. This is something very interesting the band decreases geometrically the center frequency also decreases geometrically. So, we have a situation where the ratio of the band to the center frequency is a constant. So, we have a name for that kind of analysis in the literature on wavelets or time frequency methods. We call it constant quality factor analysis or constant q analysis let us write that down psi in effect does constant quality factor analysis or constant q analysis and this word quality factor comes from a term used in the context of band pass filtering. For band pass filters or band pass functions the quality factor or q as it is often denoted in grief is the ratio of the center frequency to the band or the bandwidth. You know the word bandwidth of course has to be taken with a pinch of salt. What does bandwidth mean there are different definitions particularly when you do not have a clear brick wall situation. You have a smooth variation of magnitude with frequency as you do here. So, there is a maximum and the frequency falls off on either side. Typically, we use the word bandwidth to denote that range of frequencies within which the magnitude remains within a certain percentage of the maximum magnitude. So, for example, where the magnitude remains between 70 and 100 percent same of the maximum magnitude. Or where it remains even more specifically for most situations we talk about what is called the half power bandwidth where the amplitude or the magnitude response falls to the square root of half from the maximum. And the square root of half has the significance that at that point where it falls to the square root of half the power of a sine wave is half of what it would be in proportion to the original as compared to the maximum point. So, if at the center frequency the point where the magnitude response is a maximum the power ratio of input to output is say 100 units. Then at the point where the power the you know the magnitude falls to 1 by square root of 2 the power would be only 50 units the ratio power ratio output divide by input. So, it is called the half power point very often we talk about half power bandwidth in any case it does not matter what percentage we use 70 percent so be 60 percent so be whatever it be with this notion of bandwidth the ratio of the center frequency to the bandwidth in this sense is a constant as we stretch or compress the Fourier transform of psi. And therefore, of course you know whatever you do in terms of stretching or compressing in the time domain you are doing exactly the opposite in the frequency domain. So, as you go up the ladder you are going towards higher frequencies and you are also spanning a larger bandwidth. As you come to lower steps as you go descend in the ladder you go to lower rungs of the ladder you are essentially going to smaller center frequencies and using a smaller bandwidth. Now, let us bring in the idea of time resolution and frequency resolution here. If we use bandwidth as a measure please note as a measure of the range of frequencies that are emphasized by the function psi. Now, why am I saying once again that these frequencies are emphasized let me just recapitulate I am saying this again and again because one must firmly understand this. I am saying that those frequencies are emphasized because in finding the dot product of a function x t with any translate of this function psi t or one of the stretched or compressed versions of psi t passable's theorem tells us that you are also multiplying the Fourier transform of x with the Fourier transform of that particular translate and dilate of psi or for that matter phi whatever it be. Now, we also understood that translation has no effect on the magnitude dilation does and when we multiply the Fourier transform of x by the Fourier transform of phi or psi as the case may be appropriately dilated. One is automatically emphasizing multiplying that part of the band which lies in the region of large magnitude of Fourier transform of phi or psi by a larger number and the other parts are being multiplied multiplied by a tapering number. So, in effect there is a filtering operation also being done by phi and psi. Effectively phi is doing a low pass filtering operation and psi is doing a band pass filtering operation. Let us make a note of this is very important. So, effectively phi is doing a low pass operation psi is doing a band pass operation then it almost seems trivial what is so great we could have built a band pass or a low pass filter otherwise why did we have to do all this hard business well you see the beauty is in the two domains together and this is where the whole catch lies and this is where the whole struggle lies. You are able to do some kind of a crude low pass operation I say crude because nobody will agree if you look at the frequency response the Fourier transform of phi that it is really very close to a good low pass filter crude in that sense you are doing a crude low pass filter operation, but with the proviso that you are also confining yourself in time. So, you are saying you are able to say with some confidence and that confidence depends on how well localized that Fourier transform is around 0 frequency. So, you are able to say with some confidence that when I multiply x t by a certain dilate and translate of phi I am emphasizing that band of frequencies around 0 which is covered by the appropriate dilate of phi. So, if you take phi itself and if you focus your attention on the main lobe of the Fourier transform you may say in a crude sense that you are emphasizing the frequencies around 0 up to the extent of 2 pi the main lobe goes up to 2 pi and you are doing this in a time region in which phi lies. In fact, that can be sent non crudely. So, phi is indeed very localized in time I think nobody will disagree with that. So, is psi. So, when you multiply by a certain dilate in a translate of psi you are in effect doing a kind of localization in frequency around that point of maximum as you saw it lay somewhere near 2 pi before 2 pi actually and as you take different dilates of psi you are taking different bands and this is being done in the time zone covered by that particular translate. This is a serious statement we are making we are making a statement about localization in two domains simultaneously in time and frequency and if you recall in the very first lecture when I introduced the subject of wavelets and time frequency methods this is one of the things I mentioned as a fundamental challenge in signal processing. In fact, I went to the extent of saying the same challenge appears in different manifestations in different subjects. In signal processing we see it as a conflict between time and frequency where is the conflict? The conflict is partly seen now partly I say you see as you notice in time we are very correct in saying that we have localized after all phi t and psi t and their translates and dilates are non-zero only over a finite region of time. So, localization in time in this case is not under question at all it is localization in frequency which is somewhat suspect we can crudely say that because if you focus your attention in the main lobe then in some sense it is localized but there are these side lobes in the Fourier transform both of phi and of psi. So, now we want to ask the question what ideal would I like to strive towards? If I were to have my way how should I make the Fourier transform of phi and psi look? We know how they should be in time they should be packed into a finite region of time we are able to do that I would also like to pack them into a finite region of frequency simultaneously. Now what would that region of frequency be? Let us use our understanding of signals and sampling a little bit here you see let us write down the dot product of x t with a particular integer translate of phi as a sampling problem now. If you take this product if you wish I can put complex conjugates maybe I should put a complex conjugate there it would be as dot product in the strict sense, but even if I do not put a complex conjugate and confine myself to real functions x I am doing rather well. In fact we will do that for the moment because we do not want to mix too many issues let us confine to real functions and then I have this is of course equal from passable theorem to the Fourier transform of phi t plus tau times the Fourier transform of x integrated over all omega and this is easy to evaluate. So, essentially we have a product of Fourier transforms x and phi x cap and phi cap multiplied together and then an inverse Fourier transform is being computed at the point tau. So, this is like you know even if you were to use a complex function the only change would be here they would need you would need to put a complex conjugate that is why I said that that is not such a serious issue at the moment we will just focus on real functions and interpret. So, here when you multiply by phi cap omega you are in effect doing some kind of a low pass filtering and when you take the inverse Fourier transform you are calculating what comes out of that crude low pass filter whose impulse response is essentially phi, essentially phi I mean do not worry about inversions or you know time inverse it relates to phi very closely phi. Now what we are saying is when you sample this when you put tau equal to all the integers. So, if you take this and substitute tau by different integer value. So, when we sample at tau equal to n n all integers what is going to happen we are going to take the original Fourier transform you see when we sample if you take a function let us say y t with Fourier transform y cap omega and you sample this sample ideally if you like at all integers that essentially means your sampling at a sampling rate of 1. So, that amounts to taking the original Fourier transform translating it by every multiple of 2 pi divided by 1 which is 2 pi on the angular frequency axis and adding up these translates. So, let me write that down in terms of an algebraic expression what we are doing essentially is we are taking the original Fourier transform translating it by every multiple of 2 pi divided by 1 if you please every multiple of that and summing up these translates some constant possibly that constant relates to the sampling process. Let us ignore that constant for the moment our attention is here. So, in order to reconstruct y from its samples what should we have desired we should have desired that these translates do not interfere with the original. So, it would have really been nice if we had been able to ensure that these carbon copies created by y cap omega plus 2 pi k. So, these are non overlapping with the original and that is ensured by ensuring that the low pass filter cuts off at capital omega equal to pi let me sketch that for you. Had phi cap omega being an ideal low pass function with a cut off of pi then then we could have this aliasing process would leave y cap omega unaffected. So, that is the ideal towards which we are striving as far as phi goes. Now, what is the ideal towards which we are striving as far as psi goes let us see you see when you go from v 0 which is what brought us to phi to v 1 what is v 1 just essentially v 0, but compressed by a factor of 2 in time and therefore, expanded by a factor of 2 in frequency. So, for v 1 I am talking about the ladder m r a ladder har ladder we expand by 2 in frequency we are talking about frequency we are talking about frequency domain behavior. So, we expand by 2 in frequency that means we are asking for a low pass filter with cut off 2 pi instead of pi. Now, we also have an interpretation for the incremental subspace obviously if v 0 is going to contain information between 0 and pi and v 1 is going to contain information between 0 and 2 pi then the difference subspace w 0 should contain the information between pi and 2 pi simple. So, what we are saying in effect is psi is aspiring to be a variable. So, the band pass function between pi and 2 pi and of course, this is for going from v 1 from v 0 to v 1 when you go from v minus 1 to v 0 you use a corresponding dilate of psi which is aspiring to be a band pass function between pi by 2 and pi when you go from v 1 to v 2 then you bring in a dilate of psi which aspires to be a band pass function between 2 pi and 4 pi and so on. Let us draw the ideal situation. So, the aspiration the ideal is the following this is the aspiration for phi at least in terms of magnitude I show the aspiration for the corresponding phi of omega by 2 and then I will used a kind of dashed line here to show the aspiration for psi this is the ideal towards which we are going this one dot dash this one solid this one only dashes. Now, things have begun to fall into place in fact, now we can also see what we mean when we forget about phi entirely and use only psi what are we doing in the frequency domain or rather what are we aspiring to do. So, you know you can what I am saying is instead of thinking of all the shells up to a point and removing one shell think of the whole onion as only shells only size. So, what is happening then the following is happening in the frequency domain in fact now I need to work carefully around 0. So, I will draw a big pi here and a big 2 pi there. So, we will start with v 1. So, this is w 0 w minus 1 will essentially do this ideally w minus 2 would be here between pi by 4 and pi by 2 and so on. Each time you go towards 0 you are contracting this band by a factor of 2 and therefore, both the center frequency and the bandwidth have been reduced by a factor of half and of course, you can visualize going in this direction too. So, just for completeness I think I should draw w 1 and w 2 though not on the same graph it is difficult to do. So, we will draw it separately. So, to be specific we should say down the ladder here up the ladder here. So, I will show 2 steps not quite proportional but that is ok. This is w 0 w 1 will essentially take this from 2 pi to 4 pi w 2 will cover 4 pi to 8 pi here. This is 8 pi please note again as I said forgive my drawing it is not quite proportional, but it is indicative pi here 2 pi here 4 pi there 8 pi there. Now, we know what we are doing? As we go up the ladder we are going to double the center frequency each time and double the bandwidth ideally and once again we are going to show the behavior as far as these spaces v go. So, here we show what happens with w let show what happens with v. So, I could show it just on 1. So, I will just show for completeness 3 of them this is what v 0 does this is what v 1 does. So, v 0 is the solid line v 1 is just the dashed line and v minus 1 is the dot dashed line that I am drawing now. So, these are what are called the complete subspaces these are well I should not use the word complete in the rigorous sense, but I mean these are the entire set of shells up to that shell. And the others which we drew a minute ago the w's were just one peel or one shell at a time. Now, we understand perfectly what we are doing in frequency we are trying to do and now we also understand perfectly where the challenge lies. We are aspiring to do this and we also want to do something similar in time. We want to confine ourselves to a certain region of time and we also want to focus on a particular region of frequency. Ideally focusing means being only in that region and 0 outside. So, the first question that we need to answer is is it exactly possible can we be compactly supported in time and frequency simultaneously. Let us put the question question is also important here can we be compactly supported. This is a technical term compactly supported I should not spend too much of time on explaining the details, but non-zero strictly on a finite interval is a simple way of saying it at the moment simultaneously in time and frequency. Unfortunately or may be fortunately because it brings up and opens up a whole new subject the answer is no. If you talk about exact behavior it is impossible to be compactly supported in both domains. That is not a very deep result in the theorems of Fourier analysis though it is an important result. It is the relatively weaker weaker in the sense not of requirement, but in terms of the depth of proof or depth of implication. It is more easily proved easier to indicate or to justify. You cannot be compactly supported in time and frequency simultaneously. Let us make that statement very clear answer no. A function and its Fourier transform cannot both be compactly supported. In fact, I shall give an indication of the idea behind the proof and I shall leave it to the class I shall leave it to the students who are listening here to Delve Daper. The idea behind the proof why not? Well suppose X f suppose X t has the Fourier transform X cap f or X cap omega. Let us take omega if you like. X cap omega be compactly supported. In other words let us specifically X cap omega be non-zero only between omega 1 and omega 2 in magnitude. Of course, needless to say omega 1 is greater than equal to 0 and therefore also omega 2. Then it is very clear that the Fourier transform the inverse Fourier transform which gives us back X t is a finite integral. So, X of t is then going to have a finite integration involved plus the same thing on the negative and the same integrand. Now the central idea in the proof is the following. I can take derivatives on both sides and I remember I have a finite integral on both sides. When I take a derivative with respect to time of X t then if I look at the integral here that derivative essentially acts only on e raised to power j omega t and that operation of taking the derivative into the integral is valid because this is a finite integral. The same thing holds good for the second integral here. So, in effect you are talking about a function which has an infinite number of derivatives because after all each of the integrals involved would be a finite integral here. So, I have just I am not really giving you a rigorous proof. I am just indicating the central idea in the proof. It relates to the fact that the function which is compactly supported in the frequency domain must have a certain kind of smoothness as seen in time. No matter how many derivatives you take here you do have an expression for the derivative there. That means the derivatives exist and in fact can also be shown to be continuous. So, there is a there is the quality of infinite smoothness in that function X t in some sense. As I said all this is only indicative of the proof. Now I encourage those of you who are more mathematically minded to take this proof to completion. Show that because of this finite integral here and the fact that the function must be smooth as much as you desire in terms of derivatives it cannot be compactly supported in time as well. In effect what you are saying is you are asking for an analytic function a function which has an infinite number of smooth derivatives to be compactly supported in time. There is a problem there well that was indicative of the proof that was indicative of the central idea as to why you cannot have compactly supported functions both in time and frequency together and this is where the whole challenge starts. But now we need to ask a slightly more relaxed question and that will be the issue that we shall discuss in much greater depth in due course now. The question is suppose we do not ask for strict compact support. That means suppose we are not saying that a function must be non-zero outside sorry non-zero only inside a certain compact interval only inside a certain finite interval and 0 everywhere else. We do not mind a certain amount of energy of that function or most of the function in a certain sense being concentrated in a certain region in time and also in frequency. Then can we get a function which is both compactly or not compactly but in that sense restricted in time and frequency. And of course as we expect the answer is yes if we are willing to give up a little bit we can get something. If you are willing to give up exact compact support. So, if you are willing to allow some leakage outside that region of time and therefore also outside a certain region of frequency. But be content with the fact that in a certain weaker sense the function is concentrated in a certain region of time and in a certain region of frequency. Then can one first have this kind of broad concentration in time and frequency together. Well the answer is yes because it depends on what you mean by that weaker sense of concentration. In fact phi and psi are in that sense concentrated both in time and frequency in a weaker sense. If you focus only on the main lobe and of course the main lobe has a certain amount of the energy then yes indeed of course phi is simultaneously localized in time and frequency. So, what is that general sense that we are going to allow. Well that sense will come from essentially either what we might call the statistical property of variance or if we want to use a mechanical analogy the idea of center of mass and radius of gyration or the volumetric occupancy of a body. So, we will think both of the function and its Fourier transforms as one dimensional bodies and we can think of their center of masses. And then we could think of how much the body spreads around the center of mass by using what is called the idea of radius of gyration. Another perspective is if you think of probability density functions based on the function and its Fourier transform. You could ask what is the mean of that density either in the time domain or in the frequency domain. And then you could ask what is the variance of the density again either in the time domain or in the frequency domain. And now there is a clear way to formulate can we have finite variance both in time and frequency and there as we expect the answer is going to be yes that is not a problem. Now, the more difficult question how small can the function be simultaneously in time and frequency in this broader sense. So, how small can you make the variance in time and frequency simultaneously that is the deeper question and that is the whole idea behind the uncertainty principle. In fact, now we are beginning to understand why we need it to go to better and better multi resolution analysis. Why could we not be happy with the Haar? The Haar is somewhat concentrated in frequency, but well concentrated in time. I had one point asked you to find out the Fourier transform of the Dobash functions as well. So, you know if you look at the Dobash functions as you go from length 4 to length 6 to length 8 and if you look at the Fourier transforms you would find that they are slightly better approximations to that ideal low pass filter with cut off pi and ideal band pass filter with band between pi and 2 pi as we desired. So, what we are going to do subsequently now is essentially to essentially bring out this concept of uncertainty more deeply and then to investigate whatever we have been doing in the language of uncertainty starting from this point onwards. Thank you.