 A very warm welcome to the eighth lecture on the subject of wavelets and multirate digital signal processing. In this lecture, we shall build more intimately the connection between the filter banks that we talked about in the previous lecture and the underlying continuous time functions. The scaling function phi t and the wavelet psi t, we suspect all the while that this connection exists. After all, we built the filter banks out of the idea of multi resolution analysis with the hard multi resolution analysis as an example. Before we go further, we must now make a few generalizations which will help us then to build that connection more intimately. So, let me quickly put down what we intend to do in today's lecture. We intend today to relate the scaling function phi, the wavelet psi and the filters that we talked about in the filter banks namely the analysis and the synthesis filter banks. Towards that objective, the first step is putting down a generic structure for the analysis and the synthesis filter banks. So, let us consider the generic structure for the analysis filter bank first. Incidentally, we should be talking about a two band filter bank here and that is because we are talking about dyadic MRAs here. Recall that dyadic refers to powers of two changes. So, what we are talking about is the generic structure first of a two band filter bank, two band analysis filter bank. Now, I will start from the Haar. In the Haar, it looks like this, Haar MRA I mean. It looks like this, recall. This was the structure and we also analyzed the frequency domain behavior of these two filters. In fact, this turned out to be a crude low pass filter and this turned out to be a crude high pass filter. Of course, high pass and low pass as understood in the discrete time sense. Now, we also recall two other properties of these two filters that we had brought out the last time. One was what we call magnitude complementarity. Magnitude complementarity in the sense, if we simply summed the filter system functions together, you got the identity system function one. The second was what was called power complementarity. So, if you sum the magnitude square, you got a constant. A constant independent of omega I mean, which means if you took a sine wave and looked at the power of the sine wave emerging from each of the two branches, those powers would add up in a complementary way for each frequency. Magnitude complementarity, power complementarity. Now, you know, if we looked at the Haar case, it was not very clear which idealization we were moving towards, but then we had put down the idealization the last time. So, let us now put down the complete idealization. We have a crude low pass filter there. What would be the refined low pass filter towards which we are moving in this analysis filter bank? Now, when we say towards which we are moving, what do we mean? Why should we move? Why cannot we be content with the Haar multi resolution analysis? That is also a question that we need to answer. We will have to take the answers to these questions one by one. So, let us first answer the question, what is the idealization towards which I am trying to move? So, let me put down first the actual frequency responses once again and then the ideal frequency responses towards which we are trying to move. So, the actual Haar analysis filter responses, the 1 plus Z inverse by 2 filter had a response that looks like this, essentially mod cos omega by 2 between 0 and pi. Of course, recall that there is periodicity and the 1 minus Z inverse by 2 filter had essentially a response that looked like mod sin omega by 2 and you will recall that there was magnitude and power complementarity here. In fact, cos square plus sin square is 1 and therefore, there is power complementarity and if you just add up these system functions together 1 plus Z inverse by 2 plus 1 minus Z inverse by 2, you get 1 and that is magnitude complementarity. Now, you know if you look at these two responses and if you mark them around the centre, the centre is pi by 2. So, if you mark them around the centre, you see a certain symmetry in these responses about the centre. That gives us a hint where we are moving towards in the ideal sense. Ideally, we are trying to make this a low pass filter with pi by 2 as the cutoff and again we are trying to make this a high pass filter again with pi by 2 as the cutoff. So, let us put down the ideal analysis filter responses towards which we intend to move. So, the low pass filter should have a response that looks like this between 0 and pi. It must be 1 between 0 and pi by 2 and 0 between pi by 2 and pi and this entire pattern must be mirrored between minus pi and 0 and of course, then repeated the whole pattern between minus pi and pi, then repeated at every multiple of 2 pi. All this is of course, naturally from the properties of a discrete time Fourier transform or a discrete frequency response. So, I am just showing the region between 0 and pi and what happens between minus pi and 0 and then at around every multiple of 2 pi follows naturally. So, this is the low pass filter response. Similarly, the high pass filter response which I will draw here. The high pass filter needs to have a response of 1 between pi by 2 and pi and of course, whatever is between 0 and pi is mirrored between minus pi and 0. The response is 0 between 0 and pi by 2 and then of course, periodically repeated at every multiple of 2 pi. Let us call this LPF ideal. So, LPF ideal is the high pass filter response between and high pass filter HPF ideal. So, these are the frequency responses of the analysis filters towards which we desire to move. Let us now put down the ideal frequency responses of all the filters, analysis and synthesis in a two band filter bank. So, where are we moving? In what direction are we moving? So, the ideal two band filter bank would have the following structure. The input sequence here being fed to LPF ideal here, HPF ideal there, down sampled by 2 subsequently. This is the analysis portion followed by the up sampled by 2, again low pass filter ideal here, high pass filter ideal here and the outputs from this are to be added to produce the overall output. So, here I have just marked the outputs to fit into the drawing, the output from this and the output from this are added together and produce the overall output. This is the ideal two band filter bank. Notice in the ideal two band filter bank, the analysis filters and synthesis filters are identical. There is no difference. In fact, on one of the branches, both of them are ideal low pass filters with a cut off of pi by 2. On the second branch, both of them are ideal high pass filters, discrete time filters with a cut off pi by 2 once again. So, you know you see complementarity there in some sense and in fact, if you look at it, these are obviously magnitude and power complementary. If you take the frequency responses and add them together, they of course, add up to a constant. If you take the square of the frequency responses and add up the squares, they again add up to a constant, trivially because there is no region of overlap. That is simple. So, in fact, that is where we are moving towards. Now, it is a moot point so far as I said, why we need to move any farther from where we are? Why do we need to work harder than what we do for the hard wavelet? Are we lacking something in the hard wavelet? Well, of course, one thing that you can see we are lacking is the distance from the ideal filter. We are far from the ideal. If we look at the two frequency responses of the harsh analysis side and therefore, also the synthesis side, I had left it as an exercise to calculate the frequency responses for the synthesis side. They are almost the same. We are I mean we are very far from the ideal. So, that is of course, clear. But why we need to move farther from the hard can be answer in many ways. We shall take up this answer slowly part by part. But before we take up that answer, what we need to do is now to go the other way. We came from continuous time to discrete time. Now, slowly we want to see if my design of a multi-resilient solution analysis relates to the design of the two band filter band that I have just drawn here. So, you see we have ideal filters here and we have written down the frequency responses of the ideal filters. But in practice, the ideal filter can never be attained. Now, you know without going too deep into each point, I would like once again to recapitulate why the ideal filters cannot be attained. So, it has nothing to do with technology or the lack of computational power. There is something fundamentally troublesome about these ideal filters that makes them unachievable or unattainable. Let me cite these points one by one for the sake of completeness and revision from a more basic course. So, why are these ideal filters unattainable? Well, the first reason is that the ideal filters are infinitely non-causal and this can be checked by constructing the impulse response. So, from the frequency response, we can go to the impulse response. We also know how to do that. We can go to the impulse response by taking what is called the inverse discrete time Fourier transform. So, if you have the ideal frequency response, let us call it H ideal as a function of omega and the corresponding impulse response can be obtained by the inverse DTFT of this and how do you calculate the inverse DTFT? I leave this calculation for the class to do, but I would like to put down the important steps here. So, that is done as 1 by 2 pi integral from minus pi to pi H ideal omega e raised to the power j omega n d omega. In fact, I leave it to the class to compute this for the ideal low pass filter and the ideal high pass filter with a cutoff of pi by 2. I shall just put down the answer for the ideal low pass filter. So, for the ideal low pass filter, this would turn out for example to be sin pi by 2 n divided by pi n wherever n is not 0. So, this is the impulse response H n and it will be half for n equal to 0. I leave it as an exercise to verify this integral and I also leave it as an exercise for the class calculate the ideal high pass filter impulse response. I leave it as an exercise easy to do once one has a basic introduction to the inverse discrete time Fourier transform from a basic introduction to discrete systems. Anyway, the point was if we looked at the impulse response, there are three things that forbid one from realizing this filter. Coming back the first thing was infinite non causality the disqualifications of the ideal filter point number one infinite non causality. Now, when is a discrete linear shift invariant system causal, it is causal if the impulse response is 0 for all negative values of the integer index. So, H n is equal to 0 for all negative n is the requirement necessary and sufficient for causality. In this case of that matter in the case of any ideal filter, you could take either the high pass or the low pass filter in this case for both of them you would notice. It is infinitely non causal meaning that even if I were to delay the impulse response sequence by a few samples any finite number of samples you could never make it causal. So, it is infinitely non causal a serious disqualification which means that if you wish to realize a causal filter bank you cannot. So, you know it requires anticipatory behavior if you are doing it in time, you need to use the future to work in the present a strange situation to be in. Now, you know non causality by itself is not always a disqualification it is infinite non causality which creates a problem infinite non causality means one cannot make the filter causal by introducing some delay that is what is a disqualification here disqualification number one disqualification number two the system is unstable that is if you look at summation and n mod h n it is divergent a terrible thing to happen. Now, this is also indicative of a subtle point you know just because it is a filter does not mean it has to be stable. What it means you know the moment you say filter the moment you say it has a frequency response what it means is that when you give a sinusoidal input a bounded input of course, the sinusoidal input the output is bounded in fact the output is a sinusoidal of the same frequency, but that is to do with sinusoids you could very well have some other peculiar input where the output is unbounded now that is troublesome. So, you know you could have a situation where you get a certain behavior as far as sine waves go, but you are not guaranteed that the output will always remain within bounds for a bounded input a bounded input may not produce a bounded output the system is unstable a discrete time system is stable if its impulse response is absolutely summable of course, we are talking about LSI systems here linear shift invariant systems. So, this linear shift invariant system whose frequency response corresponds to the ideal low pass filter whether cut off pi by 2 or any other cut off or for that matter a ideal an ideal high pass filter all such systems are unstable which can be shown by showing that the impulse response is not absolutely summable if you try and calculate the absolute sum of the impulse response it would diverge again I leave this as an exercise for you to show take the ideal impulse response of the low pass filter for example, with a cut off of pi by 2 and try and show that its absolute sum is diverged not very difficult but an interesting exercise disqualification number 2 unstable disqualification number 3 a serious one 2 the ideal filter is irrational and what does this mean now let me first explain the meaning of irrational literally you see if we say a filter is rational or a linear shift invariant system is rational if you look at the system function that means the z transform of the impulse response and find that it can be expressed as a ratio of 2 finite series in z. So, numerator a finite series in z denominator a finite series in z if a system function of the LSI system could be expressed as a ratio of 2 finite series in z we say this system is rational. So, of course, when we talk about rational system we are automatically talking about linear shift invariant systems it is only linear shift invariant systems which could be rational or irrational and rational or irrational refers to systems who have a system function linear shift invariant systems who have a system function which means the z transform of the impulse response exists in some non null region of convergence. Now, the z transform of the impulse response of the linear shift invariant system could either be rational which means it is a ratio of 2 finite series in z or it could be irrational in which case it is not. Let me give you an example of an irrational system function the example of an irrational system function could be e raised to the power of z inverse with mod z greater than 0 as the region of convergence. And of course, the corresponding impulse response can easily be seen to be 1 by n factorial u n where n factorial is defined by 0 factorial is 1 and n is n factorial is equal to n times n minus 1 factorial recursively for n greater than equal to 1. Interestingly this system is stable. So, it is not that all irrational systems are unstable, but irrational systems have a fundamental problem. Irrational systems are unrealizable at least today. We do not know any neat way of realizing irrational systems. Rational systems can be realized with a finite amount of resource. What do we mean by resource adders, multipliers, delays? These are the basic resources of a discrete time realization. A rational system can be realized with finite resource. An irrational system in principle requires an infinite amount of resource to realize. Now, you know this is all falling in place. The non causality, infinite non causality and instability came as a bit of a surprise, but this third disqualification is not quite a surprise. What we are saying in effect is that if you want an ideal filter be willing to put in infinite resources to get it. You know this is the whole, I mean I would say actually irony of discrete time system design. I keep mentioning this whether it is a basic course or an advanced course like this. The irony of many design problems is that you know which ideal you are striving towards and you also know that you cannot achieve that ideal with finite resources. But you know what keeps engineers and mathematicians and scientists and what have you active all the time is that you also know that you can go arbitrarily close to the ideal provided you are willing to invest more and more resources. And there are many ways of doing it. There are different ways of investing resources and going closer to the ideal. Perhaps some parts take you closer to the ideal faster at least in a certain range of resources and some parts slower. And again there are compromises. If you go faster in one sense, you may go slower in the other sense. Nature drives the engineer, scientist and mathematician to no end. Anyway, that was just a philosophical diversion. Coming back to this problem, the ideal two-band filter bank is unrealizable. But we can go tantalizingly close as we desire. So, you can build a two-band filter bank arbitrarily close to the ideal if you are willing to invest more and more resources. And over the past 15 to 20 years, people have come out with so many different designs. Now, why have people sought these different designs? Again, that question needs to be answered. But then, now we will first answer the easier of the two questions. Suppose I do happen to design different two-band filter banks. So, first what would a realizable two-band filter bank look like? We must first put that, we must write that down in terms of a drawing first. So, a realizable two-band filter bank is like this. So, I straight away write it in terms of system functions here. This is a realizable two-band filter bank provided H0Z, G0Z, H1Z, G1Z are all rational functions, rational system functions. And of course, H0Z and G0Z aspire to be ideal low-pass filters. So, this is a realizable two-band filter bank. With cutoff pi by 2, H1Z and G1Z aspire to be high-pass ideal filters with cutoff pi by 2. Now, having put down the structure more generally, we must now ask what is the connection between the multi-resolution analysis and this filter bank that we are trying to design? So, to answer that let us look at the Haar once again. In fact, let us begin with phi t in the Haar. So, for the Haar MRE, we notice something very interesting. Phi t belongs to V0, which is a subspace of V1. So, phi t should be expressible in terms of the basis of V1. And what is that basis of V1? It is phi 2 t minus n for integer n. It should be expressible in these terms. It is very interesting. You know phi t is an element of V0. V0 is a subspace of V1 and the basis of V1 is again the dilates of phi t by a factor of 2 and their integer translates. So, phi t can be expressed in terms of its own dilates and translates. This leads to what is called a recursive dilation equation on phi t. And lo and behold, what is that dilation equation? That is also not difficult to determine. In fact, we can even see it graphically. Indeed, you know if you recall, phi t looks like this. I have kind of scaled up the drawing. And all that you need to do to get this recursive equation is to notice that this can be redrawn like this. So, it has two components in it. And the first component is phi 2 t, the second phi 2 t minus 1 and the whole thing is phi t. So, we have a beautiful dilation equation which governs phi t. Phi of t is phi of 2 t plus phi of 2 t minus 1. A beautiful dilation equation which governs phi t. Now, let us look at the coefficients in that dilation equation. So, let me put the dilation equation before you once again. The coefficients are 1 and 1. Let me try and call this a sequence. So, if you agree, I will talk about the sequence which is 1 at n equal to 0 and then 1 subsequently as shown at n equal to 1. This is a way of denoting a finite length sequence. The number below the arrow tells us the value at the point of the arrow and all other numbers tell the value of the sequence at adjacent points. So, this, for example, means at n equal to 0, the sequence takes the value 1 and at the next point which is of course, n equal to 1, the sequence would take the value 1, 2. So, this is the sequence corresponding to the dilation equation coefficients. Let us carry out a similar exercise for the wavelet now. So, let us take the har wavelet and let us express the har wavelet also in terms of the basis of v 1 and what is our ground for doing so? Let us put it down clear. You see, recall that psi t or the har wavelet, for example, also belongs to v 1. So, it should be expressible in terms of its basis. What is that basis? Phi 2 t minus n for integer n and again if we look at it graphically, it is not difficult to do. Graphically, one can sketch phi t actually and psi t 2. So, this is psi t and you can see the phi t is embedded in it. So, you have one here and you have one there. This is easily seen to be phi of 2 t and this is easily seen to be minus phi 2 t minus 1 and therefore, we have a very simple dilation equation for psi t. Now, here it is not recursive, but it is a dilation equation all the same. So, dilation equation for psi t, psi of t is phi 2 t minus phi 2 t minus 1 and once again let us put down the coefficients of this dilation equation as we did previously. So, you know the coefficients again would be the coefficients involved in expanding in terms of phi 2 t minus n. As I can see from this dilation equation, the coefficients are 1 and minus 1 respectively at 0 and 1. So, I put that down. Now, things are beginning to make sense and perhaps even ring a bell. Let me put both these coefficients sequences before you once again and then I am sure it will ring a bell. Look at the dilation equation coefficients for the phi t itself. So, in fact, let me write it down. Dilation equation coefficients for phi t and let me then put before you the dilation equation coefficients for psi t. So, let us write it down in this for psi t. Does this ring a bell? Yes indeed. If you look at the impulse responses either on the and this is side or the synthesis side of the low pass filter and the high pass filter. These coefficients are essentially those impulse responses. A minor difference you see on the analysis side you have a factor of half. For the moment keep aside that factor of half otherwise these dilation equation coefficients are just those very impulse responses. Synthesis side similarly perhaps with a plus minus ambiguity, but otherwise the same. So, we have very intimate relation which we have seen here. The coefficient sequences in these dilation equations that govern phi t and psi t are actually the impulse responses of the filters. Now, in fact I will go one step further. We shall now progress to show that if I know these impulse responses I can go the other way too. So, here I have by serendipity so to speak by surprise or chance discovery come up with this relationship. Now, we will take that serendipity that discovery further. So, indeed let us note at the moment that within a scaling factor if h of n so to speak is the impulse response of the low pass filter in question in the two band filter bank. Then essentially what we have is the following dilation equation. So, we will continue the dilation equation phi of t is summation on n, n over the set of integers h n over the set of integers phi 2 t minus n. This is the essential dilation equation for phi and conversely if g n is the impulse response of the high pass filter in the two band filter bank. Then we have psi t is summation n over the set of integers g n phi 2 t minus n so in the time domain we have made an intimate relationship. The low pass filter impulse response allows us to expand the essentially phi t in terms of its own dilates and translates the scaling function in terms of its own dilates and translates. The high pass filter helps us expand the wavelet in terms of the dilates and translates of the scaling function. Once again low pass filter impulse response a recursive expansion of the scaling function in terms of its own dilates and translates. The high pass filter impulse response and expansion of the wavelet in terms of the dilates and translates of the scaling function. Now, we want to go a step further, we want to show that once you have this dilation equation, we can actually completely characterize phi t and psi t knowing the two band filter bank. I shall in the next couple of minutes only give the strategy for doing so, but we shall actually do this in the succeeding lecture, in the lecture to follow. So, let me put before you the strategy that we are going to follow and for that purpose, let me put down the equations before you once again. So, let us recapitulate the two equations we have written. We have this dilation equation relating phi t to its own dilates and translates. What we shall do in the next lecture is to take the Fourier transform on both sides and noting that we can express the Fourier transform of phi t or rather phi 2 t minus n in terms of that of phi t. We shall have a recursive equation in the Fourier domain on the Fourier transform of phi. From this, we shall be able to completely characterize the Fourier transform of phi in terms of the discrete time Fourier transform of the sequence h. Having done so, we shall then progress to this dilation equation here and we will relate the Fourier transform of the wavelet to the Fourier transform of the scaling function effectively and then noting that the discrete time Fourier transform of the sequence g n can be used to make this relationship. We shall obtain the wavelet from the scaling function. So, it is with these two steps that we shall begin the next lecture. For the time being, let us keep our curiosity alive to see how beautifully we can enmesh the design of the two band filter bank and the design of a scaling function and a wavelet for building a whole multi-resolution analysis. With that note of curiosity and anticipation, let us conclude this lecture. Thank you.