 A very warm welcome to the 7th lecture on the subject of wavelets and multirate digital signal processing. Recall that we had ended the previous lecture by hinting at the structure of the synthesis filter bank corresponding to the higher multi resolution analysis. Today we begin by looking at both the analysis side and the synthesis side once again. I mean the analysis filter bank and the synthesis filter bank in the higher multi resolution analysis. So, let us put down the two filter banks clearly. You will recall that on the analysis side we had a structure like this. We had the sequence let us call it x of n corresponding to the function in v1. Essentially it was subjected to the action of two filters. A filter of the form half 1 plus z inverse and another one of the form half 1 minus z inverse followed by a down sampling by 2. We had also derived the structure of the synthesis filter bank and interestingly we saw the structure was very similar. In fact it looked almost like a mirror image here except for the fact that you had an upsampler instead of a down sampler followed by the two filters once again. So, here the filters had the form 1 plus z inverse and 1 minus z inverse. Now the only catch is I am going to allow for an ambiguity of sign here plus minus 1 minus z inverse a small variation from last time. A subtle point but important. Now there are several things to which we must pay attention. One is if you look at the structure of the analysis and the synthesis filter banks the filters are almost identical. These are the two filters for the analysis side and these are the two filters for the synthesis side. Please note I am allowing an ambiguity here. What would the physical meaning of the ambiguity be? It would just determine where I place the sum sample and the difference sample. With a plus sign the sum sample will get placed at the even locations and the difference sample at the odd locations. With a minus sign it would be the other way around. So, let us leave that ambiguity for the moment. If we try and resolve the ambiguity right away we might actually get more confused. So, allowing that little ambiguity for the moment let us analyze this whole structure in general and then resolve the ambiguity. Anyway coming back to the filter bank structure as we see in one very beautiful thing that we notice about the Haar filter bank is that the filters on the analysis side and the synthesis side are almost the same except for this sign ambiguity. And if we really look at it these two filters are also very meaningful in another domain of which we shall soon have more information. Now what we are going to do from here onwards is to look at this general structure. You see we want to understand why the Haar multi resolution analysis though attractive is not adequate. I have been saying all the while that if we understand the Haar multi resolution analysis and if we understand the filter bank corresponding to the Haar multi resolution analysis. We understand quite a great deal about filter banks and multi resolution analysis most of the concepts are captured. Then why should we look for other multi resolution analysis that is a subtle point which we shall see by going into another domain. So, our objective here is to look at the frequency domain behavior and how would we get a frequency domain representation of the Haar filter bank. So, what we will do is we will progress as follows. First we shall look at the frequency domain for the filter bank and how would we do that? We would do that by substituting z equal to e raise the power j omega. Let me once again recapitulate a few concepts from discrete time signal processing for the benefit of the class although one would have done these in a course on discrete time signal processing. It helps to put some of this discussion in perspective. You will recall that we had defined the z transform last time. We had said that if you have a sequence x of n it is z transform is capital X of z defined by summation n running from minus to plus infinity x of n z raise the power minus n with z belonging to a region of convergence. Now, in particular if the region, so let this region be script R in particular if this region includes the unit circuit. So, you know let me say a little bit about the regions of convergence of a typical z transform. Typically R has the following patterns R is between two circles centered at the origin. So, it has an appearance something like this and both of these are centered at the origin. So, you have R2 and R1. So, you know here the radius is R1 I mean here the radius is R2. Now, it could be true that R2 might be infinity or R1 might be 0. So, we have to allow these possibilities. Most generally R1 could be 0, R2 could be infinity. Moreover, the boundaries may or may not be included. The boundary circles may be included or excluded. In fact, if these values R1 and R2 are non-zero and finite most of the time the boundaries are excluded. It is only when R2 becomes infinity or when R1 becomes 0 that there is a question. Is the boundary included or excluded and that is not a trivial issue. Whether the boundary is included or excluded makes a difference to the properties of the system. Well, all this essentially to recall a few points about the z transform and now coming to the frequency domain. So, in particular unit circle that is mod z equal to 1 it is a circle you see mod z equal to 1 is a circle of radius 1 in the z plane. So, if the unit circle is included in R then we have a frequency response for the system. The system has a frequency response that is you see we talk about the system having a frequency response if the in the underlying z transform is a system function. If the underlying z transform is a system function and of course, otherwise in case we are talking about a sequence then we say the sequence has a discrete time Fourier transform. So, if we are talking about the z transform of a sequence we say the sequence has a discrete time Fourier transform. This is an important concept a DTFT for short either way. What we are saying is in case the unit circle is included in the region of convergence then we have an even more interesting interpretation of the system function or the sequence in question. And in fact, if we look at all these z transforms encountered in the Haar filter bank they all satisfies. So, we are now entitled or we are now well poised to look at the Haar filter bank in the frequency domain and what do we mean by the frequency domain with mod z equal to 1 or z equal to e raise the power j omega. Indeed consider the analysis site the analysis filters both of them have a frequency response obviously. Let us take the filter 1 plus z inverse by 2 and let us look at its frequency response. Of course, we would obtain the frequency response by substituting z equal to e raise the power j omega and that would give us 1 plus e raise the power j omega by 2 minus j omega by 2 if you like which we can simplify. We can take an e raise the power minus j omega by 2 common here and then put e raise the power j omega by 2 plus e raise the power minus j omega by 2 inside the bracket and leave the half factor as it is. So, this is what we have here. Now, recognize that this is essentially 2 times cos omega by 2 and therefore, we have half e raise the power minus j omega by 2 2 times cos omega by 2 which gives us e raise the power minus j omega by 2 times cos omega by 2. This is the frequency response. Let us sketch the magnitude and the phase of this frequency response and recall that for discrete systems it is adequate to sketch the magnitude and phase of for that matter to determine the magnitude and phase response in the region omega going from minus pi to plus pi. We do not need to go outside that range because after all the frequency response is periodic with a period of 2 pi. So, whatever occurs between minus pi and pi is going to be repeated around every multiple of 2 pi. So, the magnitude of this response would be mod cos omega by 2 and let us confine as I said to omega between pi and minus pi. Whereupon cos omega by 2 turns out to be non-negative in fact, essentially cos omega by 2 is real and greater than equal to 0 in this region. Thus phase contribution from this term is 0. So, if you look at the overall frequency response here namely e raise the power minus j omega by 2 times cos omega by 2. This has no phase contribution for minus pi less than equal to omega less than equal to plus pi and the phase contribution comes only from here. The phase contribution from this term is going to be equal to minus omega by 2. So, now, we can sketch the phase and magnitude contributions very clear. Let us sketch the magnitude response. The magnitude again you know this is a real filter, this is a filter with a real impulse response. Now, when an impulse response is real the frequency response is conjugate symmetric. So, whatever is at omega is also at minus omega in magnitude, whatever is at omega in phase is the negative of what is there at minus omega in phase. So, we need only sketch the magnitude response and the phase response between 0 and pi. We can ignore minus pi to 0 because there is going to be conjugate symmetry. So, let us sketch it only between 0 and pi. You can see that this is the kind of pattern that is going to show essentially of the form cos omega by 2 and for the moment we can already see that this is something like a crude low pass filter. I say it is a low pass filter because it emphasizes the lower frequencies and deemphasizes the higher frequencies. I say it is crude, it is far from ideal. In fact, if you had an ideal low pass filter then of course, you need to specify the cutoff 2. So, an ideal low pass discrete time filter with a cutoff of pi by 2 would look something like this. I mean in the magnitude sense. So, it would have a magnitude response that looks like this and ideally the phase response would be 0. So, it would be 1, the response itself would be 1 between 0 and pi by 2 and 0 between pi by 2 and pi and of course, mirrored as it is between minus pi and 0. So, ideally we would have this magnitude and phase equal to 0. Where are we in reality? We are trying to replace this ideal filter here with this crude approximation to the low pass filter here. So, please remember this is the ideal towards which we are striving and this is where we have reached in some sense with this very simple Haar multi resolution analysis or Haar filter bank. Now, we shall understand this even better when we look at the magnitude response of the high pass or the other filter in the filter bank, but for the moment let us complete our discussion by also drawing the phase response. So, the phase response of this filter is as follows. The phase response as we noted was minus omega by 2. So, in the region from 0 to pi at pi it would of course, take on the value of minus pi by 2 and this is a straight line. So, in fact, you get what is called linear phase. Now, linear phase is something very attractive in discrete time signal processing. In fact, one of the reasons why people go to discrete time signal processing from analog signal processing is because you can get linear phase. Why is linear phase important? Let us spend a minute in reflecting on this. You see what is the phase response denote or for that matter what is the frequency response denote? The frequency response tells us what happens to a sine wave when it passes through the system. So, in fact, to understand this better let us go to continuous time first. Suppose we fed a sine wave let us say cos omega naught t plus phi naught times a naught. So, amplitude a naught frequency capital omega naught phase phi naught to a continuous system to a continuous time filter with frequency response. Let us say h as a function of omega what would come out? Very simple. You would evaluate the frequency. This is of course a complex number as a function of capital omega. You would evaluate this at omega equal to omega naught. The magnitude would multiply the magnitude. The angle would add to the angle here. So, in other words what would come out is mod h evaluated at omega naught a naught cos omega naught t plus phi naught plus angle of h evaluated at omega naught. What is the effect of this angle? As you can see the effect of this angle is to change the phase and a change of phase is equivalent to a change in time. So, it is like shifting the time shifting the sine wave on the time axis. So, angle h omega naught essentially shifts the sine wave shifts the sinusoid by well you know t is to be replaced by t plus something and that plus something is given by this divided by omega. So, I mean let us write it out explicitly that way. Let us reason out what it should be explicitly. So, we rewrite that expression. We write it as mod h omega naught times a naught cos we keep phi naught as it is. We will take omega naught common and then we will write t plus the angle of h omega naught divided by omega naught. I am sorry. Well, I will write omega naught t plus this into omega naught. So, again you know you have angle here. So, this into omega naught I will write it this way. So, now I can rewrite. So, forgetting about this part see this part is essentially a change of magnitude. It is only this part which I wish to focus on t has been replaced by t plus angle h omega naught by omega naught. So, essentially this quantity angle h omega naught divided by omega naught is like a time shift. Is that right? It is an important observation we make. Now just in case it is independent of omega naught we have a good situation. So, you know this time shift actually is what is called a necessary evil in the frequency response. You know most of the time when we give specifications for designing a filter whether it is in the analog domain or in the discrete time domain. We do not really want a phase response. The phase response comes as a necessary evil and we have to work around it. Now what does the phase response do? At each frequency it creates a time shift. So, the phase response creates a time shift dependent on frequency. What is a good situation to have? An ideal situation is no phase response. This is unachievable. In fact, it is unachievable because of causality. You know if you want the filters to be causal then you cannot ask for zero phase. Incidentally that is why you can use zero phase filters in dealing with images, in dealing with two dimensional data because in two dimensional data causality is not an important requirement. But when you are dealing with one dimensional data in time causality is essential. You cannot avoid it and therefore you cannot avoid a phase response either. Now if you must live with a phase response what kind of a phase response would you like to live with? You would like to make sure the phase response does not treat different frequencies differently. So, if it has to shift a sine wave in time so be it but then shift all sine waves by the same time and that is exactly what we are saying when we talk about linear phase. Linear phase means angle h omega naught by omega naught is independent of omega naught. That means it is a constant. Let us say it is some tau or tau naught if you like which means that the angle of h omega naught is of the form some omega naught times tau naught. It is linear in tau naught in omega naught. That is why it is called linear phase. Now this is what I am trying to emphasize at this point. This very simple filter bank, the Haar filter bank has this beautiful property of linear phase. By the way this is something unusual in filter banks. It is not easy to design filter banks of larger order with linear phase. In fact, only one class of filter banks has this property. Linear phase in some sense has to be compromised with something else. So, in the Haar you can have your cake and eat it too. So, you can have linear phase, you can have orthogonality. But later we will see that if you know if you want to get something else out of your filter bank, if your filter bank must be better in some sense you have to sacrifice linear phase. Anyway, so far so good we have linear phase. We are not doing badly and if you look at the second filter in the Haar filter bank, we shall have something similar. So, let us look at the second filter, the second analysis filter and that is of course 1 minus z inverse into half. Let us again find out how this filter looks in the frequency domain. The frequency domain would show it as z equal to e raised to power j omega, whereupon we have 1 minus e raised to power minus j omega by 2 and we play the same trick. We take e raised to power minus j omega by 2 common and we have e raised to power j omega by 2 minus e raised to power minus j omega by 2 and once again one can recognize this is essentially 2 j times sin omega by 2. So, now I can simplify this, put it all together. Once again I look at the magnitude response first. The magnitude response is the magnitude of this and that is easily seen to be mod sin omega by 2. Let us sketch the magnitude response as a function of omega. Again, we will sketch it only between 0 and pi for the reasons that I have just explained. So, it will have an appearance something like this. This is going to be 1 here. This is mod sin omega by 2. Now, for the phase response, now please remember last time we had a convenient situation. We had two terms, one of them contributed no phase, the other one contributed a phase. So, we were comfortably put. This time we have to be a little careful. So, you know let us make life easy by first looking at only 0 to pi. Remember we are going to have conjugate symmetry. So, let us consider omega from 0 to pi and let us look at the frequency response expression. J e raised to the power minus j omega by 2 times sin omega by 2. Sin omega by 2 is non-negative. So, no phase contribution here. However, both of this term and this term have a phase contribution. In fact, the phase contribution is 90 degrees or pi by 2 from here and minus omega by 2 from here. So, overall the phase contribution or the phase response is I mean only between 0 and pi, pi by 2 minus omega by 2. This is contributing essentially j in the expression and this part is coming from e raised to the power minus j omega by 2 in the expression. Let us sketch this response. I mean the phase response. So, of course, at omega equal to 0, it is going to be pi by 2 and at omega equal to pi it is going to be 0. This is the situation. Now, once again we have linear phase. Well almost not quite linear. If it were strictly linear phase, this would have been a straight line indeed, but a straight line passing through the origin. So, it is not really linear phase. This is called pseudo linear phase, seemingly linear phase. In fact, for complete less, let us draw the magnitude and the phase response all the way from minus pi to pi for both of these filters now for the sake of completeness. So, the situation is for the filter half 1 plus z inverse the overall magnitude response should look like this between minus pi and pi I mean essentially mod cos omega by 2 and the phase. This starts at plus pi by 2 here and goes up to minus pi by 2 there passes through the origin of course, a straight line. For the second filter the overall magnitude response looks like this essentially a sin mod sin omega by 2 and the phase response looks like this starts at pi by 2 there and goes to 0. It is a straight line segment. Here it would start. Now, that is interesting. You know the phase on this side between minus pi and 0 needs to be the negative of the phase between 0 and pi. So, it will be a mirror image. So, these are lines straight lines. So, we call this pseudo linear phase. Now, one point needs to be understood. There is a peculiar situation at the point omega equal to 0 here. The phase is both pi by 2 and minus pi by 2. How can this be possible? Well, the answer comes from the magnitude response. The magnitude response at that point omega equal to 0 is 0. When the magnitude response is 0 at a point the phase response has no meaning. The phase response could be anything. You see it means that sine wave at omega equal to 0 is anyway being destroyed. So, what consequence is the phase response? That is why there is an ambiguity in phase or a discontinuity in phase at the point omega equal to 0 in this phase response. A small detail, but important when we try to understand this filter bank completely. Anyway, coming back to these magnitude responses now. If we consider the two magnitude responses together. In fact, let us first consider the frequency responses together, both magnitude and phase. A very interesting property emerges. You see, suppose we add them. So, suppose you take you know I do not even need to substitute z equal to e raise to the power j omega. We keep it as it is. So, half into 1 plus z inverse plus half into 1 minus z inverse. It is very easy to see. This is equal to 1. A very interesting consequence. What does it physically mean? It physically means that if I were to send a sample sine wave of frequency omega into one of these filters and then the other and if I took these two sample sine waves from the two filters and put them together by adding, you would get back the original sine wave. So, sine wave at frequency omega is split by these two filters in such a way that the parts can simply come together and reconstruct the sine wave as it is. Now let us look at something more interesting. What can we say about the power? So, recall that if you give a sample sine wave, a sample sinusoid to a discrete time filter, let us say the angular frequency is omega and the frequency response here is h of omega. Then the power that emerges from here is proportional to mod h omega squared. So, in other words whatever is the power of the sample sinusoid with angular frequency equal to omega here is multiplied by mod h omega squared when it emerges at the output. So, the squared magnitude of a frequency response is indicative of the change in power of the sine wave when it goes through that discrete time filter. What can we say about the power change of a sine wave when it goes through either of these two filters here, let us see. So, that means in other words we are asking the question, what is the magnitude squared response? In the first one it is mod cos omega by 2 the whole squared and in the second one it is mod sine omega by 2 the whole squared and lo and behold when you add these you get 1 as well. That is a very very interesting observation. Not only does it happen that when you add the two responses together you know you literally took 1 plus sin inverse by 2 and 1 minus sin inverse by 2 the actual frequency responses irrespective of z. In fact I said there you did not even need to substitute z equal to erase the power j omega. You just added the system functions together and you got 1. So, if you put a sine wave of frequency omega I mean a sample sinusoid of frequency angular frequency omega and looked at the corresponding emerging sine wave on the top branch and the lower branch and just added them together you would get back the original sine wave. Not only that what we have just shown is that when you multiply if you look at the power emerging from the upper branch and the power emerging from the lower branch the powers also add. So, in fact you have two kinds of complementarity in the filters this is something very very interesting here. So, if you call the frequency responses of the upper branch and lower branch respectively as h upper omega. And h lower omega upper branch lower branch then two properties are immediately satisfied h upper omega plus h lower omega is equal to 1. This is called magnitude complementary property and in addition mod h upper omega squared plus mod h lower omega square plus h lower omega squared is identically equal to 1. This is called the power complementary property a very very interesting result. The hard analysis filter bank is both magnitude complementary and power complementary. In fact I leave it to you to study the synthesis filter bank and come to a similar conclusion. The filters are magnitude complementary and power complementary whatever it be this is something striking. Now you see what I mean when I said the filters have individual properties and collective properties magnitude complementarity and power complementarity are collective properties. The low pass and the high pass nature if you recall the second filter that we had was high pass because it emphasized higher frequencies and deemphasized lower frequencies. So, the low pass and high pass properties are individual properties. The magnitude and the power complementary properties are collective properties. So, we have filters with individual and collective properties forming two filter banks the analysis filter bank and the synthesis filter bank. Now you know the idea of a filter bank is very deeply entrenched in multi resolution analysis. In subsequent lecture lectures we shall study this connection even further. So, for today we shall conclude the lecture here by noting that we have already established even more deeply the frequency domain behavior of the filter bank that we brought out in the previous lecture. Thank you.