 A warm welcome to the 32nd lecture on the subject of wavelets and multirate digital signal processing. In the previous lecture, we had introduced the idea of the wave packet transform. Essentially, we had hinted at the aim behind the wave packet transform, namely to be able to decompose the high pass branch or the detail as well into subspaces. We had made one important observation. The decomposition of the high pass branch proceeds somewhat differently from the decomposition of the low pass branch. There are some counterintuitive observations in the frequency domain. However, we had considered only the ideal two band filter bank when we talked about the wave packet transform the last time. If we wish that we should be able to realize a wave packet transform to implement it, then we must look at how the wave packet transform would operate given realistic filter banks and that is what we aim to do in the lecture today. Therefore, in the lecture today, we intend to talk about some identities pertaining to multirate digital signal processing which are often called the noble identities. Noble in the sense that they are extremely important in building multirate structures and in analyzing multirate structures. Further, we will use the noble identities to build a wave packet transform in the context of the hard multiresolution analysis. So, with that little introduction to what we intend to do in the lecture today, let us proceed to talk about the so called noble identities that govern the behavior of multirate systems. You know the essential idea in the noble identities is actually to look at what happens when we cascade. So, here in you know these are called noble identities because they occur very frequently whenever you wish to iterate multirate systems. So, for example, in the discrete wave layer transform, we have iterations of down samplers and filters one after the other and in fact, on the synthesis side you have up samplers and filters. So, the noble identities try and deal with this situation. How can you aggregate together different down samplers and filters or different up samplers and filters? Let us first look at the down sampling context, the analysis context. So, let us put down the basic theme. The theme in the noble identities is to deal with cascades, cascades of sampling rate changes and filters. Let us take an example. An example could be something like this, what happens in the discrete wave layer transformation. So, you have the filter followed by a down sampler, again followed by a filter and down sampler and so on. So, it is this kind of cascade that we wish to be able to address and therefore, let us consider the noble identity for down sampling first. Now, the noble identity for down sampling emerges from the following consideration. How can we represent a down sampler followed by a filter? Let us call that filter H of z. So, we describe the system function of the filter as H of z. The noble identity essentially answers the question how to interchange. So, can I replace this by some other filter followed by a down sample? That is the question that this so called noble identity tries to address. Now, you know we could answer or we could address this problem by assuming that a sequence x of n is fed to a down sampler followed by this filter H z. So, let us assume that this filter has a system function H z and the corresponding impulse response is small h of n. Let us call the output y here, y of n I mean. Now, it is very obvious what y of n is. Essentially, y of n is the sequence here convolved with the impulse response h n. The sequence here has at every sample location, the value of this sequence at twice that sample location. So, for example, the sequence at this point would have at the point n equal to 3, the value x 6 or for that matter at 0 it would of course, take the same x of 0 at minus 1 here it would take the value x of minus 2. So, if we call this x 1 here, then what we are saying in effect is x 1 of n is x of 2 n for all n belonging to the set of integers and therefore, y of n which is essentially x 1 convolved with h evaluated at n can be described as follows. Y of n is obviously summation k running from minus to plus infinity either x 1 k times h of n minus n minus k or you could do it the other way. We will write both the expressions and see which is convenient. Now, x 1 k is easy to write. So, x 1 k let us take the first of these two. So, it is very clear that y n becomes summation k going from minus to plus infinity of 2 k times h of n minus k. Now, you know what we are saying here is in effect that we are doing almost what a convolution does you know remember our objective is to interchange the position of the down sampler and the filter. So, from this expression here we are trying to get an equivalent expression for a system comprised of a filter followed by a down sampler in which the operation should give the same expression as here. So, the question that we are asking is how does this fit into filter followed by down sampler. Let us see whether it is easier to answer this question by using this expression or by using the expression here. Please remember that when we have a down sampling operation, we multiply the index by 2. So, suppose we were to use this expression here and note that x 1 of n minus l is actually x of 2 n minus 2 l then in some sense we have already taken care of the requirement of 2 n right there. So, instead of using this expression let us use this expression to arrive at an answer to this question. Let us use y of n is summation l going from minus to plus infinity h of l x 1 of n minus l and note that x 1 of n minus l is in fact x of 2 n minus 2 l where upon we have y of n is summation l going from minus to plus infinity h of l x of 2 n minus 2 l and now we are almost there in answering the quest. You know if you look back at the drawing that we had here this down sample by 2 has been taken care of in this we already have a 2 n here. So, what we could do is to regard this 2 n index as being before the filter you know. So, the 2 n has been created by the down sampling operation. So, if I replace the 2 n here by n I would get what appears just after the filter. In other words just before the equivalent down sample in filter and down sample. So, at this point we have summation l going from minus to plus infinity h l x n minus l n minus 2 l. Now it is very clear that what is the filter should be you know this is almost like a convolution except that it appears as if you are taking only the even points in the impulse response. So, you are allowing only the even points in the impulse response to operate and the odd points are being annulled. This looks like a convolution the h l seems to be located at the 2 l th point all other places. So, 2 l plus 1 th places are 0 that is the effective h that we have here it is as if this came from the 2 l th place and at the 2 l plus 1 th place we have all 0 0. So, in other words if we define h 1 h 1 n in such a way. So, h 1 n is equal to 0 when n is odd or in other words n not a multiple of 2 and equal to h n by 2 when n is a multiple of 2. In that case what we have here then is that y n is simply summation l going from minus to plus infinity h 1 l x 2 n minus l and once again before the down sampler this is n. So, it is very clear what is happening here the filter that operates before the down sampler is essentially one with impulse response h 1. So, essentially the impulse response of the equivalent filter preceding the down sampler is the original impulse response up sampled by a factor of 2 it is as simple as that. So, conclusion is that a down sampler followed by a filter with system function h z or impulse response h n is equivalent to h of z squared down sample by 2 filtering by h of z squared and then down sampling by 2. Now, in other words the impulse response of the filter of the filter preceding the down sampler is the original impulse response sampled by 2. So, h n now this is what is called the noble identity for down sampling. In other words whenever you have a filter transfer function where the odd samples of the impulse response are 0 and the even samples are could be non 0. We could replace that filter followed by a down sampler by an equivalent operation where there is a down sampling first and then a filtering operation. Now comparatively what is the advantage or disadvantage of each of these structures we must ask that question why should we want perhaps to look at this structure as opposed to this. There are two reasons one is of course purely analytical. So, you may want to get equivalent structures and today we would like to do that when we discuss the wave packet transfer, but the other reason is also computational. For example, if you look at this structure carefully what you are doing is first killing or removing the samples which are not going to be involved in the computation in down sampling and then you are doing a filtering operation. On the other hand what you are doing in this structure here is first filtering that means for example before down sampling you are going to generate many samples here which are ultimately just going to be thrown away. To generate samples at this point you are going to do a lot of work because you are going to do a convolution operation here. So, this is an inefficient structure whereas this structure is the efficient version of this inefficient structure. We should make a note of that that is a very important conclusion we have drawn. This structure is efficient computationally that is because we are doing away with samples that do not figure in the computation and this structure is inefficient computationally. However, when it comes to analysis which is what we need to do today to build the notion of the wave packet transform is going to be the other way around. We are going to go from what seems to be an inefficient structure to an equivalent. So, for example what we are going to do today is something like this. We are going to go from a cascade like this. So, you would have say an H z followed by a down sampler and another H z followed by another down sampler and so on and we are going to try and interchange. So, we are going to bring this filter here. So, we are going to put the down sampler past and that means we are going to move towards bringing the down samplers together and this is going to help us build an equivalent structure when there is a cascade. So, you know either motion the motion from efficient so-called efficient to so-called inefficient or the motion from so-called inefficient to the so-called efficient structures are important in the context of multirate digital signal processing. Now, before I proceed to go to the precise application of this replacement let me first address the next noble identity and that is the noble identity of upsample. Now, you know one principle that is very useful in many multirate systems and in fact whenever we wish to build examples or structures in multirate systems is what is called the principle of transposition. The word transpose has been used frequently in discrete time signal processing. You will recall that when we did not have upsamplers and down samplers. So, when we had essentially all linear shift invariant operations if you had a signal flow graph in which there were only the traditional linear shift invariant signal flow operations namely multiplication by a constant summing up of several branches delay by multiples of the sampling time and so on. Then we could define what is called the transpose of a signal flow graph. The transpose of a signal flow graph is obtained by reversing all arrows in the signal flow graph, but keeping the multipliers as they are and when you reverse all arrows what is the branching point could become a summation, what is the summation point can become a branching point. So, the idea of transposition can also be extended to when there are sampling rate operations sampling rate change operations like down samplers and upsamplers. The only difference is that when we reverse the arrows we must also reverse the nature of the sampling rate change operation. So, if you reverse the down sampler you must get an upsampler of the same factor. With this change we define the notion of transposition to get us one more identity when there is one in multirate systems. So, transposition in multirate systems means the following operations. It means first reverse the direction of signal flow, second when reversing respectively down sampler a little bit of caution has to be used, put there a corresponding down or respectively up sampler of the same factor that is important same factor. Now, what we are going to do is to use this principle of transposition to arrive at the noble identity for upsampling and of course then we can prove it. So, let us put down the noble identity for down sampling once again. Essentially it says down sample followed by h z is equivalent to h z squared followed by down sample. Now, let us use the principle of transposition. Transposition would mean that you reverse all the arrows here. Now, when you reverse the arrow here you are going to start by reversing this you know. So, when you reverse and go across the down sampler in the other direction you must put there an upsampler of the same factor according to the rule of transposition. This is as it is this is discontinues to be h z squared. Again when you go past and reverse here you have h z as it is, but then here you must put an upsampler in place of the down sampler of factor 2. So, let us draw the corresponding transpose. The transpose would therefore be this look at this is the transpose of this h z first and then upsample by 2 is equivalent to upsample by 2 and then an h z squared. This is the transpose of this upsample by 2 the other way and then h z squared and this is essentially the noble identity for upsample. What is the noble identity for upsampling say? It says that if you filter followed by an upsampling by 2 it is the same thing as upsampling by 2 and then filtering by a filter whose impulse response is the upsampled version of the impulse response of this filter. Of course, in both cases there is an upsampling involved. The other way of looking at it is going from here to here there is a down sampling and going from here to here there is an upsampling. So, you know there are different ways of remembering this. Now, of course, what I have proved here or rather what I have written down here is essentially a noble identity for down sampling and upsampling by sample factor sampling rate changes of factor 2. We can generalize in a very straight forward way to any other sampling rate change factor. In fact, let me write down in general the noble identities for any upsampling or down sampling factor of course, integer factor and in fact I leave as an exercise for you to do three things. One is just as we proved the noble identity for down sampling by a factor of 2 I ask you as the first exercise to prove the noble identity for upsampling by a factor of 2. Subsequently, in the next exercises I leave it to you to prove the more general versions of the noble identities for down sampling and upsampling. So, let me write down these exercises and leave you to do them and attempt them later. So, exercise number 1 I explain the exercises and I leave it to you to do them. Prove of course, here I am talking about upsampling by a factor of 2. Please remember when I talked about transposition that is not a proof that is more like a mnemonic and a to the memory that was not a proof. If we wish to treat that as a proof then we must prove in general that transposition leads to a valid structure. In fact, let me put that before you as a challenge. So, challenge that transposition leads to a valid alternate structure, a slightly abstract thing to prove. For example, what we are saying is transposition gave us the ability to prove a new theorem from a previous one here. Now, to show that this is in general true, to show that when we transpose a certain structure or when we transpose two equivalent structures we get another pair of equivalent structures. This is the challenge slightly difficult job. Anyway, that is a challenge that is not what I would call a traditional tutorial exercise, but I do like to put some challenges before the class. Anyway, let me put the second exercise before you. The second exercise is prove the more general noble identities down sample by m followed by h z is equivalent to filter by h z raise to the m followed by down sample by m for any positive integer n and this is for down sampling and correspondingly for up sampling we have m any positive integer and this is for up sample. Anyway, with that little background on the noble identities let us apply the noble identities now to the context of the Haar multi-resolution analysis. Our objective was to go towards the wave packet transform in the context of the Haar MRA. So, let us take two iterations. You know actually the wave packet transform makes sense when you go down at least by two steps in the ladder of the multi-resolution analysis. So, let us go down two steps. So, let us take the Haar MRA. We have 1 plus serinverse. Forget about the factors of half and so on. Let us just focus on the basic filter system function 1 plus serinverse and 1 minus serinverse. This is the analysis filter bank for the Haar. Now, call this whole thing the bank B and what we do in the wave packet transform in Haar would be to repeat B here and repeat B there. In other words, we would have four structures of cascade. You know visualize it. You have this and then you have this whole structure being repeated here. So, you have two structures of cascade emerging from this, this cascade with this and this cascade with this. When you repeat this whole structure here, you have this cascaded with this and this cascaded with this. So, there are four structures and let us draw those four structures of cascade. Let us take just one of them first to make matters simple. So, consider there are. So, let us make a remark. There are four cascade structures distinctly. In fact, we can capture them in one expression. We could write 1 plus or minus serinverse followed by down sample by 2 followed by 1 plus or minus serinverse followed by down sample by 2 again. Now, it is very easy to use the noble identities to replace this. So, as I said here is one example where I would like to interchange this pair. So, I would like to bring this filter in place of the down sampler and the down sampler in place of the filter. So, I can put the down samplers together. Using the noble identity for down sampling, I would have down sample followed by 1 plus or minus serinverse is equivalent to 1 plus or minus serinverse followed by down sampling by 2 z to the power minus 2 remember here instead of z to the power minus 1 there. Therefore, the four distinct cascade structures become as follows. Essentially, 1 plus or minus z inverse times 1 plus or minus z to the power minus 2 followed by down sample by 2 and down sample by 2 again. And if you down sample by 2 twice it is equivalent to down sampling by 4 that is easy to see. When you down sample by 2 the first time you are taking every second sample and putting it in half the location. When you down sample by 2 the second time you are again taking every second sample and putting it back in half the location place. So, in effect you are taking every fourth sample and putting it in one fourth the place of location. So, what we are saying in effect is down sample by 2 followed by down sample by 2 is equivalent to down sample by 4. And therefore, we have four filters that come out of this process. In other words what we are saying is when we consider this movement from v 2 you know what we are doing if you remember in the analysis process is to decompose v 2 into v 1 and w 1 and then followed by a decomposition of v 1 into v 0 and w 0. Now, w 1 is also being decomposed here into what we might call w 1 0 and w 1 1 and that is the beauty of the wave packet transform this is special to the wave packet transform. So, in fact you know notionally we can show the filters also here. So, when we operate the filter 1 plus z inverse we are going from v 2 to v 1 and when we operate 1 minus z inverse we go from v 2 to w 1. Here again it is 1 plus z inverse which takes us to v 0 and 1 minus z inverse that takes us to w 0. Here again it is 1 plus z inverse that takes us to w 1 0 and 1 minus z inverse which takes us to w 1 1. And therefore, let us take these four filters let us write down these four filters systematically. So, 1 plus z inverse so v 2 to v 1 to v 0 will come from 1 plus z inverse there and 1 plus z to the power minus 2 there. So, we have 1 plus z inverse into 1 plus z to the power minus 2 which is 1 plus z inverse plus z to the power minus 2 plus z to the power minus 3. So, in fact this sequence tells us how to expand at an element of the basis here in terms of the basis here. And as you can see this corresponds to the sequence 1 1 1 1 at the points 0 1 2 and 3. So, we are saying the sequence 1 1 1 1 expands phi t at an element of the basis here. Typical basis element in v 0 in terms of phi 4 t minus k phi 4 t in its translates you know when you go from v 0 to v 2 you are going to dilate by a factor of 4 that is what is meant by down sampling by 4. And indeed it is quite clear indeed phi t is phi 4 t plus phi 4 t minus 1 plus phi 4 t minus 2 plus phi 4 t minus 3 and that is very easy to see that is essentially saying that this is this plus this plus this plus this here you have 0 to 1 1 there 1 4th half 3 4th there and 1. Now, once we have done this for one of these filters in cast k it is very easy to do them for the others. And that in fact gives us the basis for each of these spaces w 1, w 1 0, w 1 1, w 1 0, w 1 0, w 1 0, w 1 0, w 1 0, w 1 0, w 1 0, and w 0. So, correspondingly if you if you go back to the structure here you will realize that to come from v 2 to w 0 you should do a 1 plus syrinverse and then followed by a 1 minus z to the power minus 2. And to go from v 2 to w 1 0 you should do a 1 minus syrinverse followed by a 1 plus z raise to the power minus 2. And to go from v 2 to w 1 1 you must do a 1 minus syrinverse followed by a 1 minus z raise to the power minus 2. So, let us do this and complete the job. So, what we are saying is v 2 to v 1 to w 0 should be obtained by 1 plus syrinverse followed by a 1 minus z raise to the power minus 2. Essentially a 1 plus syrinverse minus z raise to the power minus 2 minus z raise to the power minus 3 and that gives rise to the sequence 1, 1, minus 1, minus 1 and the function 1, 1, w 1, w 1, w 1, w 1, w 1, minus 1, minus 1, 0, half, 1, the har wavelet as expected. Similarly, we could complete the others. So, v 2 to w 1 to w 1 0 is essentially 1 minus syrinverse and 1 plus z raise to the power minus 2. So, essentially the sequence 1 minus 1 again 1 minus 1 and correspondingly the function 1, minus 1, 1, minus 1. So, this function and its integer 2 is essentially translates span w 1, 0. And finally, I leave it to you to show for w 1, 1 we would have 1, minus 1, minus 1 and 1. So, I do not really need this, this is just to exemplify 0, half, 0, 1, 4th here, half there, 3, 4th here and 1. This function and its translates. So, therefore, we have now built the wave packet transform in the context of the har multi resolution analysis. What we have done is to establish the one function whose integer translates span each of these subspaces v 0, w 0, w 1 0 and w 1 1. So, in fact, the property of a single function and its integer translates spanning spaces holds also for the spaces emerging from the wave packet analysis. With this then we have illustrated one instance of the wave packet analysis and we shall go on to other uses of the noble identities in the next lecture. Thank you.