 A warm welcome to the 26th lecture on the subject of wavelets and multirate digital signal processing. This lecture is intended for a proof of the theorem of multiresolution analysis, dyadic multiresolution analysis. And therefore, we have essentially focused today on continuing from where we were the last time where we had informally introduced the meaning quote unquote meaning of the theorem of multiresolution analysis and today we set out to prove it. Let me once again put before you the thoughts with which we concluded the previous lecture. We said that one way to interpret this whole question of discretization of the translation parameter is to raise the issue of sampling, but generalized sampling noting that we were dealing with band pass functions instead of band limited functions. We noted in brief that if you look at the discretization of the translation parameter in the spaces v, v 0, v 1, v 2 in the ladder it amounted to a version of the band limited sampling theorem, the conventional sampling theorem. Because if you looked at the spaces v 0 contained in v 1 contained in v 2 and then v minus 1 contained in v 0 and so on in the ladder, we were talking of spaces of band limited functions with the band doubling each time, but of course the bands are all around the frequency 0. So, it is the all inclusive band up to a certain frequency and then double and then 4 times and 8 times as you go up the ladder. So, naturally the sampling frequency needs to double each time as well and that is what we observed. On the other hand, we looked at the ideal case of a band pass function and we said that if the band was strategically placed, for example, if we looked at the band between pi and 2 pi and sampled such a band pass signal, a signal contained in this band only in this band restricted to this band at a sampling rate of 2 times pi instead of 2 times 2 pi. You know, we got a band pass version of the sampling theorem. We saw that translates of the spectrum created by sampling this band pass function at a sampling an angular sampling frequency of 2 times the band 2 times pi instead of 2 times 2 pi which is the highest frequency. We still got the translates of the original spectrum being non overlapping with the original spectrum and therefore, we could put a band pass filter and reconstruct the signal even after sampling. Now, sampling functions in the ideal sense, I mean the ideal band pass reconstruction filter is going to have an impulse response which is unrealizable and therefore, wavelets is a way of band pass sampling and reconstruction practically that is the interpretation we had given it. And therefore, we were asking the question, can we prove that under the axioms of an MRA, we can extract this wavelet function psi t which will allow band pass sampling. This was the interpretation that we had given it and we set forth to prove it today. Well, the strategy that we would employ in the proof is to look at a typical function in the incremental subspace, let us call that function f t. Now, what is it that characterizes this function f t? There are two things that characterize it, one f t is orthogonal to every translate of phi t M integer that is because phi of t minus M for all integer M forms an orthogonal basis for the space v 0 and since f t is orthogonal to every function in v 0, remember that is how we describe this incremental subspace as being the novelty in v 1 over v 0, novelty in the sense of dot products. So, f t is orthogonal to every phi t minus M, M over the set of integers which of course, gives the complementary property, but the inclusion property namely that f t belongs to v 1 is captured by the fact that f t can be expressed, it can be expanded in terms of phi 2 t minus N for integer N, essentially the inclusion property the fact that f t is a part of v 1. Now, let us use these two properties together and come up with some interesting observations. So, let f t in particular be the sum for N going from minus to plus infinity f square bracket N. Now, this is the set of coefficients in the expansion of f t with respect to phi 2 t minus N coefficients of expansion and of course, we must not forget that phi t itself belongs to v 0 and v 0 is contained in v 1. Therefore, phi t could be expanded in terms of phi 2 t minus N, M integer we must not forget this and we also know what helps us expand phi t in terms of phi 2 t minus N. Remember that when we have a discrete time filter bank which on iteration leads to the wavelet, it is the coefficients of the low pass orthogonal filter you know the orthogonal filter bank the discrete filter bank which leads on iteration to a wavelet. For example, in the case of the Haar wavelet or the Dobash wavelet the low pass filter coefficients on iteration give you function phi t and in fact, there is a recursive dilation equation that relates phi t to its own dyadic dilates and translates and in that recursive dilation equation it is the impulse response or the coefficients of the low pass filter which come into picture. Therefore, we can write down phi of t is also summation on N from minus to plus infinity in principle in general H of N phi 2 t minus N. So, these are the low pass impulse response coefficients. Now, we shall use the orthogonality of f t to phi t minus N to establish something interesting. So, we note that if phi t is expandable in this way, then phi t minus M shall be expandable in the following way and now using the orthogonality of f t with phi t minus M we get summation on N f N phi 2 t minus N in a product with summation on N. Now, here the summation is over all the integers in both cases, but we will use a different variable here to distinguish it from N. So, we could use summation on L here H L phi 2 t minus 2 M phi 2 t minus 2 M phi 2 t minus L is 0 for all M. Now, we again invoke the orthogonality of phi with respect to its own translates. So, we note that the dot product of phi 2 t minus say k 1 and phi 2 t minus k 2 is essentially phi 2 t minus k 1 phi 2 t minus k 2 bar d t integrated and if we only care to put 2 t is equal to lambda, we would get this integral to be phi lambda minus k 1 phi lambda minus k 2 bar d lambda a factor of half outside no other change and this is of course, half delta k 1 minus k 2 from the orthogonality of phi. So, phi is orthogonal to its own translates in fact, it forms an orthogonal basis of v 0. The orthogonality of phi to its own integer translates of course, guarantees the orthogonality of contracted versions of the same phi t. The contraction must be the same that is in this case 2. When using this we have the dot product of f t with phi t minus M is therefore, you know if I go back let me put back that summation for you. In this summation we would expand. So, summation on n f n times this term and summation on l h l times this term. So, I can bring the summations outside the dot product expression. It is summation on n summation on l inner product f n well in fact, we could even take the coefficients outside. So, this inner product could be operated last f n h l bar and then an inner product phi 2 t minus n phi 2 t minus 2 n minus l. So, this is the summation. And using the orthogonality of these phi's we notice that only when n is equal to 2 m plus l does this dot product survive. So, we can eliminate the summation on n and leave only a summation on l and that leaves us with summation on l. For that particular n equal to 2 m plus l we have f 2 m plus l h l bar and then the dot product reduces to half and this quantity is familiar to us. This is the familiar quantity with which we deal. Essentially it is the cross correlation of the sequences f and h. So, if we come back here this summation summation on l f 2 m plus l h l bar forget for the matter for the time being about the factor half. It is not that terribly important because ultimately we are going to equate this to 0. What is important here is that there is a dot product between the sequences f and h with an appropriate shift mutual shift and that is essentially a cross correlation. So, we are essentially looking at the cross correlation of sequences. Maybe we should use square brackets and h and how is this cross correlation defined? Well, this cross correlation is often denoted by r f h. This denotes the secondary arguments evaluated at a shift let us say of p. Summation of l f p plus l h l bar the complex conjugate is important only if we are dealing with complex responses otherwise it is not so terribly important. I leave it as an exercise to find the z transform of this. I provide a few hints. We notice that this cross correlation as a function of p is by itself a sequence. In a way this is a sequence almost obtained by convolution of f and h, but with a difference. You know in fact had this been minus l here it would precisely have been a convolution of f and h complex conjugate, but the presence of plus instead of minus makes it slightly different from a convolution and as we expect the z transform of this sequence is going to be related to a product of the z transforms of f and h. Now, I leave it as an exercise to prove the follow. Prove that the z transform of the sequence the cross correlation sequence is f of z times h of z inverse where then I continue f of z is the z transform of f n and h of z is the z transform of h n also happens to be the system function of the low pass filter. In fact, if you recall this low pass system function characterizes the whole filter bank. It is very central to the filter to the orthogonal filter bank. Once you know h z the system function of the low pass filter you know the high pass filter on the analysis side and you know the low pass and high pass filters of the synthesis. So, with that observation we notice that the cross correlation are f h p evaluated at 2 m for all integer m. In other words that all even locations is 0 that is what we are stating in the result that we derived a few minutes ago. Recall when we put this here and when we said that this must be equal to 0 for all m what we are saying in effect is that the cross correlation of f and h when evaluated at all even locations must be 0. Let us in fact make a note of it. It is a very important conclusion we have drawn. This is the orthogonality requirement. Now, we know how to deal with the situation when we wish to look only at even locations. In fact, we know the operation that does that down sampling by 2. So, suppose you were to notionally take this cross correlation sequence and down sample it by 2 you would get an all 0 sequence that is what we are essentially saying. Let us put that down graphically. What we are saying is if we take r f h p and subject it to down sampling by 2 we get a 0 sequence and therefore, if we look at this situation in the z domain we will of course have a 0 z transform here also. So, let us interpret in the z domain. Now you know it is easier for us to do the following. We could say take r f h p down sample by 2 and up sample by 2 and that would also be an all 0 sequence and of course, the z transform is 0 here. How do we deal in the z domain with this pair of operations? You recall that if r f h p has the z transform capital R f h z what I am saying is if r f h p on z transformation results in capital R f h of z then r f h p subjected to down sampling by 2 and then up sampling by 2 results on z transformation into half r f h z plus r f h minus z and therefore, what we have then is r f h z plus r f h minus z plus r f h minus z. It must be identically 0 or in other words f z times h z inverse plus f minus z times h of minus z inverse is identically 0 and in fact, we can now rearrange this. We can rearrange this to get some very interesting insights into f z. So, that would give us the condition f z by f of minus z is essentially minus h of minus z inverse divided by h of z inverse. Now, this is where we have an interesting insight into f this typical function f which belongs to the incremental subspace. You know in this ratio f of z by f of minus z we have managed to eliminate what is specific to f. What I mean by that is if you think of w 0 as a country with citizens comprised of functions in w 0 and each citizen having a passport then you could think of first a blank passport and then a passport with entries made for that person. What we want to identify is the nature of the blank passport in that country because that characterizes the citizen of that country in general. The specific entries corresponding to that person are not of immediate importance to us at the moment. We wish to characterize the space w 0 in general. So, in this ratio f of z by f of minus z we have got on the right hand side something that has no specificity at all. It depends only on the low pass filter essentially of the filter bank. Therefore, we can note that when we take a ratio the difference between the numerator on the left hand side and the right hand side must in general be a function of z. So, we could in general say from this that f of z must then be of form some capital lambda z a function of z times minus h of minus z inverse. So, this is the factor which got cancelled cancelled in that ratio. What I mean is when we took this ratio f z by f of minus z some factor must have got cancelled to result in something independent of that particular function and that cancelled factor is lambda z. Therefore, lambda z capital lambda z contains information specific to that particular citizen f t in w 0. So, the citizen specific information as you might call it for f t is contained in lambda z. I am not saying that all of lambda z must be citizen specific, but whatever citizen specific information there is is all captured in that lambda z. Now, let us also make an observation by equating the denominators. We also have f of z is lambda z times h of z inverse rather f of minus z I am sorry and I put back the ratio before you to explain. What I am saying is if lambda z is the factor that has got cancelled then f of minus z must also be lambda z times h of z inverse and therefore, we have two equations now. We have f of z is minus lambda z times h of z inverse minus z inverse and f of minus z is this, but then here if we substitute z by minus z we get f of minus z from here would be minus lambda of minus z h of z inverse and now if we compare this and this we note that since h of z inverse is a common factor and is not identically 0 these must be identical. So, we must have lambda of z is equal to minus lambda of minus z we must have and this amounts to saying that lambda z plus lambda minus z must be 0. What does this mean in terms of sequences? You see it means that if lambda z is the z transform of a sequence then that sequence should be 0 at all the even locations which means that the sequence should have been obtained by you could think you see when a sequence is 0 at all the even locations we are effectively saying that the sequence could have been obtained by upsampling another sequence and then shifting by one place. You see when you upsample a sequence you introduce 0's at all the odd locations upsample by a factor of 2. So, when you upsample a sequence by a factor of 2 you introduce 0's at all the odd locations. Now, if you wish all those 0's to shift to the even locations all that you need to do is to shift this upsample sequence by an odd number of samples. So, in other words it is a sequence let us call it lambda tilde lambda tilde n which on upsampling by a factor of 2 and then shifting by an odd number of samples gives us the z transform lambda z. This is of course the z domain representation and what we are saying in addition is that to get f z we effectively have to cascade this. So, you have lambda tilde n and then you have an upsampler by a factor of 2 you could shift by any odd number of samples and then subject this to the action of h of minus z inverse and what we get here is essentially the sequence f this is what we have essentially said. Now, we could choose that odd number of samples strategically. You know h of minus z inverse in this discussion if you look at this sequence of steps here this h of minus z inverse looks very much like the analysis high pass filter. The only difference between this and the analysis high pass filter is that you need to put an odd delay here. So, we could in particular choose the odd number of samples to be equal to l minus 1 where l is essentially the low pass filter lengths. So, for example for adobash 4 or in other words adobash filter bank where the length of the filters is 4 l is equal to 4 and if you recall l minus 1 is the delay that we need to introduce once we have replaced h z by h of minus z inverse to get the analysis high pass filter. What I am saying in effect is recall that z raise the power minus l minus 1 times h of minus z inverse is essentially the analysis high pass filter and therefore, what we are saying in effect is that we could obtain f n as follows. We could take this hypothetical lambda tilde which has been up sample by 2. We could subject it to the action of the analysis high pass filter z raise the power minus l minus 1 h minus z inverse and this would produce f for us and once we have f n f of n the sequence f we can reconstruct the function f of t and lo and behold f of t is therefore, summation of n f n of course, when not stated we mean the summation runs over all the integers. So, summation f n phi 2 t minus n now let us denote the impulse response of the analysis high pass filter by g n. So, let g n be the inverse z transform of z raise the power minus l minus 1 h of minus z inverse. Essentially g n is the impulse response of the analysis high pass filter where upon what we are saying is lambda tilde up sample by 2 subjected to the action of an LSI system linear shift invariant system with impulse response g n results in f f of n if you wish to be very precise and f of n must therefore, be and for that we shall first introduce an intermediate sequence here. Let us call it lambda intermediate lambda intermediate of n this lambda intermediate is of course, convolved with g n to get f n f n is lambda intermediate convolved with g n and that can be written as summation k running from minus to plus infinity minus lambda intermediate k g n minus k. But then lambda intermediate k is non zero only at even k and therefore, we need to replace this summation only with summation on k and k replaced by 2 k here only even locations. So, this summation can be rewritten as summation k again going from minus to plus infinity lambda intermediate at 2 k all of the points are 0 remember because it is up sample g n minus 2 k and lambda intermediate at 2 k is simply lambda tilde at k an interesting conclusion and this is f n for you. So, we will just get our bearings once again what we have concluded is that f n has this form f n is summation over all integer k lambda tilde k g n minus 2 k and therefore, we shall now write down f t and f t the typical prototypical function in w 0 has the following form f t is summation on all k lambda tilde k g n minus 2 k times. Now, this is you see there is a summation this is f of n the whole thing is f of n and this needs to be multiplied by phi 2 t minus n and sum over all n. So, sum over all n times this now here we have an external summation on n and an internal summation on k let us interchange the order. So, let us do the summation on k outside and the summation on n inside. So, interchanging the summation we would get summation on k outside lambda tilde k and then I will write the summation on n inside as follows summation on n over all n g n minus 2 k phi 2 t minus n and now we play the standard trick k is fixed here. So, let us put n minus 2 k equal to another variable let us call it 2 k. Since, k is fixed when n runs over all the integers so does q and therefore, we can rewrite this as summation q running over all the integers g of q phi of now well n is clearly q plus 2 k and therefore, we have 2 t minus q minus 2 k here and now we can take 2 common from this and write summation on q again interpreting it over all the integers g of q times phi 2 t minus k minus q very interesting now here we are almost at where we wish to be you know if you look back at the expression that you have got here this t minus k is a shift on the continuous variable. So, suppose for the moment you forget t minus k and replace it by t and you notice that you are essentially making a linear combination of phi 2 t minus q with the coefficients g q you realize that you are trying to create a function in v 1 here define therefore, the function psi t to be summation on all integer q g q phi p 2 t minus q essentially psi belongs to v 1 and then what we have is effectively this prototypical function f t is summation on all k lambda tilde k psi t minus k that is what we have so effectively what we have proved is that this prototypical function f t in the orthogonal complement of v 0 in v 1 that is w 0 is expandable in terms of integer translates of psi t and that is exactly where we wanted to go if you could capture the single function psi t and all its integer translates could form a basis essentially could span the space w 0 our job is more or less done and therefore, we more or less prove what we wanted to this was the psi t for which we were looking we have shown that psi t minus k k over all the integers spans w 0 because it can represent any function in this orthogonal complement by linear combinations. Now, some details need to be completed we need to show that psi t is orthogonal to its integer translates. So, we of course, have shown that psi t minus k spans w 0 what we have not shown is that the psi t minus k form an orthogonal basis we also technically need to show that these psi t minus k are orthogonal to all integer translates of phi phi t minus m if you please for integer m and these are the details that we need now to take up and complete. Let us put down what we need to do the proof is almost complete except to demonstrate the following things psi t minus k over all integer k form an orthogonal set further the inner product of psi t minus k and phi t minus m is 0 for all k and m belonging to the set of integer. So, essentially that the integer translates of psi and the integer translates of phi are orthogonal. Now, how would we go about proving this we again use the same strategy of cross correlation we would expand psi t in terms of its basis phi 2 t minus n and phi t also in terms of the same basis and we shall use the z domain properties of the low pass filter that we have to prove these two results. We shall do this in the next lecture before we proceed to discuss variations of dietic multi resolution analysis that emerge from here. Thank you.