 A warm welcome to the 29th lecture on the subject of wavelets and multirate digital signal processing. In this lecture, we shall continue to discuss one more variant of the idea of multiresolution analysis. In the previous lecture, we had built on orthogonal or perfect reconstruction with one filter by extending it to bi-orthogonal or perfect reconstruction starting the design from two filters. And we had taken the specific example of the 5-3 filter bank in JPEG 2000. We saw that when we extended the multiresolution analysis to two filters instead of one filter, in other words, where we built the idea of of course, what we did was to build a filter bank, not really the multiresolution analysis underlying it. So, we have essentially built a perfect reconstruction filter bank, where the filters were of unequal lengths. But we saw that we could get the advantage of linear phase, we could get the advantage of symmetry in the impulse response. And further, we could extend what we did in the hard case to a piecewise linear function. So, if we take the 5-3 filter bank and if we look at the three length low pass filter, the filter with impulse response, the square of 1 plus 0 inverse essentially. It would essentially give us the triangular function as the scaling function. And the disadvantage with the triangular function was that it was not orthogonal to all its translates. It was orthogonal once you translated by two units or more, but it was not orthogonal to the one translate, 1 and minus 1 translate. And this was the reason why we needed to venture to other shores or venture to other lands as they said by looking at variants of the filter banks that we had discussed up to that point and bringing in the idea of a biorthogonal filter bank. Now, today we will take again the same 1 plus 0 inverse the whole square, the length 3 low pass filter that we talked about in the 5-3 filter bank, but we will deal with it in a slightly different way. And that would bring us to the idea of orthogonal multi resolution analysis with splines where we need to make a compromise in the nature of the scaling and wavelet functions that we construct and also in the nature of the filter banks that we would build that underlie this multi resolution analysis. In fact, what is going to happen as a consequence of our demanding and orthogonal multi resolution analysis today is that we shall have to go from finite length to infinite length filters and with some more difficult things as we shall see. Anyway, with that little discussion to put things in perspective, let us look again at the 3 length low pass filter in the 5-3 filter bank. So, if you look at the length 3 filter in the 5-3 filter bank, it essentially has 1 plus z inverse the whole square as the system function which is 1 plus 2 z inverse plus z raise the power minus 2 or be it with the factor of half if you like. So, the half is not terribly important, but let us keep it there for the moment. Now, we know what scaling function is generated out of this. So, in fact, let me not repeat all that discussion. We know that the corresponding scaling function is phi 1 t where of course, as you know phi 1 t obeys this dilation equation multiplied by half there and phi 1 t has an appearance like this. Now, our main problem and the reason why we needed to go to a bi-orthogonal filter bank as opposed to an orthogonal filter bank was that this is not orthogonal to its translate by 1 unit. So, if I translate this by 1 unit and I take the dot product essentially between phi 1 t and phi 1 t minus 1 or phi 1 t plus 1 that is if I consider these 2 dot products, these are non-zero and that was our main bone of contention because of which we could not be satisfied with this phi 1 t to construct an orthogonal multi resolution analysis out of it. Now, what we wish to do today is to explore that possibility. Even though phi 1 t is not orthogonal to its integer translates, can we build an orthogonal multi resolution analysis not out of phi 1 t, but perhaps out of a function that looks similar to phi 1 t. In other words, out of a function which is piecewise linear. So, what we are trying to do, let me put it down in clear terms, what we are trying to do today is to build an orthogonal multi resolution analysis with scaling functions which are piecewise linear even though they are not exactly phi 1 t. So, we are saying can we build a multi resolution analysis with piecewise linear functions for phi and psi that is the question which we are trying to answer today. And to answer that question, we must first relax the requirement of orthogonality as we have seen. So, you know if you look at the notion of orthogonality of phi t to its own integer translates, one can express this requirement in terms of the autocorrelation of phi. So, what we are saying is if we consider the autocorrelation function or I mean the autocorrelation of the continuous function phi t, here it is phi 1 t and if we sample this autocorrelation at the integers, all except the 0th sample must be 0 that is what orthogonality means. Let us put that down here. So, scaling function, let the scaling function be phi t. Scaling function phi t is orthogonal to its integer term translates, phi t minus m, m over the set of integers essentially means that the autocorrelation of phi, let us denote it by r phi phi evaluated at the shift tau, when sampled at tau equal to m, m integer that means, sampled with the rate of 1 gives an impulse sequence. What we are saying in effect, let me put it down mathematically is the sequence r phi phi tau, tau evaluated at m, overall integer m is an impulse sequence, discrete impulse of course. Now, we need to deal with this in the frequency domain. So, when we sample an autocorrelation, the Fourier transform of the autocorrelation is going to get aliased. In fact, we know what is the Fourier transform of the autocorrelation from our basic understanding of signals and systems theory. So, if phi of t has a Fourier transform phi cap omega, then r phi phi or the autocorrelation of phi has a Fourier transform given by the squared magnitude of phi cap omega. This is a basic result in signals, systems and transforms. If you recall, the Fourier transform of the autocorrelation function is the power spectral density in the Fourier domain. Anyway, I leave it as an exercise for you to prove this. It is a basic result in signals and systems theory. Exercise review the proof, but what we intend to do is to use this result to our advantage. Now, when we sample sampling r phi phi tau at tau equal to m, sampling rate of 1 essentially, mean in the Fourier domain, summing up all aliases. In other words, constructing the sum, summation k going from minus to plus infinity, phi cap omega plus 2 pi k mod squared. You know, recall what you need to do when you sample is to shift the Fourier transform on the frequency axis by every multiple of the sampling frequency. The sampling frequency on the angular frequency axis is 2 pi divided by 1. So, if you like, I can write 2 pi divided by 1 here to make matters very clear and every multiple of this, every integer multiple. So, 2 pi by 1 times k for all integer k, shift the original Fourier transform by all these multiples of the sampling frequency on the angular frequency axis and add up all these translates, add up all these aliases. So, this is the Fourier, of course, there is a constant. So, you know, there could be a constant here. Let us call that constant kappa 0, constant associated with sum and we can just ignore that constant for the moment. Even if that constant is there, we can, you know, take care of it appropriately in the rest of our discussion. So, we should not pay too much of attention to this constant when we discuss further, but we know that it is there. Anyway, you know, if this, if this sequence is an impulse, then it is discrete time Fourier transform. So, I am using the abbreviation DTFT for discrete time Fourier transform. It is discrete time Fourier transform, which is essentially, essentially this, essentially the Fourier transform of the sampled autocorrelation, must be a constant. So, the Fourier transform, the discrete time Fourier transform of an impulse is a flat, a constant function and therefore, we now have a clear cut criterion in the Fourier domain. In order that the function phi t be orthogonal to its integer translates, we require that this quantity, this sum of aliases of the power spectral density must be a constant. So, in other words, what we are saying is the following. We are saying phi t is orthogonal or phi t minus m for all integer m forms an orthogonal set is equivalent to summation k going from minus to plus infinity, phi cap omega plus 2 pi k mod whole squared is a constant. Well, it is not too difficult to prove this both ways. So, if this is a constant, then essentially what we are saying is when you take mod phi cap omega the whole squared and add it to its aliases, that means you sample the autocorrelation at the integers, you get the Fourier transform of an impulse, the discrete time Fourier transform is invertible. So, this result works both ways. Anyway, the requirement of orthogonality to integer translates amounts to a requirement of now we introduce the term sum of translated spectrum. So, you know remember when we were trying to discretize the scale parameter, we had brought in the sum of dilated spectrum. Here, we have a sum of translated spectrum. We shall call summation k going from minus to plus infinity phi cap omega plus 2 pi k mod whole squared as the sum of translated spectrum of phi. You know if you look at it, it is indeed that you are taking the original spectrum translating it by every multiple of 2 pi and adding up these translates summing. So, the name is very clear and we shall abbreviate sum of translated spectra by s t s. So, s t s and again we are going to have primary and secondary arguments. The secondary argument here is phi and 2 pi and the primary argument is essentially omega because you are taking a sum of translated spectra of the spectrum of phi, phi cap and the translations are all multiples of 2 pi. Secondary arguments and primary argument. The primary argument of course is frequency as expected. So, in general to define this term clearly s t s of phi with a translation t and primary argument omega is essentially sum k going from minus to plus infinity phi cap omega plus t times k mod squared. Anyway, with this little notation introduced, we take the same strategy as we did when we relaxed the condition for the sum of dilated spectra. So, you know when we talk about discretizing the scale, we need to essentially relax the requirement of the sum of dilated spectra to be a constant to where it is between two constants, between two positive constants. So, we said well even if we cannot quite get the sum of dilated spectra to be a constant, we will be happy if it is between two positive strictly non-zero and finite constants. Now, something true, something similar will be true for this case and in fact, now we will also bring out a beautiful relationship between relaxation of this requirement in the tau domain or in the shift domain and in the frequency domain. Now, if we look back, it is easier to start from the tau domain. If we look back at the function phi 1 t for example here, so remember phi 1 t looks like this. Now, if you ask me about the dot product of phi 1 t with its integer translates, the relaxation that we are asking for is that the dot product is of course, 0 definitely from shifts of 2 and larger in magnitude that is shifts of 2, 3, 4 and so on and minus 2, minus 3, minus 4 and so on. But then you know it is only for 1 and for minus 1 that we are asking for a relaxation here and we can even actually calculate those two dot products. The dot product of phi 1 t with itself would have a certain value, it is of course, going to be the energy in phi 1 t and if you take the dot product of phi 1 t with its translate by 1 and translate by minus 1, they are expected intuitively, you can see they are expected to have a smaller value. So, in other words, the relaxation that we are asking for is that this autocorrelation sequence, the autocorrelation sample at the integers is not quite an impulse but close to an impulse. That means, it is a sequence which is non-zero for very few values around n equal to 0 and that manifests in the frequency domain as the sum of translated spectra not quite being a constant, but being between two positive constants. We shall exactly calculate these quantities now and prove what we are saying mathematically. So, consider for phi 1 t in this case, consider the autocorrelation r phi 1 phi 1 evaluated at 1 and minus 1. It is not at all difficult to see that they are equal shifting by plus 1 and taking the dot product or shifting by minus 1 and taking the dot product, they give you the same answer. And this is essentially equal to integral from minus to plus infinity phi 1 t phi 1 t minus 1 dt which I shall calculate graphically now. Now, graphically this amounts to finding the area under the product of the following two functions. So, this is what phi 1 t looks like and this is what phi 1 t minus 1 looks like. So, the product is non-zero only in this region between 1 and 2 and in fact when we take the product and integrate, we do not really have to worry about its being between 1 and 2. I mean it would be as well if this is the same thing is shifted to lie around 0. So, this integral r phi 1 phi 1 of 1 or r phi 1 phi 1 of minus 1 is essentially the following integral. You know you could look at a function of the form 1 minus t between 0 and 1 and the function t between 0 and 1 and you could take their product integrate between 0 and 1 and that would be essentially this autocorrelation point. So, it is essentially the function t between 0 and 1 and you could take their function point. So, it is essentially integral t times 1 minus t d t between integrated between 0 and 1 which amounts to integrating t minus t square between 0 and 1 and that is a very easy integral to evaluate that is half minus one third and that is easy to see to be one sixth. So, we have the autocorrelation at 1 and minus 1. Now, let us find the autocorrelation at 0 to complete the discussion. So, r phi 1 phi 1 at 0 is essentially integral phi 1 t into phi 1 t. So, phi 1 square t d t integrated of course from 0 to 2 and that is very easily seen to be M n. You know if you look at it graphically it is very easy to see that this amounts to integrating the two halves so to speak. So, it is two times the integral from 0 to 1 of phi 1 square t t from the symmetry about t equal to 1 and this is an easy integral to evaluate. This is two times integral from 0 to 1 t whole square d t. So, that is t cube by 3 into 2 integrated from 0 to 1 clearly equal to 2 by 3. Therefore, we have a very clear set of autocorrelation values now and we can actually find out the discrete time Fourier transform of the autocorrelation sequence. So, the discrete time Fourier transform or the d t f t of the autocorrelation sequence r phi 1 phi 1 tau tau at the integers is essentially the d t f t of the sequence 2 by 3 at the point 0 1 by 6 at the point 1 and 1 by 6 at the point minus 1 and the outside that interval of course it is 0. I will not in fact need to even write this that is understood when we use a notation like this and this is a very easy discrete time Fourier transform to evaluate. This is 1 by 6 e raised to the power j omega plus 2 by 3 plus 1 by 6 e raised to the power minus j omega. Now please note I have used small omega here because we are talking about discrete time Fourier transform. So, I should be using the normalized frequency, but then I could as well here you know we are interchanging the ideas of analog and discrete time and therefore, we can as well replace this by capital omega. So, in fact what we infer is that summation k going from minus to plus infinity phi cap omega plus 2 pi k mod whole squared is essentially 2 by 3 plus 1 by 6 e raised to the power j omega plus 1 by 6 e raised to the power minus j omega possibly to within some constant. So, you know you may have to multiply this by some constant depending on the you know scaling of phi 1 t if that is the case. So, let us forget about this ignore this that is not of great consequence to us. What is of consequence is only this and this is easy to expand this is essentially 2 by 3 plus 1 by 6 into e raised to the power j omega plus e raised to the power minus j omega which is 2 by 3 plus 1 by 6 into 2 cos omega and that is 2 by 3 plus 1 by 3 cos omega or in other words 2 by 3 into 1 plus half cos omega. Now as expected this is always non negative what is more it is also very clear that the sum of translated spectra namely 2 by 3 into 1 plus half cos omega strictly lies between 2 positive bounds. The lowest possible value that this can take lowest positive value is obtained when cos of omega is minus 1 in other words this is 1 minus half. So, this is bound to lie between 1 by 3 when cos omega is minus 1 and when cos omega is plus 1 this becomes 1 so you know a relaxation of the requirement in time or in tau has also led to a corresponding relaxation in the frequency domain. And now we could employ the same strategy as we did when we relax the requirement of the sum of dilated spectra. Now that we can see the sum of translated spectra lies between 2 positive bounds we could say well even though phi 1 t by itself cannot give us an orthogonal multi resolution analysis can be construct out of phi 1 t by using the sum of translated spectra another function let us call it phi 1 till day in such a way that phi 1 till day gives us an orthogonal multi resolution analysis. So, let us explore that possibility in fact let us strategically define such a phi 1 till day as we did taking inspiration from the sum of dilated spectra. So, let us define phi 1 till day t in terms of its Fourier transform unlike the case of sum of dilated spectra here we will also be able to give a meaning to the definition that we make. So, we will define phi 1 till day cap of omega to be phi 1 cap omega divided by phi 1 till day by the square root positive square root of the sum of translated spectra of phi 1 with translation of 2 pi as a function of omega. And we justify this definition by noting that the denominator is between 2 positive bounds. In fact, the denominator is known to be between let us put that back here the denominator is known to be between one third and one. So, we are justified in making this definition here this division will not blow up towards infinity and neither will it go all the way down to 0. So, no frequency would get annulled by this going to infinity and there would be no blow up of this definition when the denominator goes to 0 this definition is meaningful. Now, let us explore the sum of translated spectra of phi 1 till day with of course, 2 pi as a translation parameter and omega as the primary argument of course, by definition this is summation k going from minus to plus infinity phi 1 till day cap omega plus 2 pi k mod squared and let us substitute this now again here before we directly make a substitution we would like to establish a property an important property of the sum in the denominator. So, the periodicity of the sum of translated spectra that is easy to establish. What we will show is that the sum of translated spectra of any function phi with translation parameter 2 pi is periodic with the period of 2 pi that is very easy to show indeed by very definition the sum of translated spectra phi with a translation of 2 pi and the argument replaced by omega by 2 pi is essentially some k going from minus to plus infinity phi cap omega plus 2 pi plus 2 pi k mod squared and this as you can see is going to be equal to summation k going from minus to plus infinity phi cap omega plus 2 pi k plus 1 mod squared and we once again note when k goes from minus to plus infinity k plus 1 also goes from minus to plus infinity. So, you could have as well replaced k plus 1 by k we could as well write k here and therefore, proved this is the same as s t s phi 2 pi evaluated at omega. So, that was a little aside was the kind of corollary that we needed to prove. Now, we will prove the main or we will establish the main result the sum of translated spectra of phi 1 till day. So, you know the sum of translated spectra of phi 1 till day is now going to be summation k going from minus to plus infinity phi 1 cap omega plus 2 pi k mod squared divided by the numerator has the square root of the sum of translated spectra when you square it becomes the sum of translated spectra of phi 1 2 pi evaluated at omega plus 2 pi k. Now, we invoke the periodicity of this. So, this omega plus 2 pi k is redundant here and in fact, this can be replaced by just omega and once this is replaced by omega then this s t s has nothing to do with the summation of the summation index k. So, it can be brought outside. So, this s t s becomes essentially 1 divided by s t s phi 1 2 pi evaluated at omega and the numerator we have summation k from minus to plus infinity phi 1 cap omega plus 2 pi k the whole squared. But, this is familiar this is essentially s t s. In fact, this is the same as the denominator as you can see and therefore, this is clearly s t s phi 1 plus 2 pi k. Phi 1 tilde I am sorry s t s phi 1 2 pi evaluated at omega divided by s t s phi 1 2 pi evaluated at omega and once again invoking the fact that s t s this quantity lies between one third and one. It is all right to cancel this quantity from the numerator and denominator and to obtain that this is equal to 1. This is justified because this does not go to 0 or infinity. So, we have shown a very important result. We have shown the sum of translated spectra of phi 1 tilde evaluated at omega is a constant. In fact, that constant is 1 that is interesting. And we know what that means that means that phi 1 tilde is now orthogonal to its integer translates phi 1 was not orthogonal to all its integer translates the trouble was with 1 and minus 1. But, phi 1 tilde is orthogonal to all its integer translate and now we shall look at the nature of phi 1 tilde. So, let us make that remark very clearly phi 1 tilde is orthogonal to its integer translates all integer translate unlike phi 1. Now, you know that is so far so good. I mean if one cannot describe phi 1 tilde what is the point in talking about its orthogonality. So, we must be able to get a way of constructing it and getting a feel of what this phi 1 tilde looks like. So, let us do that before we do anything else. If we cannot do that then all the rest of the discussion is meaningless. So, let us try and obtain the nature of phi 1 tilde. What does phi 1 tilde look like? In fact, let us go back to the Fourier domain. So, we have phi 1 tilde cap omega is phi 1 cap omega divided by the sum of translated spectra of phi 1 with a translation parameter of 2 pi evaluated omega, but with a square root here. Let us write this down explicitly. So, let us write the denominator down explicitly. So, we have phi 1 tilde cap omega is essentially phi 1 cap omega divided by the square root positive square root of 2 by 3 into 1 plus half cos omega. Now, let us write this down in the form of an exponential or binomial expansion. So, let us rewrite this 2 by 3 to the power minus half into 1 plus half cos omega to the power minus half. Now, look at this expression. This is of the form 1 plus some gamma to the power minus half. Now, 1 plus gamma to the power minus half can be expanded with our knowledge either of the Taylor series or of the generalized binomial theorem. So, 1 plus gamma to the power minus half given mod gamma is strictly less than 1. Please note, can be expanded as follows. Well, you know 1 plus gamma to the power r in general for real r using what is called a generalization of the binomial theorem can be expanded as 1 plus r times gamma plus r into r minus 1 by 2 factorial times gamma squared plus and so on. So, I will write one more term r into r minus 1 into r minus 2 by factorial 3 times gamma cubed and then continue. Now, you know the precise terms in the expansion are not so critical. What is critical is the nature of the terms. A typical term here is of the following form. It is of the form some constant let us call it suppose the p th term some constant kappa p times gamma to the power p and this is essentially kappa p times cos omega to the power p into half to the power p that is interesting. Now, let us once again recall of course here p is a positive integer. Now, you know it is a basic result in trigonometry or for that matter even in complex analysis that cos omega to the power of p can be expanded in terms of e raised to the power j omega. In fact, we can easily do that we can simply expand cos omega to the power p is essentially e raised to the power j omega plus e raised to the power minus j omega by 2 to the power p. So, it can be expanded as a series in e raised to the power j omega e raised to the power j omega k for integer k. Now, if we take each of these terms we can see that essentially as a contribution of the form e raised to the power j omega k where k is an integer. We can aggregate the coefficients of e raised to the power j omega k coming from each of these terms and therefore, essentially with this expansion we could aggregate coefficients of e raised to the power j omega k k integer and we could show that phi 1 tilde cap is of the form some summation k going from minus to plus infinity some C k tilde these coefficients times e raised to the power j omega k times phi 1 cap omega and this is very easy to invert in time. In fact, if we look back at this expression what we are saying essentially is the Fourier transform of phi 1 tilde is the Fourier transform of phi 1 multiplied by essentially the discrete time Fourier transform of the sequence C k tilde here. Now, if you take any one term here in this expansion e raised to the power j omega k times phi 1 cap omega it is the Fourier transform of phi 1 shifted by k. So, if we take the inverse Fourier transform on both sides essentially we have phi 1 tilde t is of the form summation k going from minus to plus infinity C k tilde phi 1 t plus k. So, phi 1 tilde is essentially a linear combination of phi 1 shifted by integer translates and even when you shift phi 1 which is a piecewise linear function by integer translates it still remains piecewise linear. When you sum together piecewise linear functions the sum is piecewise linear. So, it is very clear at this point that phi 1 tilde is going to be a piecewise linear function. What else is it going to have what are the what is the nature of these C k tilde is and how do we construct the multi resolution analysis we shall see in the next lecture. So, we will proceed from this point in the next lecture. Thank you.