 A warm welcome to the 25th lecture on the subject of wavelets and multirate digital signal processing. Let us put in perspective what we are going to do in the lecture today. In the previous 2 to 3 lectures, we have been moving from continuous translation and scale to discrete scale, where the discretization of scale is logarithmic in nature and then specifically to dyadic scale, where the scale is discretized in powers of 2. However, the translation parameter is still continuous and today we intend to discretize the translation parameter with dyadic scale, leading to the specific situation of dyadic multi resolution analysis that we began this course with. Now, before we proceed to prove a very important principle, which essentially is the principle of discretization of translation in the context of dyadic scale called the theorem of multi resolution analysis and that is how I titled the lecture today. So, I have focused today on the theorem of dyadic multi resolution analysis. Essentially, an effort to discretize the translation parameter knowing that the scale has been discretized in a dyadic manner in powers of 2. But before we proceed to prove that theorem to establish the results of that theorem, we would like to bring out the link between what we did in the previous lecture and where this theorem stands in the next few minutes. You see, yesterday we had indirectly talked about orthogonal filter banks. We had said that if the so called wavelet and the corresponding synthesis wavelet. So, remember we had talked about a psi and a psi tilde, a psi on the analysis side and a psi tilde on the synthesis side. And we said in principle, the more general situation is that you have different wavelets on the analysis side and the synthesis side when you discretize the scale parameter in a logarithmic manner. So, what it really means is that the most general situation is where the analysis wavelet and scaling function is not the same as the synthesis scaling and wavelet function. And this is a discovery for us beyond what we have been doing all the while before we started talking about uncertainty. Now, we would like to christen to give a name to those kinds of filter banks which lead to different wavelets on the analysis and the synthesis side. And then we would like to make specific the case where they are the same. So, therefore, let us give these names. Filter banks with different analysis and synthesis wavelets and scaling functions are called bi-orthogonal, bi-orthogonal filter banks. So, these are more general. Now, please remember that when we talk about filter banks here, we are always referring to filter banks with perfect reconstruction. And I think that point must be written down very clearly for emphasis. We are always talking about filter banks with perfect reconstruction. In particular, when the analysis and the synthesis wavelets are the same, wavelets and scaling functions, then those filter banks are called orthogonal. We shall be largely focusing on orthogonal filter banks today. But before we proceed to the filter bank where we have essentially discretized of course, last time when we talked about filter banks, we were talking about continuous time filters. Remember, now we have moved one step ahead from the kinds of filter banks that we referred to earlier on in this course where we only spoke of discrete time filters. Now, we have continuous time filters in the filter bank and we will just recall a couple of important steps that led us to the discussion towards the end of the previous lecture. Namely, we said that the k-th branch in general, the k-th analysis branch would take the input x and subject it to a filter with frequency response psi cap a naught raise the power of k omega, complex conjugate. A naught as you call was n number greater than 1 and of course, k ran over all the integers. k-th synthesis branch essentially took the output of the k-th analysis branch as its input and subjected it to the action of a filter whose frequency response was psi tilde cap a naught raise the power of k omega and all these k-th synthesis branches were added together on the synthesis side to produce the output. We also specified what psi tilde was. We began by specifying psi tilde in terms of its Fourier transform. In fact, it is very difficult to specify in terms of the time domain expression. So, we had specified psi tilde cap omega to be psi cap omega divided by the sum of dilated spectra of psi with the parameter a naught evaluated at omega. SDS as you recall was an abbreviation for the sum of dilated spectra and we had defined the sum of dilated spectra as follows. Dilated spectra because of this term the frequency variable is replaced by a dilated version of the frequency variable a naught raise the power k times omega. That is why a sum of dilated spectra and we noticed that provided there existed, this means there exists. So, there exists c 1 and c 2. So, that c 1 is strictly greater than 0, c 2 is strictly finite strictly less than infinity and this SDS of psi and a naught for all omega is between c 1 and c 2. We were guaranteed that psi is admissible and this followed from the upper bound here. Further, we were guaranteed because of c 1 psi tilde cap was meaningful and in fact, because of this condition on upper and lower bounds, it was also true that psi tilde was admissible and in fact, the bounds on SDS for psi tilde cap were essentially 1 by c 2 and 1 by c 1. Now, with all this recapitulation, we also said that in case the sum of dilated spectra SDS psi a naught was constant for all omega, you had an orthogonal filter bank there. In fact, the sum of dilated spectra being a constant for all omega essentially means that there is no difference in the way different frequencies are treated and in fact, we could see that if we had an ideal filter, if we had an ideal so called quote unquote ideal frequency response for psi. For example, a frequency response which was 1 between pi and 2 pi, then we were automatically get perfect reconstruction and consequently a constant sum of dilated spectra. So, we begin to see the interrelationships. Of course, that is not the only case, the ideal case is not the only case and at this point, I would like the class to reflect on other possible cases that they might be. I should not answer the question right away because some curiosity should be evolved. So, I put this exercise or this challenge before the class partial challenge partial exercise. Come up with examples of psi cap which satisfy the requirement with C 1 equal to C 2. So, essentially the sum of dilated spectra being a constant anyway not just the ideal case. Well, what I wished to do before we proceed to the dyadic case is to note one more variation that can be introduced. You see here, we are admitting the idea of bi orthogonal filter banks. Filter banks where the analysis and the synthesis side are different in terms of the wavelets and scaling functions involved. Now, we can see in a minute that once we have this condition satisfied by psi, we can also construct an equivalent orthogonal filter bank from that psi where the analysis and the synthesis wavelets are the same, but different from psi and that is done as follows. So, construction of an orthogonal filter bank. So, let us define psi double tilde again in terms of its Fourier transform. Psi double tilde cap omega is defined to be psi cap omega divided by not the sum of dilated spectra, but the square root the positive square root of the sum of dilated spectra. Now, again please note that because of the upper and lower bound on s t s, it is meaningful to put this in the denominator. After all, we are guaranteed because of the positivity that 0 is strictly less than c 1 to the power half. I mean I am talking about the positive square root less than or equal to square root the positive square root of s t s psi a naught omega continued this way less than equal to c 2 to the power half positive square root again and this is of course, strictly less than infinity and therefore, it is meaningful to divide in the denominator that is a meaningful operation. With that observation, we shall first prove that psi double tilde cap or rather psi double tilde is admissible and to prove that it is admissible, we only need to establish the sum of dilated spectra. So, we need to consider the sum of dilated spectra of psi double tilde with a naught evaluated at omega. If we can say something about this, we have done our job and in fact, that quantity as you can see with great ease is simply summation k going from minus to plus infinity psi cap a naught raise the power k omega mod squared divided by well please remember the sum of dilated spectra is independent of multiplication of the independent variable by a naught to the power k. Anyway, what I will do is to you know still write down. So, here we square this. So, we get the denominator psi as it is a naught omega and what we are saying is that this is not affected. So, this is when we replace omega by a naught to the power of k, I will just recall that reasoning. You see what we are saying is S T S of any psi a naught of omega is equal to S T S psi a naught to the power of m omega. We proved this yesterday, but let us just quickly recall the proof in general. Indeed S T S psi a naught raise the power of m omega is essentially summation k running from minus to plus infinity psi cap a naught raise the power of k a naught raise the power of m omega mod squared and please note that this is essentially a naught raise the power k plus m times omega you know the spectrum evaluated that frequency modulus squared m is fixed. So, when k runs over all the integers k plus m also runs over all the integers. So, this is equivalent to summation k going from minus to plus infinity psi cap a naught to the power k omega the whole square and that proves what we wanted to. So, with this observation we have essentially S T S psi double till day a naught evaluated at omega is summation k running from minus to plus infinity psi cap a naught raise the power k omega mod squared divided by S T S psi a naught at omega. And note that this is essentially a naught S T S psi a naught at omega. And therefore, the numerator and denominator cancel on account of the bounds the lower and the upper bound and this becomes equal to 1 for all omega. And therefore, we have a constant something even more than bounds we have a constant sum of diated spectra in the context of psi double till day. And therefore, psi double till day can be used both on the analysis and the synthesis side the fact that it is sum of diated spectra is a constant means that it is admissible. And therefore, psi double till day is a wavelet and an orthogonal wavelet at that. So, psi double till day is admissible on account of satisfying upper equal to lower bound on S T S equal to 1 and psi double till day is an orthogonal wavelet psi double till day can be used both on the analysis and the synthesis side. So, what we have done is to complete the process of constructing an orthogonal an equivalent orthogonal wavelet given a bi orthogonal situation a wavelet psi which gives you upper and lower bounds on its sum of diated spectra. Now, a few words before we proceed to the next step the discretization of translation one must note that in all this discussion one has not discretize the translation one is still talking about continuous translation parameters tau. And when we talk about orthogonal here we are talking about orthogonality keeping the continuous translation parameter we are working in continuous time here not in discrete time. So, for example, if you were to take the Har wavelet you could certainly show that it has these bounds, but one must not confuse the fact that the Har wavelet gives orthogonality with those discrete shifts with the concept of orthogonality as we have just talked about a minute ago where the translation is continuous. So, for example, the Har wavelet will not satisfy the upper and the lower bounds being equal to this being being the same being equal to one for example, or any constant for that matter because there the orthogonality is with respect to discrete shifts and that is a weaker requirement. In fact, even though it is a weaker requirement that is what we strive towards in implementation because we do not want to retain again the whole continuous translation parameter here though notionally it is good to know. So, now what we are going to do is to accept a wavelet psi which has this property of admissibility and reconstructibility because of the lower bound. So, it has a C 1 and a C 2 and we are going to ask can we discretize it given a naught equal to 2 specifically. So, we will concentrate on a naught equal to 2 we will accept a wavelet which gives us a sum of dilated spectra being between two positive bounds and we will discretize the translation parameter strategically to construct a dyadic multi resolution analysis. And that is where we once again recall the axioms of a dyadic multi resolution analysis. So, now we put all our discussion in perspective what we did earlier on in this course it is a special case of the most general wavelet transform. So, the dyadic multi resolution analysis or M R E whose examples constitute Har M R E, Dobash M R E's. And so on are essentially the special case where a naught is equal to 2 the wavelet obeys the requirement S T S between two bounds between two positive bounds for all frequencies essentially the requirement for discretization. And of course, here it obeys the requirement with a naught equal to 2 that is what we mean it may not obey the requirement for all values of a naught greater than 1, but it obeys the requirement with a naught equal to 2 for this. So, as you can see these bounds would of course, depend on a naught in general well where the wavelet has these two properties and the wavelet admits discretizing the translation parameter. How we discretize the translation parameter now can be understood by a slightly different consideration how should we go about discretizing the translation parameter should we do it in the same way for all the branches or should we do it differently. The answer is very simple. You see if you look at what you are doing on the kth branch on the kth branch if you like the kth analysis branch the output is essentially is broadly a band pass function. It is a function which is significant in a particular band of frequencies not around 0 band pass remember and for different values of k there is a logarithmic variation of this band. Now, we shall invoke a generalization of the sampling theorem as we know it to the case of band pass functions and I shall rather than give a general statement illustrate with an example. Suppose we consider a band pass function where the band on omega lies between pi and 2 pi. So, suppose we consider a band pass function like this what could be the sampling rate you see after all when we talk about discretizing the translation parameter we are essentially talking about sampling the output of this filter on the kth analysis branch and feeding those samples instead of the continuous function to the input of the kth synthesis branch. So, how do we sample in such a way that we do not lose something that is an equivalent question to how do we discretize the translation parameter. So, you see if you wish to sample this you have two alternatives you could sample it obeying the Nyquist criterion thinking of the highest frequency as 2 pi or you could sample it remembering that this band is blank here and you have an occupancy only of pi. Now, this second approach where you sample noting that the occupancy is only pi can be used in specific circumstances and this is one of them and in fact you can in principle sample this at a rate equal to twice or more not more again more you know has to be taken with a pinch of salt not any value more, but definitely with a sampling rate twice of the band. So, in other words the band occupancy is pi and therefore, we could use a sampling rate equal to 2 pi divided by this band occupancy pi which is 2. What I am trying to say is that you see when you use a sampling rate of 2 effectively you are saying that so what does it really mean you know this is this is the well let me explain it in a slightly different way you see if you if you simply use the Nyquist criterion we should have f s as the sampling frequency in such a way that 2 pi f s is equal to 4 pi all right. So, f s is equal to 2 in that sense we could use a sampling rate of value 2, but we can also do with a sampling rate 2 pi f s is equal to only 2 times the band which is pi this is the band occupancy pi and therefore, f s is equal to 1 this is what I am trying to say this is the band pass sampling theorem where you can look at the band occupancy and decide your sampling rate not the maximum frequency and suppose we do that suppose we use a sampling rate of 1 what are we doing we are essentially adding all aliases which are shifts of the original spectrum by 2 pi multiplied by 1 times k for all integer k. So, take the original spectrum shifted by all multiples of 2 pi times the sampling rate 1 essentially 2 pi k for all the spectrum all integer k and add up these translates let us do that. So, let us take the original spectrum in fact now I will show both the positive and the negative sides of the frequency axis for completeness I will contract the drawing to make it clear and so on. So, my original band is here when I translate by 2 pi backwards I get this alias. So, I will just show that this would go back here when I translate by 2 pi forward this comes here. So, I show that with dot and dash and this goes there of course, I could keep doing this. So, I now must translate in all multiples of 2 pi. So, I must take a translation by 4 pi a translation by 6 pi and so on. So, when I translate this band by 4 pi backward and forward when I translate this by 4 pi backward I would have this coming here and of course, when I translate this by 4 pi forward I am going to 5 pi. So, I do not need to bother there when I translate this pi multiples. So, you know 6 pi and so on keeps going further back. So, I do not need to bother about that and this 2 I need to worry about what will happen when I translate it by 4 pi and 6 pi. So, once again when I translate this by 4 pi it comes here and then 6 pi and 8 pi and so on. So, now you can see that other translates are not going to interfere with this original part of the spectrum this and this is unpolluted is unpolluted unaffected. So, I could retrieve the original band pass signal by putting a band pass filter between pi and 2 pi this is the band pass sampling here. Now, you know one should not generalize this too quickly this does not mean that wherever I put that band pass sampling rate of pi I could use a sampling rate of 2 pi I mean essentially you know twice that band and not twice the highest frequency that is not true in general. It is true depending on the location of the band as well and that is why the band pass sampling theorem is a little more complicated than the low pass sampling theorem the conventional Nyquist theorem as we know it, but it is certainly more economical and in fact what we are doing in dietic multi resolution analysis is essentially and very implicitly very covertly invoking the band pass sampling theorem. If we just take a minute to understand this much of the rest of the discussion would be far more clear. So, what I have shown you for between pi and 2 pi can now be repeated for between 2 pi and 4 pi and then between 4 pi and 8 pi and also between pi by 2 and pi and so on the same principle would hold. And for all those cases one could use a sampling rate which is twice the band and not twice the highest frequency using the same kind of an argument. So, what I am trying to say is that for different branches on the analysis side I would have to use different sampling frequencies now and in fact those sampling frequencies are also going to be related logarithmically they are going to be in powers of 2 and recall that in the dietic multi resolution analysis as we know it that is exactly what happens when you go from v 0 to v 1 you have twice the number of points twice the number of functions in a certain interval when you go from v 1 to v 2 again the number of functions is doubled when you go from v 0 to v minus 1 the number of functions is halved. So, all that is essentially a manifestation of this band pass sampling theorem. Now, we must ask the question which we intended to all the while when we discretize the translation parameter with a naught equal to 2. So, let us now focus on a naught equal to 2. So, we need to use a logarithmic change of the form 2 to the power k of sampling. So, what we are saying in effect is that on the kth branch the sampling rate relates to 2 raise to the power k and we do not explicitly need to ensure it because that is what the dietic M R A axioms do. Let us now put down the theorem of dietic multi resolution analysis as we know it and as we are going to now prove formally step by step. The axioms of a dietic M R A are as follows. The first axiom you will recall what we are going to do is to put these axioms in the light of the discussion that we have had. The first axiom is you have a ladder of subspaces v 0 contained in v 1 contained in v 2 and so on. I shan't go over the details of these v 0's and v 1's, but I shall bring out the connection between these subspaces and the discussion that we have been having. Essentially, each of these subspaces for example, v 0 is a subspace where the functions are band pass in a certain band, v 1 is the subspace where the functions are band pass in the next higher band, v 2 in the next higher band and each time the frequency is doubled, frequency occupancy is doubled as you go downwards the frequency occupancy is halved and therefore, as you go upwards the sampling rate is doubled as you come downwards the sampling rate is halved. So, that was the first axiom. The second axiom is that of course, the union axiom. So, union v m m over all z with a closure is l 2 r which essentially is an axiom of perfect reconstruction or when you collect all those incremental subspaces together you go back to the whole input x that is what we are saying. So, it is an axiom of reconstruction. Now, the third one is of course, an axiom where we are saying essentially that we always remain in l 2 r. So, as we go downwards if we are going towards smaller and smaller bands finally, we are going to reach a constant function with 0 power that is what we are saying. The fourth axiom is important you see the fourth axiom says that if x of t belongs to v 0 then x of 2 raise to the power of m t belongs to v m and implicitly this provides for logarithmic sampling. If you reflect just a minute on this it is essentially a statement of logarithmic sampling. In fact, logarithmic sampling with the logarithm of 2 and of course, the axiom of translation that is if x t belongs to v 0 then x t minus m belongs to v 0 for all integer n essentially says that we have a uniform sample. So, now we have interpreted these axioms in the context of the discussion that we have had and finally, we have the axiom of an orthogonal basis. The orthogonal basis axiom says that we have there exists a phi t. So, that phi of t minus n with all integer n is a basis for v 0 and given this and given the fourth and the fifth axioms we have a corresponding orthogonal basis for each of the v n's in the ladder. Now, what is the meaning of this axiom in the context of our discussion? This axiom essentially gives us a way to reconstruct from samples. After all the coefficients in the expansion of the function with respect to phi t are essentially like the generalized samples of the fang of the output after filtering. Now, you know actually v 0 is a collective subspace, but what we are trying to do here. So, you know although here we are sampling we are sampling a collective subspace not an incremental subspace. The theorem of multi resolution analysis is going to give us an incremental subspace. What we are saying here is a collective subspace you know we are saying that if you go all the way up to 2 pi for example, you need to sample using the conventional Nyquist theorem at 4 pi. If you go all the way to 4 pi you need to sample at 8 pi that is what we are saying here. This is when we write down this orthogonal basis we are essentially giving a scheme for generalized sampling and reconstruction from the samples, but with the conventional Nyquist theorem. Now, we put down finally the theorem of multi resolution analysis which says that given axioms 1 to 6 there exists a function psi. Of course, this psi t is a function in L 2 R of course, and in fact psi t is also in v 1 such that psi 2 raise the power of m t minus n for all integer m and all integer n forms an orthogonal basis for L 2 R. This is the theorem of multi resolution analysis of dietic multi resolution analysis. Now, let us put down the statement of the theorem in the language of our discussion. In this statement we are saying we have this psi t which covers the incremental subspace between v 0 and v 1 and we can use translates of that psi t to cover the incremental subspace. What are we saying there effectively? We are saying we can band pass sample that band which is covered by the incremental difference between v 0 and v 1. So, you know if you think about it this wavelet its spectrum would cover a band now and not a band starting from 0 and we are saying we can move from low pass sampling to band pass sampling and we can discretize the axis with band pass sampling. So, for example, in the same in the light of what we discussed a few minutes ago, if psi essentially covers the band between pi and 2 pi we are saying you could do by sampling that band at 2 pi and not at 4 pi. So, you could be sampling you know you could go from v 0 to v 1 when you represent functions in v 1 you need to sample at twice the rate as compared to when you represent them in v 0 if you use the conventional theorem. What we are saying is well you could represent functions in v 1 by a low pass sampling up to v 0 and a band pass sampling for the increment. Then when you go from v 1 to v 2 you could either use a low pass sampling for v 1 and then a band pass sampling for w 1 or you could use a low pass sampling for v 0 a band pass sampling for w 0 and a band pass sampling for w 1 and if you take this argument to the extreme the entire representation of a function in L to R can be done by band pass sampling each of these incremental bands that is what the theorem of multi resolution analysis essentially says. And in the next lecture our endeavour shall be to prove this theorem formally starting from these axioms now that we know where this theorem fits in the entire scheme of things. Thank you.