 A warm welcome to the 23rd lecture on the subject of wavelets and multirate digital signal processing. Let us spend a couple of minutes in recalling the discussion in the previous lecture. We had in the previous lecture, built up the idea of decomposition and reconstruction in the short time Fourier transform and in the continuous wavelet transform. The central theme in decomposition and reconstruction was to project in the sense of projection on a vector, project on the basis vectors, the function to be decomposed and to reconstruct the function from its components. By multiplying each component by a vector in that direction, this simple idea enabled us to interpret decomposition and reconstruction both in the short time Fourier transform as also in the continuous wavelet transform. The short time Fourier transform was indexed by translation and modulation. The continuous wavelet transform was indexed by translation and scale and we saw that there was a little bit of asymmetry between translation and scale. Translation could be dealt with easily. Scale needed an additional weighting factor to deal with it when we reconstruct it, when reconstructing the function from its continuous wavelet transform. So, with that little recapitulation, let me put the discussion in the current lecture in perspective. So, in the current lecture, we are going to talk about admissibility in detail first and later we are going to proceed to the discretization of scale, the scale parameter in a particular way. Now, let us go back to the discussion that led us to admissibility or the idea of being able to accept a function psi t as a wavelet. We had only partly arrived at the answer and in fact, we had worked rather hard to evaluate a triple integral to arrive there. It also seemed to be getting out of hand because the triple integral needed to be dealt with one integral at a time. And although we had almost come to the final step, I think it is worth recapitulating the important steps that took us to the final point where we were. Let me just put down some important steps before we proceed. Some important steps in reconstructing x t from c w t x psi evaluated at tau and s. Now, the most important step was the evaluation of this triple integral here. The innermost integral was the so called component. So, that was integral x cap omega s to the power half psi cap s omega complex conjugated e raise the power j omega tau d omega. So, this was the innermost integral essentially corresponding to the c w t and with this we then had two outer integrals which took care of the translation and the scale parameter. And recall that with the scale parameter, we needed a weighting function the translation parameter did not required. So, we said we needed to reconstruct by doing the following by taking the component as it were multiplying by a so called unit vector and integrating over all components. The only catch being that we cannot just do this we need a weighting function to deal with scale. So, the integral on tau runs from minus to plus infinity, the integral on s runs from 0 to infinity and now let us write it all together once again I am repeating as you notice I am repeating a few steps it is worth doing that because this was a little formidable as a proof. So, it would be helpful to look at the important steps in the proof again anyway. So, the overall integral is as follows so I will write the triple integral here. The innermost integral is on omega from minus to plus infinity the next integral is again from minus to plus infinity and the final integral is from 0 to infinity here. So, this is the triple integral now you have the innermost integral taking you with x cap omega s to the power half psi cap s omega complex conjugated e raised to power j omega tau this takes care of this part. Now to write the rest times 1 by s to the power half psi t minus tau by s f s d tau d s before which there is a d omega there and we said we wanted to reverse or we wanted to change the order in which these needed to be dealt with. So, we would take this the last take this first and take this second this was the approach that we used. Now, without repeating all the discussion let me just put down one more important intermediate step what we saw is that after removing or after taking care of integral d tau first what was left by choosing f of s to be 1 by s squared was the following essentially it was 1 by 2 by integral from minus infinity to plus infinity x cap omega and integral involving s in a funny way 0 to infinity mode psi cap s omega the whole squared d s by s seemingly dependent on omega, but that is where our whole discussion ended in the previous lecture and we wanted to build on it today seemingly dependent on omega, but we want to do away with the dependence on omega here. So, this multiplied by e raised to power j omega t integrated over omega this is where we were and we noticed that it is essentially this which is causing a struggle had this been independent of omega this term here then this would be a constant which could be extracted and what would be left then is just the inverse Fourier transform of x cap omega which is just x t. So, let us summarize that step if we could make the integral from 0 to infinity psi cap s omega mod squared d s by s independent of omega and you see when it becomes independent of omega it is dependent on no other parameter. So, it would just be a constant let us call that constant c dependent on psi you see it of course depends on what psi you use, but beyond the dependence on psi there is a dependence on nothing else once you fix the psi this should then be independent of omega if we can make it. So, and that is what admissibility is all about. So, anyway if this could be done then what is left is essentially 1 by 2 pi times this constant c psi times the integral from minus to plus infinity x cap omega e raised to the power j omega t d omega which is essentially c psi times x t so simple and so beautiful. So, in fact now we also have an interpretation for this constant c psi the constant c psi actually tells us the factor by which x t has been multiplied in this process of reconstruction. So, we have a meaning to that integral too, but now let us look at that integral which we wish to make independent of omega little more carefully. So, consider the integral from 0 to infinity psi cap s omega mod squared d s by s and put as usual s omega equal to a parameter alpha. Now, we must remember that s always runs from 0 to plus infinity and omega is capable of running all the way from minus to plus infinity. So, of course, one thing that we notice is that omega equal to 0 is a problem. So, except for omega equal to 0 suppose we consider omega greater than 0 first in that case alpha equal to s omega would run from 0 to plus infinity when s runs from 0 to plus infinity. And of course, if you take the differential on both sides here as we did we would have d alpha is omega d s and of course, alpha is omega s and since omega is not 0 we have d alpha by alpha is d s by s. Whereupon this quantity integral from 0 to infinity psi cap s omega mod squared d s by s simply becomes integral from 0 to infinity psi cap alpha mod squared d alpha by alpha this is beautiful. Now, this is only based on the function psi it has nothing to do with capital omega. Let us consider the case of capital omega less than 0. Now, there we have an interesting situation. So, when capital omega is less than 0 again of course, we have alpha is s omega when s runs from 0 to plus infinity alpha runs from 0 to minus infinity. And therefore, we would have the integral becoming integral from 0 to minus infinity psi cap alpha mod squared d alpha by alpha. Now, we have a little bit of trouble here it is not easy to do this integral or related to the previous one. So, let us make a substitution again substitute beta is minus alpha whereupon when alpha goes from 0 to plus infinity beta goes from 0 to minus infinity or vice versa. So, when alpha goes from 0 to minus infinity beta goes from 0 to plus infinity that is what we wanted. So, let us make the substitution here also one can see that d beta is minus d alpha and beta is minus alpha whereupon d beta by beta is equal to d alpha by alpha as before. So, we have an interesting situation this is also for omega less than 0 we have 0 to infinity psi cap minus beta mod squared d beta by beta. So, we have an interesting situation you see we want to make the whole expression independent of capital omega. We seem to have different expressions for omega greater than 0 and omega less than 0. And if we want to make this whole expression independent of omega the two expressions must be equal and must both be finite the catch is in making them equal and finite. So, making this integral on S I mean independent of capital omega means two things one integral 0 to infinity psi cap alpha mod squared t alpha by alpha is equal to integral 0 to infinity psi cap minus beta mod squared t beta by beta and both of these are less than infinity. Now you know I do not need to specify that this is positive infinity because it is obvious that we are talking about positive alpha here this is all positive you see you are taking beta over a positive range. And this is a non negative quantity mod psi cap minus beta whole squared or mod psi cap alpha squared. So, they are all non negative quantities. So, this is an integral over a non negative integrand and therefore it must be non negative. So, it is obvious that this non negative quantity needs to be bounded there is no question of it is becoming negative. Now, this is not a serious requirement when psi t is real. If psi t is real then of course, we know that psi cap minus beta is equal to psi cap beta complex conjugated. And therefore, mod psi cap minus beta squared is the same as mod psi cap beta squared. And therefore, if I go back to the previous condition these are not separate conditions these are just the same condition the magnitude is symmetric. However, when psi t is a complex wavelet which is a distinct possibility remember. So, in fact here we are allowing the possibility of complex functions as wavelets we should not disregard or discount that possibility. If psi t is a complex wavelet what we are saying is that we need separately to take care of the positive and the negative part of the spectrum. The other way of saying it is if you are using a complex wavelet and if you insist that the spectrum must be one sided then make sure that your signal has no component on the other side. In that case that particular condition can be removed. So, for example suppose we take a complex wavelet where we are not going to take care of the negative part of the spectrum. So, second condition is not obeyed the one which involves psi cap mod of minus alpha minus beta the whole squared. What we are saying in effect is then you may only deal with such x which have non-zero components and therefore, x must be complex non-zero components on the positive part of the spectrum for omega greater than 0. Conversely if you are only going to satisfy the condition for the negative part of the spectrum meaning for omega less than 0 then make sure that your original signal also does not have any components on the positive part of the spectrum. So, if one is willing to take care of this slightly restricted situation one can use a complex wavelet. In fact there is a good reason to use complex wavelets if you are dealing with complex functions. So, this condition let me put down the condition before you once again this condition that we have just written down here is called the admissibility condition for a wavelet. Admissibility essentially refers to allowing that function to be called a wavelet, allowing that function to bring out a continuous wavelet transfer. In fact admissibility if you ask me is a condition of reconstruction, is a condition required for reconstruction and if we reflect a little on admissibility what we are saying now from a different perspective makes a lot of sense as I will just show in a minute. You see what does admissibility tell you? Let us take the omega equal to or omega greater than 0 condition. You see as I said for real psi this is enough. So, let us focus on real psi for the moment to make matters simple. It says essentially that integral from 0 to infinity psi cap alpha mode squared d alpha by alpha must be finite. So, where can the trouble come here? You see as you notice the trouble can come at the two extremes. Let me for example put before you a condition on psi or psi cap which is troubles a situation in which we shall have trouble. Let us ask the question can psi cap alpha squared look like this? So, let us take from pi to minus pi it is 1 and 0 else. Well it is obvious that it cannot if we try and construct the integral here what would it be? It would be essentially the integral from 0 to pi if you please 1 d alpha by alpha and obviously this integral is divergent. In fact as you can see the integral d alpha by alpha the indefinite integral is essentially of the form log natural of alpha and if we try and make any such substitution this is divergent. So, therefore we see where the trouble comes from. The trouble comes from the region around omega equal to 0. Interestingly one should not have the spectrum giving any significant contribution around the 0 frequency that is what we are saying. So, here the trouble comes as alpha tends to 0. The spectrum should have vanished as alpha tends to 0. Now let us see what would happen if we have alpha tending to plus infinity and we still have a spectrum that remains. What about as I said something like this a spectrum that is like this as omega tends to plus infinity this tends towards a constant. So, psi cap alpha mod squared tends to some constant. Let us say c 0 as alpha tends to plus infinity. Will this do? Well again let us evaluate that part of the integral. So, then you know if you take the integral from a large number let us say l towards plus infinity psi cap alpha mod squared d alpha by alpha and if we note that l is large enough asymptotically I mean. So, this is approximately integral from l to plus infinity c 0 times d alpha by alpha then again we run into trouble. So, this would be essentially log natural of alpha evaluated from l and going towards plus infinity plus infinity which is also divergent once again. This is the cause of divergence. So, again the spectrum should have vanished that is what we have conclude should have vanished as alpha tends to plus infinity. So, all in all what are we asking for? We are asking for the spectrum to vanish as alpha tends to 0. We are asking for the spectrum to vanish as alpha tends to infinity. We want the spectrum to vanish at both the extremes. If it does not if it persists as you go towards 0 or as you go towards infinity you have trouble that part of the integral is going to diverge. Now, let us take a third case which might just be acceptable and see if it is acceptable. So, can we allow the following say capital omega 1 and capital omega 2 0 here infinity there and let us say it is 1 for simplicity in this region. So, capital omega 1 capital omega 2 are both positive numbers and the spectrum mod psi cap omega squared is 1 only between omega 1 and omega 2 and 0 everywhere else. Of course, I am showing only the positive side the same is mirrored on the negative. So, is this allowed? Well, yes indeed as we can see if I were to put down the integral 0 to infinity as it were psi cap alpha mod squared d alpha by alpha it simply boils down to integral from omega 1 to omega 2 1 d alpha by alpha and this is very easy to integrate. This is simply log natural omega 2 minus I am sorry log natural capital omega 2 minus log natural capital omega 1 which is of course finite and acceptable. So, it makes it very clear to us what kind of a function is acceptable. Let us put back the drawing of that function. This is a kind of function that we can accept essentially a band pass function that is what we are saying. A band pass function is what we are willing to accept. In fact, here we have considered the ideal case the extreme case where it is exactly band pass, but even if it is not exactly band pass. Suppose for example, we take the Gaussian function as it were you know we must ask the question we found the so called quote unquote ideal function in the sense of time frequency product the Gaussian. If I take the Gaussian in time it is also Gaussian in frequency that is the beauty of the Gaussian. What I meant was that if I consider the function e raise the power minus t square by 2 and in fact if I also normalize it properly. So, here the variance is 1 I put 1 by square root 2 pi there. Its Fourier transform is also going to be of the form e raise the power minus omega squared and you see a Gaussian creates a Gaussian in the frequency domain. So, the question is is this admissible and the answer is a very simple no. In fact, we can sketch it this is how e raise the power minus omega squared would look and as you can see it does not vanish. In fact, as you go towards 0 it tends towards 1. So, it is very easy to see that if I try and take the modulus squared of this and start integrating with respect to omega even over a small range between say 0 and 1 it is going to diverge. So, the answer is no the Gaussian by itself is not admissible. So, what is the next best thing that we might explore make the Gaussian admissible by pulling the spectrum to 0 as you go towards 0 frequency. Now, how on earth do you pull the spectrum to 0? Well the one way to do it is to take a derivative. So, suppose we were to take the derivative of a Gaussian that is of the form minus 2 omega times e raise the power minus omega squared which of course, you could just essentially take for consideration as omega times e raise the power minus omega squared. We will just consider this. So, how does this derivative first derivative of the Gaussian look? The first derivative of the Gaussian is going to be a product see I am only going to show the positive side of capital omega. So, this was the Gaussian and this is what capital omega would look like when you multiply them. When you multiply them you can see that there is going to be a maximum somewhere and then there is going to be a tapering off again because the Gaussian fall is much stronger than this linear rise. So, their product is ultimately going to be dominated by the Gaussian fall. So, we are going to have a spectrum that looks something like this and we might ask that since this seems to prima facie satisfy both our requirements namely that it vanishes as omega tends to 0 and vanishes as omega tends to infinity, is it admissible? Well not at all difficult to answer. Indeed if I put down the admissibility integral I would have omega squared here e raise the power minus 2 omega squared there d omega by omega. And this is a very easy integral to evaluate. Now, omega t omega is half of t omega squared. What I mean is if you put capital omega squared is lambda then d lambda is twice twice omega d omega. And therefore, you have this integral boiling down to integral 0 to infinity e raise the power minus twice lambda half d lambda and this is a very easy integral to evaluate. In fact, it definitely converges. So, therefore, the derivative of the Gaussian is admissible good news bad news. The derivative of the Gaussian is no longer optimal in the sense of time frequency. So, I in fact now put down a couple of exercises. One, calculate at least approximately numerically or approximately the time bandwidth product I shall abbreviate time bandwidth product by T b p that is the sigma t squared sigma omega squared of the derivative of the Gaussian. So, e raise the power minus omega squared in frequency, but you will see in time also it has a similar. So, in fact, let me put down the time expression. You see when you take the derivative in frequency we took the derivative in frequency. Now, taking the derivative in frequency is equivalent to multiplying in time by t in time. Of course, there are constants involved constant of J and so on, but if one does all that bookkeeping properly one would see that the time and frequency expression is very similar. So, the inverse Fourier transform of omega e raise the power omega squared has the same form and I leave this to you to show as an exercise. So, we have the first exercise here. Take this first derivative of the Gaussian, look at its Fourier transform, make an attempt to evaluate its time frequency product time bandwidth product approximately if required or numerically if required and compare it with the time frequency product of the Gaussian. There would be a disappointment anyway that was the first exercise. The second exercise is essentially the following take higher derivatives. So, for example, consider the second derivative of the Gaussian and repeat whatever we have done here that is finding out the function itself its derivative then checking for admissibility getting a feel of the time bandwidth product for the second derivative of the Gaussian. Incidentally, the second derivative of the Gaussian is a celebrated function in the context of wavelets and multirate processes. Some people call it the Mexican hat function. So, you know I can give you a feel, if you look at it let us look at the first derivative first. This is what the first derivative looks like. We can get a feel a graphical feel of the second derivative. There would first be a region where that derivative of this derivative is positive this region. There would be a place where it crosses 0 somewhere here there will be a maximum there will be a 0 and then the derivative of this is going to be negative. So, you are going to have a region of positivity and then a large region of negativity and even in that region of negativity you can see very easily that finally that second derivative would also tend towards 0 here. Now, one can also see that this positive segment is large that derivative is large here and the derivative falls off slowly here but does fall towards 0. Anyway, so this looks very much like a hat. So, I leave it to you to construct the derivative of this as an exercise and to verify that it looks like a hat. So, if you take the derivative of this would look like a hat like a Mexican hat if you might want to call it that and in fact it is called the Mexican hat function just for that reason. So, I leave it to you to calculate the admissibility integral for the Mexican hat function whether approximately or numerically or exactly also to calculate its time bandwidth product whether numerically or approximately or exactly. With that little discussion on the admissibility of the Gaussian and the derivatives of the Gaussian let us see a little more about admissibility here. You see what admissibility essentially requires as we can see in general is that we need to have a sharp enough fall off as you go towards omega equal to 0 and a sharp enough fall off as you go towards omega equal to infinity. So, we might summarize all this by saying admissibility is essentially a band pass requirement. We want the function to have a band pass character and this agrees very well with what we have seen so far whether we take the Haar wavelet or whether we take the subsequent the Warsh wavelets or whether we also take the idealized situation that we considered when we brought out the ideal towards which we were moving in the Haar situation or in general. Now, we must remember that here we have still allowed the scale parameter to be continuous. So, admissibility is adequate when you are talking about reconstructing from a continuous wavelet transform, but that is the most difficult thing to do. How on earth do you construct this continuum of scale and translation coordinates? Numerically or practically it is the most silly thing to do. You have one dimensional function x t and you are trying to construct a two dimensional continuous pair of parameters tau n s. So, the natural question to ask is can we discretize. In fact, we have begun with discretizing it. We have discretized the scale parameter in powers of 2 when we looked at the Haar multi resolution analysis or for that matter any of the dyadic multi resolution analysis like for example, the Dobash series of MRAs. So, the first question that we must now ask is what are the conditions when we can discretize the scale parameter and that is the next question that we shall address. And to answer this question, let us go back to the basic idea that we had when we built the continuous wavelet transform and we tried to reconstruct. You see when we built the continuous wavelet transform, we said that essentially in the continuous wavelet transform what we are doing is to take the function and put it through a filter with frequency response of the form psi cap s omega complex conjugate here. Of course, there are constants involved here that is not the important thing. The important thing is that what you get out here indexed by tau. So, suppose you call the output variable here tau then this gives you the C w t evaluated as a function of tau at s at scale parameter s. You see if I were to take an ideal band pass function here, what are we asking for? You see with an ideal band pass function say between omega 1 and omega 2 as we did last time what happens suppose this is psi cap omega just omega without the s in magnitude. How would the magnitude of psi cap s omegas appear? So, psi cap s omega you see for example, to fix our ideas let us take s equal to 2 psi cap 2 omega would essentially go from omega 1 by 2 to omega 2 by 2. So, in general we would go from omega 1 by s to omega 2 by s here all the same omega 1 and omega 2 are positive and therefore, these two will still continue to remain positive albeit may be lesser or more depending on the value of s then omega 1 and omega 2 respectively. But, they will still be in the positive side of the spectrum away from 0. So, essentially what is going to happen with a change of s is to move this band of the band pass filter along the positive part of the spectrum and the natural condition that we should expect for being able to discretize the scale parameter is to ensure that we are covering the whole spectrum and there again we have a natural choice of how we discretize. You see when we scale by a factor of s we are also scaling as we see the center frequency and the band. So, there is a logarithmic change. So, the natural kind of discretization to consider for the scale parameter is a logarithmic discretization. In fact, we been doing this all this while we been considering a discretization in powers of 2, but now we do not need to restrict ourselves to powers of 2. We could in general allow s to take the form a 0 to the power k where k runs over all integers and a 0 is a quantity greater than 1. For example, a 0 could be 2 where we have considered the dyadic case a 0 could be 3, 1.5 anything more than 1. You see the more than 1 is not a problem even if a 0 is taken to be less than 1, but positive. For example, 0.5 when you run k over all the integers any way you are considering an equivalent a 0 which is more than 1. What I am saying is if you insist on taking a 0 equal to half and then run k over all the integers it is the same as taking a 0 equal to 2 and running over all the integers. So, you might as well restrict a 0 to be greater than 1. So, now we shall do that we will restrict a 0 to be greater than 1 and we will consider all the a 0 to the power k with k integer and in the next lecture we shall see when a discretization of S in that form is acceptable for reconstruction. Thank you.