 a warm welcome to the third lecture on the subject of wavelets and multirate digital signal processing. Let us spend a minute on what we talked about in the second lecture. We had introduced the idea of a wavelet in the second lecture and we had done so by using the Haar wavelet. Essentially, where piecewise constant approximations are refined in steps by factors of 2 at a time. In today's lecture, we intend to build further on the idea of the Haar wavelet by introducing what is called a multi resolution analysis or an MRA as it is often referred to in brief. So, let me title today's lecture. We shall title today's lecture as the Haar multi resolution analysis. In fact, let me also put down here the abbreviation for multi resolution analysis MRA. You see, the whole idea of multi resolution analysis has been briefly introduced in the context of piecewise constant approximation. So, recall that we said that the whole idea of the wavelet is to capture incremental information. Piecewise constant approximation inherently brings in the idea of representation at a certain resolution. We took the idea of representing an image at different resolutions. In fact, we use the term resolution when we represent images on a computer 512 cross 512 is a resolution lower than 1024 cross 1024. And one way to understand the notion of wavelets or to understand the notion of incremental information is to ask if I take the same picture the same two dimensional scene or same two dimensional object. So, to speak and represented first at a resolution of 512 cross 512 and then at a resolution of 1024 cross 1024 what is it that I am additionally putting in to get that greater resolution of 1024 cross 1024 which is not there in 512 cross 512. The Haar wavelet captures this. So, in some sense you may want to think of the Haar wavelet as being able to capture the additional information in the higher resolution. And therefore, if you think of an object with many shells this is a very common analogy. You know if you think of the maximum information may be as a cabbage or as an onion informally. And if you visualize the shells of this cabbage or this onion like this then the job of the wavelet is to take out a particular shell. So, the wavelet at the highest resolution wavelet translates at highest resolution at max resolution would essentially take out this at next resolution it would take out this shell and so on. So, when you reduce the resolution what you are doing is to peel off shell by shell. In fact I think this idea is so important that we should write it down. We are essentially peeling off shell by shell using different dilates and translates of the Haar wavelet. And there again a little more detail different dilates correspond to different resolutions and different translates essentially take you along a given resolution that is the relation between peeling off shells and dilates and translates. Now all this is an informal way of expressing this we need to formalize it and that is exactly what we intend to do in the lecture today. Again we would now like to talk in terms of linear spaces. So, without any loss of generality let us begin with a unit length for piece wise constant approximation. I say without loss of generality because after all what you consider as unit length is entirely your choice. You can call 1 meter unit length you can call 1 centimeter unit length or if you are talking about time you could talk about 1 second as unit length or unit piece and so on. So, unit on the independent variable is our choice and in that sense without any loss of generality let us start make the focal point piece wise constant approximation at a resolution with unit interval. So, let us write that down formally piece wise constant. So, you know the so called fulcrum or focal point is piece wise constant approximation on unit interval and let us sketch this to explain it better. So, what we are saying is you have this independent variable again without any loss of generality let that independent variable be t you have unit intervals on this and on each of these unit intervals you write down a piece wise constant function essentially corresponding to the average of the original function on that interval. So, this is the average of the function on this interval this one on this interval and this one on this interval. Now, how can we express this function mathematically with a single function and its translates. So, essentially we want a function let us call it phi of t now. So, what function phi of t is such that its integer translates can span the space what space first the space of piece wise constant functions on the standard interval standard unit intervals what are these standard unit intervals the standard unit intervals are the open intervals n n plus 1 for all n over the set of integers. Now, I wish to slowly start using notation which is convenient. So, this notation script z would in general in future refer to the set of integers. So, I think we should make a note of this script z is the set of integers and this refers to for all. So, what are we saying here let us go back we are saying we have this space now again I must I must I must I must I recapitulate the meaning of space linear space a linear space of functions is a collection of functions any linear combination of which comes back into the same space. So, if I add two functions it goes back into the same space if I multiply a function by a constant it goes back into the same space. If I multiply two different functions in that space by different constants and add up these resultants it would still be in the same space in general we would say any linear combination. So, we say the set a set of functions forms a space a linear space if it is closed under linear combinations. So, we say a linear space of functions is a set of functions such that linear combinations fall in the same set. Now, I am making it a point to write down certain definitions and derivations in this course and there is an objective behind that I believe that a course like this is best learnt by working with the instructor. So, although one could just listen and try and remember that does not give the best flavour in a course like this it does require in depth reflection and thinking and therefore, I do believe that the student of this course would do well to actually note down certain things and work with the instructor for it is then that the full feel of derivations and the full feel of concepts would dawn upon the student. Anyway with that little observation and instruction let us go back to what we are doing here. So, you see a linear space of functions is one in which any linear combination of functions in that set fall back into the same set. Now, here there is a little bit of clarification required you see in general if you consider the space of functions that we talked about a minute ago namely the space of piece wise constant functions on the standard unit intervals which are the standard unit intervals the intervals of the form open intervals of the form n n plus 1 for all n over the set of integers then there is an infinity of such functions and naturally when you talk about linear combinations you could have finite linear combinations and you could have infinite linear combinations. Now, for this point in time when we talk about linear combinations we are essentially referring to finite linear combinations. So, that is just a little clarification for the moment well the idea could be extended to infinite linear combinations too, but I do not want to go into those nice cities at this point in time they would carry us away from our primary objective. So, therefore, this set of functions that we talked about a minute ago is indeed a linear space that is why I have called it a space here and we will give that space a name. So, we will call that space v 0. So, v 0 is a set now I am going to write mathematical notation v 0 is a set of all x of t such that two things happen x of t now you know x is a function. So, when I write like this what I mean is I am suppressing the explicit value of the independent variable, but I recognize there is an independent variable here, but I am treating the whole thing as an object it is a function I am treating the whole function as an object and this object belongs to L 2 R we call that L 2 R is the space of all functions which are square integrable and this stands for belongs to. So, such that x belongs to L 2 R and x is piecewise constant on all intervals of the form n n plus 1 and n plus 1 is the n integer once we have talked about v 0. In fact, the reason for giving the subscript 0 here is that we are talking about 2 to the power of 0 as the size of the interval that is important enough I think to make a noting. So, we say v 0 because of piecewise constancy on intervals of size 2 raised to the 0 which is 1 and similarly therefore, in fact you know you could call it 2 raised to the power of 0 or you could call it 2 raised to the power of minus 0 we will prefer to use 2 raised to the power minus 0 because it will be consistent in future. So, we could similarly have v 1 then similarly v 1 is the set of all x let us define it the set of all x x belongs to L 2 R and x is piecewise constant on standard 2 raised to the power minus 1 integer that is intervals of the form n into 2 raised to the minus 1 n plus 1 into 2 raised to the minus 1 for all n integer and in general we have v n the set of all x t for completeness we should write down the definition properly and x is piecewise constant on all open intervals of the form simple enough to fix our ideas let us sketch a couple of examples. So, let us take an example of x belonging to v 2 it would look something like this. So, you would have intervals of one fourth here and in fact to be complete we should also include intervals before 0 and so on and we have piecewise constancy on these and so on there and please remember x is also in L 2 R. So, when you say it is in v 2 it is automatically of course, in L 2 R and that means that if I take the sum squared of all these constants that sum squared is going to be finite that is an important observation. The constants that we assign here must be such that when we sum the square of all of them magnitude squared of all of them that sum must converge this observation is so important that I think we should make a note of it. So, we are saying the sum squared the absolute squared sum of the piecewise constant values in all these v m must be convergent and this follows from belonging to L 2 R. Let us also take an example of a function belonging to v minus 1. So, minus 1 means intervals of size of 2 2 raise to the power minus of minus 1. So, intervals of size 2 and so on there and so on there and we have piecewise constants there and so on here and so on there. So, now we get our ideas fixed what we mean by the spaces v m. Now, the moment we put down these spaces with these examples. So, clearly we see a containment relationship. So, there is a relation between these spaces they are not arbitrary they are not just totally disjoint and unrelated. In fact, you can notice that if a function belongs to v 0 for example, which means that it is piecewise constant on the standard unit intervals it is also going to be piecewise constant on the standard half intervals. And for that matter if a function belongs to v 1 which means that it is piecewise constant on the standard half intervals it is automatically going to be piecewise constant on the standard one-fourth intervals. To exemplify this let me go back to this example of x belonging to v minus 1 that I have here. Notice that this function is piecewise constant on the standard intervals of size 2. So, obviously if you take the standard intervals of size 1 for example, 0 to 1 1 to 2 2 to 3 3 to 4 minus 2 to minus 1 minus 1 to 0 and so on the function is still piecewise constant. So, therefore, a function that belongs to v minus 1 automatically belongs to v 0 a function that belongs to v 0 automatically belongs to v 1 and therefore, there is a ladder of subspaces that is implied here. What is that ladder? The space v 0 is contained in v 1 the space v 1 is contained in v 2 and so on this way and of course, the space v minus 1 is contained in v 0 the space v minus 2 in v minus 1 and so on. And we expect intuitively as we move in this direction we should be going towards L to R. Of course, it is an important question what happens when we go in this direction that is interesting we will spend a minute now and reflect on that. So, you see what happens when we go left towards what I mean by that is of course, you have v 0 contained in v 1 contained in v 2 and then v minus 1 contained I mean contained in v 0 yes and so on here. So, on there what happens when we go this way? We think should happen what are we doing? We are taking piecewise constant functions on larger and larger intervals let us write that down. So, piecewise constant functions on larger intervals now you see what is the L 2 norm of functions as you go left towards what kind of a form will it have? It is going to have a form like this summation on n now you see remember the L 2 norm is the integral of the absolute squared of the function. And please remember the function is piecewise constant. So, you have one constant let us call it c n on the nth interval and the interval is of size 2 raise the power minus m. So, this is essentially you know you are talking about integrating mod c n squared it is a constant over an interval of 2 raise the power minus m and please remember m is negative and m goes towards minus infinity as you go left once. So, that is the same thing as 2 raise now you see 2 raise the power minus m is 2 raise the power of mod m in the context of negative m and summation on m mod c n squared now you see these subtleties point is that if this needs to be finite irrespective of how large m is we have no control on this except that this part must be finite. But then when we say finite if it is non-zero and if we allow m to grow without bound this is going to diverge. So, the only way in which this can converge no matter how large I mean large in the sense large in magnitude how large in magnitude m is no matter how large in magnitude m is if this is to converge then this must be 0 a very important conclusion. So, you are saying that if 2 raise the power mod m summation n mod c n squared must converge no matter how large in magnitude m is if or how negative then we must have summation over n c n squared tending to 0. So, essentially what we are saying is as we move left towards we are going towards the 0 function a point that takes a minute to understand, but it is not so difficult as you can see. So, now we have very clearly an idea of our destination as we move up this ladder towards plus infinity and as we move down the ladder towards minus infinity and we can formalize that. What we are saying is moving upwards now you know one has to use proper notation we would have been tempted to say something like limit as m tends to infinity or plus infinity or something like that, but you see it is not really correct to talk about limits of sets. So, we need to use the notation that is appropriate in the context of sets namely union. So, when we take a union of two sets and if one set is contained in the other we are automatically taking the larger set. So, moving upwards is attained by using union in other words we are saying the union of v m m over all the integers should almost be l 2 r now that is where the little catches I mean we would have been happy to write is equal to l 2 r, but you know we need to make a little detail here. We need to put something called a closure I will explain what I mean by a closure you know suppose you want to visualize l 2 r to be like an object with a boundary. So, suppose this where l 2 r just notionally and this is the boundary of l 2 r so to speak. So, it is a space you know now what we are saying is as we go in union that is union m over all integers of v m over all integers of v m it would cover all the inside covers all the interior, but then it might leave out some peripheral things on the boundary. So, it may also cover some part of the boundary now of course, do not ask me at this stage what we mean by boundary and interior save that you know you are talking about situations you know boundary now informally when you say boundary you are talking about functions where moving in a certain direction does not remain l 2 r moving in the other one does. So, you know it is see this boundary and the interior at the moment needs to be understood only informally. But what we are saying is as far as this union goes it can take you almost all over l 2 r it covers all the interior it may also cover quite a sizeable part of the boundary, but it might leave some patches of the boundary untouched and therefore, when we do a closure we are covering up those patches what we just did was covering up those patches. So, closure means cover up boundary patches. Now, this is a small detail and we need not spend too much of time in reflecting about this idea of closure and so on, but to be mathematically accurate we do need to note that it is after closure that the union over all m integer of v m becomes l 2 r otherwise it is almost l 2 r which means that when you take this union that is when you make piecewise constant approximations on smaller and smaller and smaller intervals you can go as close as you desire to a function in l 2 r. So, you can reduce the l 2 norm of the function to 0. So, if you look at it that is what we mean by that is what mean that is what is implied by boundary you know you can you can go as close as you like to a certain function you can make the l 2 norm 0, but still it would not quite reach there. So, you know you could just visualize that you might just be a teeny weeny bit inside that boundary, but not quite on the boundary and how teeny weeny as small as you like that that is where the union takes you that is the subtle idea of closure. Anyway as I said we do not need to spend too much of time in talking about this closure, but we should be aware of this idea because when we read literature on wavelets of that matter when we really wish to put down the axioms of multi resolution analysis properly we must be aware that this closure is required. So, much so anyway now let us take the second of our of our inferences here moving downwards so to speak. So, how would we move downwards just as union takes you upwards intersection takes you downwards. So, if you take an intersection on all m belonging to z of V m there I do not need to worry about closure or anything of that kind. I can simply put down this is a trivial subspace essentially the subspace of l 2 r with only the 0 function included this is called a trivial subspace. Now again I must make an observation here to clarify the trivial subspace is not the same as the null subspace the trivial subspace has only the trivial 0 element in it the null subspace does not have any element. So, that is the subtle distinction and we must bear in mind that we are talking about the trivial subspace. Yesterday I told you that there is this beautiful idea about just one function psi t it is dilates and translates going all over to capture incremental information. Now we need to state that formula 2, but in order to move in that direction we first need to bring in as I said another function which will span v 0. So, we need to bring in this idea of spanning we see a set of functions. Let us say f 1 to let us say f k and so on span a linear space if any function in that linear space can be generated by the linear combinations of of the set. Now again there is this subtle distinction between finite linear combinations and infinite linear combinations I do not wish to dwell on those distinctions at the moment, but what we are saying what we mean by span. So, when we talk about the span of a set of functions we are talking about all linear combinations of those functions and therefore, the set or the space of in fact the space of functions generated by all linear combinations of that set. Now we ask a question which should make our life easy what function we ask this question before, but now we answer it what function suppose we call it phi t and its integer translates span v 0 and the answer is very easy. In fact, if you were to visualize a function which is 1 in the interval from 0 to 1 and 0 else low and behold you have the answer. So, what we are saying is any function in v 0 can be written like this summation n over all the integers c n phi t minus n. So, essentially integer translates of phi and these are the piecewise constants here just to fix our ideas let us take an example here. So, what we are saying is for example, suppose I have this example of a function in v 0. So, I have 0 1 2 and 3 and so on the value here is let us say 0.7 the value here is 1.5 the value here is 0.7 the value here is between 2 and 3 is 1.3 the value between minus 1 and 0 let us say is 0.2 and so on then and this could continue then this function can be written as well dot dot dot plus 0.2 times phi t plus 1 plus 0.7 times phi t plus 1.5 times phi t minus 1 plus 1.3 times phi t minus 2 and plus dot dot dot and so on simple enough not at all difficult to understand. So, we have the single function phi t whose integer translates span v 0. Now, the subtle point is that if you were to go to any space v m. So, the same thing would carry forth. So, it is very easy to see that any space v m can be similarly constructed. In fact, we can be more precise. We can write down v m is essentially the span over all n belonging to z of phi 2 raise the power of m t minus n. So, just as we looked at the wavelet yesterday and said it is this single function which can allow details to be captured. We now have this function phi t which captures representation at a resolution. It is a very powerful idea if we think about it. If you want to capture information at a resolution at a certain level of detail that is all the information up to that resolution you have the function phi. If you wish to capture the additional information in going from one subspace to the next you have the function psi the wavelet. Now, we need to give this function phi t a name. We shall call it a scaling function. Now, of course, here the phi t that I have drawn in this context is the scaling function of the Haar wavelet or the Haar multi resolution analysis. Now, what is this multi resolution analysis I have suddenly brought in this work. So, what is this? Well, this ladder of subspaces that we are talking about here with these properties is called a multi resolution analysis or an MRA for brief. Of course, in this case the Haar multi resolution analysis. Now, what properties we need to put them down formally once again? We have introduced two of them and one more subtly, but we now need to put down axioms very clear. So, let us put down what are called the axioms of a multi resolution analysis. So, the first axiom is of course, there is a ladder of subspaces of L 2 R and we know what that ladder looks like. Axiom number one union over all integers closed V m is equal to L 2 R. Intersection over all integers V m is the trivial subspace with only the 0 element. These are not all further. There exists a phi t such that V 0 is the span over all integer n of phi t minus n 0.4. In fact, you know it is not just span, there is something more. These phi t minus n over all n is an orthogonal set. This is a deeper issue here. Now, we will explain in more detail the notion of orthogonality in the next lecture, but for the moment let us be content to put this down as an axiom. Next, if f t belongs to V m, then f 2 raise the power of m t or 2 raise the power minus m t belongs to V 0. So, for example, if f t belongs to V 1, then f t by 2 or f 2 raise the power minus 1 t belongs to V 0 for all m belonging to z. And if f t belongs to V m, then f t minus n also belongs to V 0 for all integer n. So, these are the axioms of a multi-resolution analysis. That means, this is what constitutes a multi-resolution analysis. And here we have taken the hard multi-resolution analysis to build the idea up, but the whole abstraction is that we can have several different files and then to end this lecture the corresponding size. So, where does the psi come in? It comes in what is called the theorem of multi-resolution analysis. Given these axioms, there exists a psi belonging to L 2 R, so that psi 2 raise the power of m t minus n for all integer m and all integer n span L 2 R. This is a very very significant idea. In other words, this is exactly what we said yesterday. Take dyadic and translates of the function psi and you can cover all functions arbitrarily close to any function in L 2 R as you desire. We have built this idea up from the hard example, but in the next lecture we shall try and build a little more abstraction into what we have done and proceed further from there. Thank you.