 Okay, so, well, time frequency, localizing things in time and in frequency is something that we've all seen many times before, as you see in these slides. Why is it, might it even be surprising that you would want to localize in time and in frequency at the same time, since we know how to do it? Because when people, technical people think of signals that depend on time. So we, I'm going to talk a lot about functions that depend on time. And here t generally varies either over an interval or all of r. When you think of frequency, we, our automatic reflex is to think Fourier analysis. And then you think of, of defining the Fourier transform. If you are in an interval, you do Fourier series. But if t is in r, then you define the Fourier transform. And, which I like to normalize as, as follows. You have to put your two pi somewhere, and this is where I put them. So, we think, and for instance in quantum mechanics, where typically the variable is three dimensional, and you can do a three dimensional Fourier transform. You think of, of time and frequency as two sides of one coin. You're either in time or you look at the Fourier transform in your frequency. In quantum mechanics, you would think of it as position and momentum. But you do want in many applications, like these, the sheet music, to have information about what happens in frequency but locally in time. I mean Fourier, sheet music tells a musician which notes to play when. I mean, so notes has frequency information and when. And I'm going to talk about many applications from different walks of life in which you do want to indeed have this frequency information but locally in time. And for that, you need to do something beyond just the Fourier transform because you want to keep these two aspects in there. And, okay, I have, yes. So, here actually are two examples, I mean, of a time frequency representation that we will look at. Both of them are spoken word signals. In the bottom part, so the color things here are, oops, yes. This color thing is a spectrogram and we'll come back to that. Here you have another spectrogram. And at the bottom part, we have just a very short bit of speech and you actually see the time signal here. Here the signal is already a full second long and so if you were to plot this, it would become much, much more dense. And you see, we have these places where you have something happening in time and time is horizontal here, frequency vertical. And at some places you have a wide frequency range, it's a very narrow one. Let's look at another example. Here again, I mean, here you actually see how dense the signal, how fast it oscillates for a speech signal. Here actually we have a music instrument, so we get closer to sheet music. And in fact, there are even applications where people have looked at the music and recognized the score in the representation in the spectrogram and you can find online software that will try to annotate music from a spectrogram. So just this as an introduction to situate this idea of time-frequency localization and oops, I'll try to not work too much with the microphone, not to disturb the microphone too much. So what we are going to do in order to, that was the extent of the slides, so we could extinguish the projector if somebody is up there who can perform such magic. What we want to do is, well, it's very simple, if I have my function in time, then I can try to localize it first by just cutting out a piece of it. So my function in time is something that varies, I mean, and oscillates much faster than that, as you've seen in the real signals, but I can't draw like that. What we're going to do is we're going to, we could cut little pieces from it and then for this short piece, computer Fourier series and then we could look at the next piece and for this computer Fourier series again. And since we ultimately are looking at little pieces, we will have kind of localized the function and then looked at. So what that amounts to is that we are taking our function, we are going to multiply it by some window, I mean let's take this oh well let's take it, let's use a different one and this long interval tau t, and we're going to move this at different locations, so this will tell us where we put ourselves in time, I mean here or there or there and so on. And then we'll compute a Fourier series of each of these, put an n here, okay, so by now I got it more or less, right? And so, you don't put two pi, we would have two pi for the, yeah I am going to put it in, I was wondering should I do it? And then, so these are Fourier coefficients, so let's call them, so this is a transform of my function and I have coefficients m, n, okay. So that will give me indeed a family depending on two indices, I could also if I wanted to do it continuously, I mean and I'll do both of these during the lectures that we will see, I could also define a, well let's first do it this way, discreetly. If I do that, I mean I immediately am going to amend this, if I do that then even though my function typically is nice and continuous, all the ones you've seen, even though it's speech that starts and stops and so on, the time signal in fact is nice and continuous, I have introduced discontinuities, I mean, well I've introduced discontinuities because I, as far as the Fourier series are concerned, when you truncate like this, you are doing the same as expanding a function that repeats periodically and because the truncation here typically is not at the same value, the function has not obtained the same value as it had there, you have implicitly introduced the discontinuity there and that is something for which you pay a price, discontinuous Fourier transforms of discontinuous functions decay very slowly in, so you have very slow decay in n, which you have introduced with your tool. In fact a whole lot of what we will see, I mean even things in the spectrogram, I should have pointed it out when we saw the figure, they are features that you introduce in the signal because of the tool that you use to analyze it and you want to do as little as possible of that type, that is emblematic of your tool rather than the signal you want to analyze. The thing that we had in the spectrogram and that I forgot to point out is that we had, often we had repeats in speech of actually we had something not very bent here and then a little bit more bent there and then even more bent there, which people have even given names, I mean they're the different harmonics of, but again there's something that we have put in with our analysis tool, it's not something in the signal, the signal is not something that consists of many harmonics, these harmonics we put in there by looking at it a certain way and I'll come back to that in probably next week, maybe the week after. So we want to not do that, well one way in which we can avoid having these discontinuities is by not cutting things very abruptly and so by making a window softer and so that's the very first thing I'm going to do, I'm going to remove this abrupting and I'm going to put a window function here and in fact now my window is going to be, I'm still going to translate it by a certain tau and I, but my window is now wider and so you see we start overlapping because well if I made my window soft and didn't overlap then it's pretty obvious that I wouldn't have much information in my transform on where the windows touching are very small and so if I still thought of it in terms of Fourier series then I could say well I have localized here and I would do Fourier series with respect to this wide interval and then for the next one that I've localized here would do Fourier series with respect to that interval and so on and you would then see that I pay for this overlap in the fact that my time and frequency localization are not as tightly wound together we'll come back to that that's one reason why I put in two parameters here this tau and this omega if to go back to the window to Fourier transform with with with a sharp cutoff if I had if I do my sharp cutoff like I did had done earlier then on an interval with length tau I would have my the standard Fourier transform would be to just take f of t from 0 to tau e to the i n t I would put 2 pi over tau and so that would correspond to omega equals to 2 pi over tau dt yeah I sometimes forget the minus sign and so on because it doesn't matter much but thank you for me making me consistent I mean okay so and and I would have a normalization factor in order to well let's think of the normalization factor one over tau if I call these c n then I would have that the the integral from 0 to tau of f t squared dt would be exactly equal to the sum over the c n squared no it wouldn't be what I seem to yes and so I would have found a nice orthonormal basis and in fact in in order to make an orthonormal basis out of construction like this I can choose I mean I've done that here by choosing this this strict window and taking omega 2 pi over t a tau I can do that for other windows I mean there exist other functions w for which I still will have an orthonormal basis but you always need to have this so if you want that the w m n of t which are w of t minus m tau e to the i n omega t form an orthonormal basis then you must have that m that omega times tau equals one independently of independently of w well w is normalized to norm one so integral of w squared dt is one you can only have an orthonormal basis if and but we will be interested in many cases where omega times tau is actually smaller as we need to do when we make our windows after so we already are looking at situations where we make a discrete transform but it's what physicists like to call redundant meaning the coefficients are are or what they also like to call over complete I mean the functions are over complete in the sense that there are linear combinations with nicely decaying coefficients of all the basis function of all the building block functions that sum to zero okay so I've told you that we want to look at that I mean orthonormal basis of course are very very nice because you have so this would be an orthonormal basis for l 2 of r and I have this integral this equality on the interval from zero tau and then by moving my window to different m's I would get that my the integral of f of t squared if I take my window to be one over square root tau characteristic function of zero tau t that the integral over all of r would just be the sum over all the mn of the corresponding coefficients mn squared so I would have an orthonormal basis and that's why orthonormal basis are very nice because I can then also if I take the inner product between functions have these would be the cf's and the cg's so all computations that I might want to do are extremely easy um there's no support condition on w yes and in fact if you want an example with with not compact support you could think of I mean there is a symmetry in in my transform in time and frequency I mean if you might be a good a good point to say that so um my the the way I've written my Fourier transform it is unitary and so uh uh I can view so tf of tau and omega at tf mn sorry is the inner product of um uh well I've stuck my 2 pi there I'll I'll have to live with it um it's the inner product of f and 1 over 2 pi w minus m tau e to the uh so let me try let me do it differently uh it's it's um it's a Fourier transform of a function and the function is f w minus m tau and I have Fourier transform that and looked at it in uh the argument and omega but I can also write that integral by writing the integral for the Fourier transform since the Fourier transform is unitary and so I can also write it as 1 over square to pi integral over xi of f hat xi and then I have to Fourier transform the remainder so Fourier transforming wt minus m tau e to the minus i and t if I Fourier transform that e to the minus i xi t dt 1 over square to pi then I will get uh by change of variables you see that uh so I have to add that again you see that I get e to the uh minus i m m uh tau and e to the minus uh so there's no tau uh I had an omega here didn't I yes omega tau omega and then I had e to the minus uh xi m tau the Fourier transform of w hat in uh xi plus an omega and that's it so if as I said omega tau is 2 pi this becomes 2 pi and so this doesn't contribute and so I have the Fourier transform here is w hat of xi plus an omega e to the minus i xi m tau so I get exactly the same kind of expression but where I have f hat I have a window w hat the translation now is by well the n and m's vary over all of z so it doesn't matter that I have here plus n m could you say translation is by minus omega so the role of f is taken by f hat the window is taken by omega hat a w hat uh uh n and m have changed roles so omega has become tau tau has become omega and n and m have changed roles minus n and m and so since we could build an orthonormal basis by taking a rectangular window in time we can build an orthonormal basis by taking a rectangular window here in frequency which in in time would have an infinite support so you don't need finite support and actually there was a Danish group that tried to build a function that that that did the best in both worlds that had some decay in in in in in time and in frequency and infinite support on both sides um however you cannot build a very very nice orthonormal basis that way I mean we would love to have orthonormal basis because we we like uh we always like the solution of easiness I mean well that's been my experience that whenever you can get away with something easy in in in you try to do it computationally it really helps makes for faster algorithms but uh uh so it would love to to to have an orthonormal basis but it turns out you can't make uh uh you can't choose a window that is very soft in time and very soft and that they have good decay in frequency and that will give you an orthonormal basis and that's an argument that was uh uh uh let's propose independently by uh two physicists by yon and law francis law and uh they oh sorry law francis law um and it's actually a pretty argument and and well the various way of see of seeing it but it will also introduce a tool that will be useful for us so um let's let's see so there are no orthonormal basis of windowed Fourier transforms for which w and w hat are both smooth with fast decay okay so to see that well it actually I I'm vague but I mean it's already sufficient for them to to uh decay faster than 1 over t or 1 over xi okay so um how shall we do this first of all I'm looking at situation where omega times tau is 2 pi so let's take tau equals 1 by scaling I can always reduce myself to the case where tau is 1 and omega is 2 pi so I'm looking at functions my w m n of t are just functions that are translated by integers and that are modulated by e to 2 pi n t and I'm going to introduce a transform uh I'm going to show that those can never be if w and w hat uh uh decay uh reasonably I'm going to show that these cannot form an orthonormal basis by reasoning via a transform a unitary transform that uh has been proposed by many people but uh uh in particular by uh a physicist called Joshua Zach and so uh since that the shortest name of all the people who've proposed it I mean let's call it the Zach transform so uh this is a transform that goes from l2 of r to l2 of 0 1 squared so l2 of the unit square it will be unitary and that will do something very nice to our functions which will make it then easy to see that they cannot be uh uh that the original w cannot decay fast in both time and frequency okay so the first thing I do is if I take a function any function uh uh g I define it I define a function of two variables s and t as the uh I take the sum over all l in z of g s minus l and I put here e to 2 pi i t l so what I'm doing in this transform is I'm um s here I'm in 0 1 s and t are in 0 1 so s in in l equals 0 I carve out a piece between 0 and 1 and I I uh in t I in a certain sense put it at at at the l equals 0 so in in in t I am thinking of Fourier series and I'm stacking different pieces of g in different layers so that's what this transform is doing uh I claim I mean and we'll now see that this is a unitary transform already you should note I I will think of these functions here in two ways I will think of this I must function at living on 0 1 and that is where I have it unitary but I also will think of once I have this form of the functions of of them as just like you do with Fourier series you'll think of it as an interval but you can also view it as the periodic extension of that function of which you are then looking at properties I can also look at an extension here and let me do that already now it's clear that if you take the Zakt transform of a function and you were to choose to plug in in this formula that I have here t plus 1 instead of t so I'm trying to periodize it in t then that of course doesn't make any change because l is a ditcher and so I can also view these functions as being completely periodic in t in in l it's a bit more in in s it's a bit more complicated because if I put an n and a plus 1 in t the natural thing if we look at our formula is we have here g s plus 1 minus l e to the 2 pi itl the natural thing is to absorb so if I if I make this minus k so k is l minus 1 then this becomes here k plus 1 and so I get that the natural if I want to do this extension then in order to be nice and consistent I have to introduce this phase factor either the 2 pi it so because of this funny way of stacking things I this is if you extend the function over the whole the whole plane how you should do it okay so I have a third blackboard yes a pole in between the two things there is in that a pole oh my I'm going to become this is so much fun okay ooh by the end of the lectures I will be proficient I will I should think of the best permutation to use those boards okay so we'll continue here which is not a continuation of there but anyway so um okay if I so let's now look at uzi of uh before I am going to look at uzi acting on these w's I'm going to look at uzi on my very special on on this very special family of functions which was the k the d interval 01 t minus m e to the 2 pi i and t and I've argued to you before that that family of functions is an orthonormal functions in an orthonormal basis in o2 of r because I'm just doing four-year series on intervals of length one one after the other so this let me call these functions emn of t and they form an orthonormal basis of l2 of r let's look at what happens to those when I apply uzi to them uh so uzi emn so I get a sum over l in z and I uh take the function in argument s minus l and then I have to multiply that by e to the 2 pi i t l um my s is in between zero and one if my s is between zero and one then the only contribution that I uh here I have zero one the only uh l for which this lies between zero and one is uh m equals to minus l so that means that my whole sum reduces to just the term l equals to minus m and I get uh a one here e to the minus 2 pi i t l is equals to minus m so here then I have e to the 2 pi i ns and of course the ln here doesn't contribute so this on the other hand we easily recognize that this is an orthonormal basis for l2 of the square if s and t vary between zero and one we have an orthonormal basis here so uzi maps this orthonormal basis to an orthonormal basis of the square so uzi is unitary so this is an orthonormal basis of l2 of zero one okay uh uzi is a unitary map fine we can see the same uh um but the computation we've done here will help us also immediately to see what happens to the w m n if I do a w here then all I have here is to write w s minus m minus l and I can compute that don't you hate it when you take notes when people do this to you okay so I absorb what I'm going to do is to make a change of variables of course I mean so first I can forget the l here because that's already true the e to the 2 pi i ns I can take outside and then here I'll call this so I have m plus l I'll absorb an m in here so in order to make that easy I write it here and then by changing the summation variable I get an e to the minus 2 pi i t m here I just have the uzi transform of w itself okay so I have that uh my so I started with the w m n and I wanted to be able to be an orthonormal basis that means that the uz w m ns should be an orthonormal basis for l2 of zero one squared which is the same as uh a space of functions that have this weird periodicity and for which I compute the norm by just integrating on the square so the uh kind of twisted doubly periodic functions on r2 okay these are just the multiplication by e to the 2 pi i ns e to the minus 2 pi i t n of let's call that a function w s t the zh transform of the function itself now for this to be an orthonormal basis on l2 of zero one squared there's a simple easy necessary sufficient condition namely that this has absolute value one so this is means that w s t is one for s and t in zero one but that means also that uh since since the the twisted doubly periodic extension doesn't change the magnitude I mean it only gives you phase factors that it has to be that for all s and t in r so why is it supposed to be evident oh because uh uh what I want is uh uh f uz w m n sum over this square should be equal to the integral of f t s t squared over the square but this is the sum over m and n of the integral over the square of f s t and then w s t conjugate e to the minus 2 pi i n s e to the 2 pi i t n d s d t squared but since these are an orthonormal basis on l2 of the square I get that this is the uh so I have a sum of the expansion with respect to not normal basis so I get that this is f s t squared times w s t squared the s d t and for this to be equal to the integral of f squared I will need that this is equal to one almost everywhere uh because I am going to assume the formula I gave here for the transform okay strictly speaking this formula I can only define it for functions g that have good decay I mean you are very kind in not pushing my nose into that when I wrote it but uh so let's do it for only those functions that have good decay then I once I had proven it for an orthonormal basis that I mapped an orthonormal basis an orthonormal basis I have a unitary map on a dense set and so I can extend to the full space okay if my w's if my w function has decay faster than 1 over t then this series is going to be absolutely convergent and I can just define it if g decays faster than 1 over l or 1 over t then I have a decay faster than 1 over l and it's absolutely convergent if the Fourier transform has a similar property then the function itself will be continuous and you can prove that the Fourier that this exact transform is going to be nice and continuous if you have more decay you will have differentiability but let's go for continuity I'm going to assume so I have to so orthonormal basis gives me that w has to be 1 and I'm going to assume w and w had decay faster than 1 over t or 1 over psi respectively and so I'm going to have that w is continuous and then it turns out that these two are not compatible and that's what we'll do next and for that I'm going to use the fact that I know that w s t plus 1 has to be if I extend the function and that w s plus 1 t has to be w of t times 2 pi i t I'll do it in a very pedestrian way show that this is incompatibility that this is incompatible with w equals 1 and continuity but I could also invoke a nice topological what's really happening is that I'm making a topological argument about but let's do it in a pedestrian way okay so w can be written as t s e to the let's say 2 pi i phi s t where phi s t is real this tells us that phi of s t plus 1 has to be up to an integer phi of s t so there is some integer this tells us that phi of s plus 1 t is equal to phi of s t plus t plus some integer l s t so let's now look at phi of s plus 1 t plus 1 and I can get from here to s t in two possible ways I can write it as phi s plus 1 t plus the k of s t plus 1 uh is that uh s plus 1 t sorry and that gives me phi s t plus t plus k s plus 1 t plus l s t but I can also say that this is phi of s t plus 1 plus then the argument t plus 1 plus l s t plus 1 and that I can then here continue I get phi s t plus s t plus on the k s t plus all the other stuff that I had and so you see that since all these things are equal that I get that k s plus 1 t plus l s t is equal to 1 plus k s t plus l s t plus 1 but since w is continuous phi is continuous phi being continuous means that the k and l have to be continuous since they are integers they're going to be constants it's the only way so I get here k plus l equals 1 plus k plus l which is a contradiction and in fact I didn't even need that w is 1 in fact I mean in my case I already had w is 1 but if I take any other zaktransform of a function so something that has this kind of twisted double periodicity and if I assume so if I take here some f on the zaktransform side and if I assume that double that this is bigger than some constant bigger than 0 and is continuous then I can always look at the function f divided by a capital by its absolute value and that will be continuous because of this condition and I can then repeat the same argument so it is just impossible for a function that is the zaktransform of a nice function so that this zaktransform function is continuous itself to be always non-zero to not have a zero and that's what we've done so there this is the arguments of Balian Law that there exists no function that has reasonable decay, decay faster and 1 over t or 1 over xi and 1 over xi and that will give you an optimal basis and so when I was talking earlier about this this construction that had been done by a Danish group of a window that tried to do the optimal thing what they did is they just skirted that I mean they had decay like 1 over t and 1 over xi but not better I mean because you can't do better um so it turns out that I mean so for a long time that was where things stood with respect to this kind of time frequency representation and trying to make orthonormal basis and it was an excellent motivation to look at to look at at presentations in which we would take a window function translate and and since we want a window function that is well concentrated and smooth so typically the type of function for which the zaktransform would be continuous we cannot have an orthonormal basis so what we typically will do is w reasonably smooth and concentrated and so orthonormal basis is impossible and that gets expressed by the fact that we typically take tau times omega less than 2 pi and I'll I'll come back on I mean we'll be using this and using and looking at properties of them later but I also already want to give you a little bit an advance notice of something else that we will do although it turns out that you can't make an orthonormal basis this way it it it uh became apparent in the 1990s that in fact if you go contrary to our all our impulses uh of of that we we we learn when we when we as an undergraduate or a high school student learn first about complex exponentials contrary to all those impulses we wanted to stick with science and cosines rather than with complex exponentials so if you try to do so work instead of e to the i something x i t with sin x i t and cos sin x i t you can in fact build orthonormal basis so just by looking at these and and and and and so you can find very nice window functions for which you still get an orthonormal basis and it turns out those are useful and uh uh they they uh this was a construction that uh originally was done by uh stéphane jaffar genre journée and myself and we we built them uh I insisted that we build examples that were based on starting with a w that was had a a gaussian profile and we had to take certain combinations and we found combinations that had exponential decay uh I liked taking gaussian functions because I knew that people in uh molecular and atomic physics love working with gaussians because it means you never have to compute the integral explicitly because all those gaussian integrals you know what they give and so you have very fast algorithms to compute things with and so although we we built them and I I paddled them for some years to people who are doing computations in an atomic and molecular physics and some of them used them they did not become widespread but then it turned out in a completely different physical application in the search for gravitational waves that it was very very useful to have things of that nature and they did play a role in in the lego detection of gravitational waves so out of a completely different physical angle they they they turned out to be useful so Ken Wilson cited yes so this was based I mean and and but I just wanted to give a an an an an intro to it I mean we'll come back to them later I mean and not today this construction here was based on an observation by Ken Wilson um who had pointed out that in many opportunities yes he says if you want to book or to localize in time and frequency you want to look at things that shift in time and shift in frequency yes but he says often in physics you don't want to actually localize in frequency you want to localize in frequency squared because it has to do with energy and and and energy is a function of frequency or momentum squared he was thinking quantum mechanics or momentum squared and he said then working with I mean the difference between if you think in in in on the Fourier side uh an exponential localizes I mean if I take uh okay so let me backtrack if I take my function w and I move it around uh I move it around here 2 pi i m omega t we've seen that in Fourier that amounts to taking omega hat psi uh plus and omega and e to the minus 2 pi i uh well e to the I have no 2 pi here then I have here 2 pi n omega and here I have e to the minus uh so so if if you want to localize in if I think now like those spectrogram pictures that I showed you in time and frequency I want to be nicely localized around m tau so w is nicely localized on zero and that means that this thing is nicely localized around m tau and if I want to be localized nicely in frequency again I think of w hat as something that's nicely localized around zero then I'm here at minus 2 pi i n omega so I think of it this way but if I don't care whether I am localized around this frequency or that frequency then that means I don't care I mean I don't care whether I have here this or that and that would give me a w plus minus I'm here or there in frequency and that means that I have the freedom to take cosines or sines here which are exactly combinations of two of these and so he said because I want to have localization in frequency squared uh cosines or sines are admissible and he then did numerical constructions that showed that that it might be possible to well localize things I mean the funny thing about that paper is that it was never published I mean actually I I I have written to uh Ken Wilson's widow asking him for the permission to post it on the archive so people will actually see it uh but uh I haven't heard back from her I have to call her again to see um you found it now well it you can find a later paper that uh contains the construction but it's not the original preprint yes okay so you can get it from the Cornell archive so uh uh and that's but but I would like it to be on on on on on archive because uh in order to get it from from from the Cornell archive you know have to know exactly where to to look well archive is very well searchable and and you can find it via google I did this too and and and that's how I but uh uh but even the one you get from the Cornell archive is not the original preprint that Jean La Journée and Jafar and I uh used I mean there's another version that was was prior to the unpublished paper in in the Cornell archive and um and in places where google doesn't work you can't find it and there exist places in the world where google doesn't work so I said so I have a question because before omega tau equal to pi was an assumption you started saying ah we are looking for a normal basis yes I is a necessary condition is this yes then you put under this necessary condition there does not exist are you right now you are changing two things you are saying maybe the exponential two pi i becomes sine cosine and I also relax this no what is the logic no no no there's okay so I I swapped something under the rug in saying I mean you can actually prove but I had not planned to do it in these lectures that for an optimal basis you must have omega tau equals two pi if you have an orthogonal base you must have omega tau equals two pi it's the only and and uh but uh if I can not get an orthonormal basis uh then and I do want to to have nice constructions with smooth functions and I do want to have an equivalence and we'll we'll get back to that between the l2 square of the function that I'm transforming and the coefficient then I must take omega tau less than two pi if I just relax the exponential in sine and cosine then I get an orthonormal basis with omega tau equals to two pi yes so you don't need to relax you don't need to relax if you relax the idea that you want to really localize if you allow for so in in in what what did if if you think in in frequency I was always thinking of things that were nicely if I look at w hat I want a w hat of xi plus and I wanted to to have w m n hat to be nicely localized around m xi if I relax that it needs to be localized around m xi it can also have a bump around minus uh sorry not m xi around uh m omega around minus m omega then I can make an orthonormal basis with and orthonormal basis if I just use sines and cosines still will require that omega times tau equals two pi that's one thing and that's the one that was used for for that's a wilson basis idea and that's the one for which the the construction using Gaussians linear combinations of Gaussians with exponential tails was used in LIGO on the other hand and this is a different stream we're not trying to do orthonormal basis because in general we do want if you do signal analysis you really do want to localize exactly in frequency one reason for that is that we may want in in uh in uh work to work with images for instance with with signals that have in two dimensions if we start localizing if we start looking at bases that in one dimension have two peaks and we take products of them I mean so then I'm going to have and this is something that we do with wavelets then we're going to have localization here and so in in frequency in two dimensions we would have things that are localized on those four corners and those correspond to completely different directions in space and so that is something we do not want okay by the way music the notation would be real no there's a sin cosine no true yeah and well you look at the real things and so yes you have uh and so in in in one variable yes yeah but the other thing is that in many situations so for music you look at the sound but uh the sound itself because of the way it's produced it makes sense to view it as what's sometimes called an elliptic signal so the real part of a complex function and as soon as you work with complex functions looking at at at at the positive and the negative frequent I mean even if you work with cosines and sines you have these negative frequencies that come in and you have to deal with so um there are a number of reasons why for certain applications you do want that exact localization frequency okay so I Pierre had proposed that we would have a break in the middle because otherwise it becomes very very long so maybe this is a natural point to have a break of 15 minutes and then resume we had reached a natural stopping point because I had wanted to tell you about those orthonormal basis with sines and cosines although we will do the computation at a later point um but uh so let me now go back to where we were we were looking at the window Fourier transform and uh it consisted in taking a function over the real line and uh looking at a uh a window function that we multiplied with it and that we translated and we also looked at a uh an exponential factor let me change my integration I'm sorry notation is something that I'm very fluid so I hope you won't mind the uh Paul Hamas who's a mathematician who wrote a number of books gave the advice uh that you should in order to make your life easy uh adopt notations in what you did that you then stuck through your whole life so that even if you looked at all notes you would immediately know and so on and I thought oh my god this it just doesn't work for me so I look at all notes and I never understand them but uh so uh let's do this and uh so the difference with what I was doing before was that I already discretized translation factor and as well as the uh uh so before I wrote an e to the minus i and omega t because I was integrating over t but so you should have thought of that as really an omega not a parameter that I fixed and what I was varying was n here let's oh well maybe let's write a new instead of so here I'm thinking of t and omega so I have a transform of the function f and I look in two variables t and new um so I'm now going to think of them continuously because that also is a point of view that can be very useful so uh I'm going to look at the window Fourier transform for continuous frequency shifts and the earlier window Fourier transform that I was looking at you can then think of as a kind of discretization a sampling of this continuous transform and all these different points of view turn out to be useful in different ways so we're looking at similar objects but from different angles okay so if I do this then uh it turns out that uh this t is going to map l2 of r to l2 of r2 so again we'll have a mapping and it will have interesting properties in fact if we choose our window uh such that it it has l2 norm one then uh up to a uh well in any case up to a constant the whole thing will be a uh a unitary uh will yeah will preserve inner products and norms again so let's let's let's see how that oops sorry um okay so let's take tf t new t g and integrate over r2 this thing uh first of all uh if you look at this expression as uh and you concentrate on the variable new then it looks exactly like a Fourier transform up to the square root of pi that I like to introduce so let's just do that let's integrate only over new and see what we get well we have that famous two pi because I don't have the one over square root of pi in them I have to introduce it now and uh the function of which I'm taking the Fourier transform is this so I have that integral over new is going to be the uh fs ws minus t and then I have here gs conjugate ws minus t conjugate ds over r I've just used the unitary of the Fourier transform I am now going to integrate also so that was the integral over r in new if I add another integral in r over t then I have integrated over uh two over over r2 and I have here this and now there's something very interesting because I have and again all these things make sense and all the exchanges of integration I mean I'm going to make an exchange of integration now is are justified if things converge absolutely so I will initially uh one should work with functions f and g I mean my function when my window function I always like to take nicely decaying so that integral is not a problem but I will also take f and g that are sufficient to decay that the integral converges absolutely so I have no problem and so I can then exchange integration and I prove then certain equivalences namely I'll prove an equivalent of norm which then extends to unitarity to arbitrary functions the standard argument so if I exchange order of integration then I can see that I have here an integral I can change I can absorb in integration over t this variable s and so I have that I find here um two pi and then integration over s fs gs and then I have here the integral over u of w u squared and so that if I take w to be uh of of l2 norm one is just one and I find here that this is two pi in a product of f and g in l2 of r and as promised now I have equivalent of norm and I can extend to functions that are less nice so what you find is that indeed this operation t preserves norms it it's not unitary of course I mean I'm going to a much too big space um if let me read a little bit more if I write here as an index as a subscript the window function I chose and if I in in this integral if I were to allow myself the liberty of choosing different windows then the computation that I've made would still work I would here have my window one and here have my window two and what would happen here is that I would find window one u window two u du and so I would find here the inner product of w1 and w2 in l2 of r and so that already what happens is that I get something very nice the tw's map a little l2 an l2 space of one dimension to l2 of r2 and I have I have for different windows if I have orthogonal windows I find well it's not this way it's it's orthogonal windows how map to orthogonal subspaces because if w1 is orthogonal to w2 then this inner product becomes zero and you have orthogonality so you you have this very nice mapping of of of this space into uh in fact if you take a collection of w's that form themselves of an orthonormal basis in l2 of r then the collection of their images will span l2 of r2 you can prove that I won't go into that but you have this this very nice way of mapping l2 of r into this this whole fan of of of subspaces of l2 of r2 all of them in in in a way that's non-preserving inner product preserving you might say well when am I ever going to have windows that are orthogonal to each other uh I mean because I'm thinking of them as rounded off uh uh uh window functions now I might round off a window function differently I might I might make it a little steeper or a little softer or and so on none of those functions are ever going to be orthogonal well yes and no of course those are never going to be orthogonal but it actually makes sense to want to look at orthogonal windows and this has been done uh has been put into practice by uh engineers there is uh uh uh something that's called multi tapered windowing uh which has been uh which is used which was proposed by David Thompson who was then at Bell Labs he's now I think at uh at the University in Canada and I'm not sure which one anymore in Toronto um in which uh he proposes the use of different windows that are in fact orthogonal the reason is not quite this windowed Fourier transform although it has been used in that context as well the reason he proposed multi tapering was that uh the kind of problems you have with very sharp cutoffs in in in analysis of of of data uh happen also if you just analyze data that are sampled over a finite interval uh what happens is that uh again if you just you have all your samples and you you typically compute spectra by a Fourier transform of that that whole sequence of of data you have again you're again mathematically introducing a discontinuity typically if things don't end in the same way as they started and so it is because a one way of looking at it is like saying I have implicitly taken an infinite series of which I only have a finite number of observations so I implicitly have multiplied them with a window function like that and so I am automatically am causing bad decay in the resulting Fourier series because of my windowing because I only having finite number of observations that that is very very slow because of this sharp cutoff so the what people do then is say we want to taper I mean instead of of of I mean tapering means putting a a decay here and with my windowed Fourier transform I said look we'll have overlapping windows we'll go a little wider and so on you can go a little wider if you have a little wider to go and when I have lots of data that I cut into little pieces I can do that people who work with finite number of observations of something that they know has an unlimited time series but you cannot observe for longer they only have this many so they can't go further and so they have to go inside in order to taper but then of course you get into big trouble with the experimentalists because they have worked very very hard to get those observations there and here you're saying that you're not going to give them much weight in your analysis so I mean what Dave Thompson discovered is that you should taper but you should taper with several different windows that are orthogonal and that allows you to exploit the data that you have here at the very end to maximal effect and so he proved very beautiful theorems showing that and in fact in our windowed Fourier transform world the way we look at it that corresponds to taking w's that are orthogonal the functions that he proposed are that's a different matter which I have not planned to talk about but I can in in these lectures but I can answer questions you're in breaks if you want so what he proposed was use orthogonal prolet's freudal functions and those happen to be functions that have very nice localization properties in time frequency space and I can come back to that if there's interest in some later lectures but not I hadn't planned to do so any case so yes it makes sense to talk about orthogonal windows and in in in in many different applications and even though you might think of a window function necessarily as something positive okay so we have this this way of looking at so what we have is that the integral of f t nu squared dt d nu over r2 is the integral we had of our original function up to 2 pi 1 over 2 pi and so this is a a special case of of what is sometimes called square integrable representations of a group because what I have here is the inner product of f with my shifted and and translated window and if I let's introduce an operator that does exactly that I have here g s minus t e to d the way I've defined it I should write e to d i nu s and if I write as an inner product I really should have written a conjugation there but my windows typically are real so let me strictly speaking I would have to write this here which would have made it this and that this and that and now I'm being exactly correct even if windows were complex but so then I'm allowed to write it here as this operator acting on my window and that inner product in out to r so I'm I'm I'm I'm I'm started with one function f and I'm taking inner products of f with the orbit of a group acting on w a group representation so let me introduce those operators so we have introduced these operators t nu and they actually former presentation of the Weylheisenberg group so let me in in the remaining time explain what I I'm I'm saying here again this will be a building block that we will be using in in in future lectures so let's look at what sequence of such applying to operators does to a function I mean if we use the definition then we get sorry we use the definition and we get this function in the argument minus t1 sorry don't know what I'm doing ah e to the i nu 1 s and then I have to work this out and I get g of s minus t1 minus t2 e to the i nu 2 and then this argument s minus t1 e to the i nu 1 s and you see that I'm shifting here by t1 plus t2 and in frequency I have a nu 1 s and a nu 2 s and a nu 1 s so I have exactly what I would obtain if I applied this operator to g in s except I have an extra phase factor e to the minus i nu 2 t1 and in fact and for my purposes it will be useful to change things slightly to make this a little bit I mean to me I want to I like to make that formula a little bit more symmetric and so I'll write here and minus i nu t over 2 what that means is that I don't have exactly this anymore if I insist on writing my window Fourier transform so I would have to write that this is e to the i nu t2 that's but I still have I mean in this formula here I still have that this is exactly the same as 1 over 2 pi r2 the absolute values of these things squared because the only thing I've done is a phase factor so to be consistent here I would have to be introducing those phase factors as well everywhere so here okay I have worked out a wt1 so since that contains that phase factor I have to write this phase factor now here then when I work out this here I have to add also a phase factor for that so I get here minus i over 2 nu 1 t1 minus i over 2 nu 2 t2 and now I have to be a little bit more careful about so let's let's write out all those extra phase factors because I have a minus i over 2 nu 1 t1 minus i over 2 nu 2 t2 and then now that I have taken this together to write a w I am missing the phase factor so I have to write also an i over 2 nu 1 plus nu 2 t1 plus t2 and now you'll see why I set this whole thing introduced symmetry because if I work all this out I mean so let's work out all these phase factors so I get exponential and then I have i over 2 the nu 1 t1 drops out the nu 2 t2 drops out I have a nu 1 t2 here and then I have another cross product a nu 2 t1 with a half but I have a subtracted one so there remains another half and so what I found is that w t1 nu 1 times w t2 nu 2 is equal to this phase factor times w of t1 plus t2 nu 1 plus nu 2 so I don't quite have a representation of the group because I have this extra phase factor now you can what you can do is you can extend the notion of representation from unitary representation to a projective unitary representation or I can just make my group a little bigger and let's let's do it that way let's think of let's think of of group elements as t and nu both in r and z element of the complex numbers uh with norm with magnitude 1 the one-dimensional torus and let's now define a group law to be the following I add the times I add the frequencies and I take here a z1 z2 e to the i over 2 t2 minus nu 2 t1 and let's now think of u z t nu acting on a function so this is a group law on well on the collection of of of these objects and you can easily check I mean that there's a unit element which is 1 0 0 that you have inverses and so on so it's a nice group and it is uh the Heisenberg group I mean what you have is this this this kind of of uh phase this phase factor incorporates this the the well it's not an accident that it has the same name as a Heisenberg uncertainty principle because in fact in in in this is the way in which you translations in phase space get represented into translations in the Hilbert space h2 uh h which is l2r that represents quantum mechanics and it's this phase factor that that causes uh the non-commuting of translations in time and frequency in time and moment in position and momentum that that that is that gives rise to the uncertainty principle okay so if I define this to be uh z times w t nu f then and I probably need to put a minus for a time but we'll check here for we'll check because I I always have to figure it out uh u z1 t1 nu 1 u z2 t2 nu 2 is then oops z1 z2 u uh w t1 nu 1 w t2 nu 2 and this gives rise to that and so I did need to put the minus factor here because then you find that indeed when you work it all out that this is the u of z1 t1 nu 1 composed with z2 to t2 nu 2 with the group law that we have acting on f uh so no f here because you get a phase factor from this and that phase factor no you don't need that sorry so you have a unitary representation in l2 of r of this group which is the Weill-Hasenberg group and of course in in what and what we have is so when you have a group with a group law and you have a unitary representation from g into the unitary operators so uh into the bounded operators on h but in fact unitary so the subset of of bounded operators that are unitary and you have uh that f uh ug h squared and I integrate mean uh this is a a group on which the left and the right harm measure are the same and so I integrate over that harm measure and what I get here is that up to a constant I get that this is f squared in h uh no no so in fact I have this uh and and if h is 1 then this disappears so this is called a square integrable uh group representation uh there exists a very nice theory for these it can be extended to a theory in which you don't have left and right invariant measures the same and what happens then is that you may have to you don't get a multiple of the identity operator here you might get the multiple of of another upper self-adjoint operator but but so that's what we find here and uh we have three words here I'm going to use this magic stick so what I have is that uh I have found that I have this unitary I mean that the fact that this operator is uh well sorry let's let's backtrack I have found these operators t new they're unitary when I write they are an example of this situation the group that I have had three parameters had this z as well as the uh uh t and nu but of course the integral over z is trivial since z is just a multiplication factor and you could view that integral as actually contributing that factor to pi so you can make it very nice and and and elegant I mean uh but it's much more convenient I mean in practice to forget about z and to just uh look at this and we have t new on some window well let's let's since I have here it's a couple years sorry thank you so I have this since I have unitarity it means that I mean and I could have used the same computation to justify this or I can just obtain it by polarization out of the preserve but if I now write uh so I could have I could redo that whole computation but I can also think of the fact that I have a uh uh uh that this is really the inner product of tw of tw with itself and up to well I had a 1 over 2 pi here which I had forgotten 1 over 2 pi and this is f and f and then because you can always represent inner products as linear combinations have appropriately chosen uh uh norm squared of f plus g and f minus g and f plus i g and so on you automatically inherit that equality for non diagonal things so uh I also have that this is equal to the inner product of f and g okay so here I have a window if I write that all out then I just get here f g so this is sometimes called the resolution of the identity because here I could just think this is an identity operator so what I'm saying is that if I write and I think of this now as a this this I have these rank one projection operators unscaled versions scaled and translated not scaled translated and modulated versions of my window and I each of these is a rank one projection operator I mean this is a rank one projection operator and here I look at diagonal elements of this rank one and if I integrate over the group uh uh uh the group variable I find that this gives me the identity operator so the interpretation of this formula is that I'm looking at something that localizes each one of these localizes nicely the original function on a particular place in time and frequency and of course uh uh governed by the window that I picked a different window will give me different projection and together they give me little pieces of my function which when I add them give the original function so if I think of it this way if I think of this integral on the left being defined weekly namely by how it interacts on functions I I have this I have a way of reconstructing functions by taking things that are very well localized and it tells me what I have to can put in front in order to uh now I can put many other things in front here in fact we've already seen ways of in which we change that if we have here our window and I take another window here a w twiddle that's orthogonal to my window w I will have a different coefficient function here and will still give the same function so there's zillions of functions I can put there but this is a very convenient one and it turns out it's the one that has the smallest l2 norm in the full space in the full double variable space okay so we're capitulating because I'm going to close here for today and we'll resume next week Monday morning what have we seen we have seen I have Monday morning yes so so next week Monday we'll have morning and afternoon and then the week after we'll have also morning and afternoon and and maybe I'll and that should conclude the course and maybe I'll try to to to cram a little bit more in those lectures then so that we make up for what we missed yesterday after I finish this this summing up I'll make a few announcements because so what have we done we have motivated the idea that we want to look at either discrete or continuously indexed a windowed Fourier transform to localize in time and in frequency we have seen that there was no orthonormal basis if we did it with localizing in time and localizing in frequency via complex exponentials I have mentioned in passing that you could get around that and that's something that was only realized in the 90s although it could have been realized many times before a long time before and it was inspired by computations of will by Wilson that you can get around this this this no-go theorem by taking looking at signs and cos signs but we'll revisit that we have seen that if you take it continuously indexed you actually have this very nice way of mapping l2 of r in tilt l2 of the index space the t and new space and you map on a subspace but you can map on different subspaces there and you pack them very nicely in that space and then we have seen that norm preserving map we have a different interpretation of it I mean it's still the same thing but we have this different interpretation as a superposition of projection operators of rank one projection operators that gives you the whole function again one thing that we will do with this next time is we'll wonder what happens if you integrate over less than r2 you're not going to have the identity operator but I will argue that you can localize that way you can find nice localization operators and actually some of those will turn out to be very useful okay that's it done for today