 Okay, so now you can hear me as well. So I'm from the University of Western Australia. I have a position also with CSIRO, which is the Australian Government Research Agency, and this work is jointly funded by, well it's been funded by the University Grants Council of Hong Kong some time ago when I was based there and more recently by the Australian equivalent. So let me start with turbulence and try to provide a segue between the topic of this conference and what I want to talk about. I want to start with this thing called Tarkin's Theorem. So I learnt a couple of days ago that the way to motivate talks here is you've got to start with some really old paper and then move forward from that. The best I can do is do one, but this is kind of the foundation of nonlinear time series analysis, which is what I'll be talking about later on. It's a field where we're trying to understand and quantify chaotic dynamics from measured time series from the system. And the question that Flores was looking at here was related to turbulence. So people were trying to understand turbulence by the so-called power spectrum. Trying to decompose the signal into its periodic components and understand those to understand the dynamics. The problem is if you're looking at a system which is actually chaotic this is not a terribly good way of going about it. And so what he was looking at was how to understand if indeed there existed a strange attractor in these sorts of systems, how would you look at getting that out? And in fact there is actually a kind of fluid problem in this paper. It's a telecoat experiment and I won't lecture anyone in this room about that. You all know more about this than I do anyway. So essentially all they were doing was measuring our velocity component at some point. So they've got the spatial temporal system, turbulence and they're measuring a time series out of that and trying to do something with that time series. And that's kind of where I'm coming from as well. So let me now leave the turbulence and the motivation alone and talk a little bit about the more recent work that we've done which has sought to extend the idea of Tarkin's theorem to situations where you have real data and it's really a mess to try to reconstruct. So the general framework that goes back to Tarkin's and before that to topological theorems of people like Whitney and Maine is you have a scalar time series XT and you reconstruct a vector time series VT by taking lagged components of the scalar time series. So the framework here is all that I can measure is a scalar time series. But my system is probably not one dimensional so my state vector needs to have more than one component. And I get those additional components out of the lagged versions of that. A simple, intuitive way of thinking about why this might work is that by taking successive scalar components I have information by which I can construct successive derivatives and I can therefore construct the state space in terms of a single scalar component and its derivative. So an nth order ODE system. And I'm then interested in how those dynamics evolve for the vectors. There's a lot of technical issues with that not least of which how do you know what the embedding dimension is priori, how do you know what a good choice of lag time is how do you measure these things, how do you separate signal from noise, all these issues come into play especially when you've got complex complicated signal from a high dimensional system and I'm not even going to touch on the fact that your system might be infinite dimensional. But we took a kind of pragmatic approach to this and said well okay what can we get out of the data forget about what the original system may really be hiding down there but what can we actually extract from the data. And so the basic idea is that we represent this object not as a strangest tractor, not as a reconstruction of an attractor in time delay coordinates but as a network and the reason for that was well I had a couple of PhD students at the time and the early part of the first love last decade networks were cool and people were interested in them so we'll kind of just using the network tools for this reconstruction. The bonus is that it gives you a whole new bunch of things you can measure from this network perspective for your system. So if you do that we now have basically three different objects that we're considering you've got the time series up the top here which is what you're measuring that's all the information you have about the system. You've got the standard embedding theorems which give you this thing in Euclidean space which is a representation of your attractor which gives you the dynamics successive points and here is your temporal evolution of your state variable. And we're constructing this new object over here which is a network. Now the way that we're constructing the network the nodes on the network and for any pure mathematicians in the audience network equals graph but I'm using the physics terminology just because that's where things have popped up. The nodes on the network are different states and we're connecting states not because that's where your system evolves from but because they're close together in your state space. We're going to change the rules of the game later on but I just want to introduce you to this earlier idea first, primarily so I can show you some pretty pictures. This is a time series generated from the Karatek-Rossler system that's the original system in XYZ coordinates, it's three dimensional ODE, it's chaotic, it's got the classic stretching and folding, the dots are your observations sampled at some regular time interval and the yellow lines are just joining the dots. The thing on the bottom is the network that we get out and what's immediately clear to me at least is that that kind of well if you zoom in on it right you can see oh I killed it, you zoom in on it, it's not handling that well you saw it for a second but if you zoom in on it you can see there's this recurrent sort of self-similarity in the structure it's complicated interconnection between points. Sorry, yes but not just the distance between points so we're looking at distance between points but we only connect points to their k-nearest neighbors so rather than having like an epsilon ball where all things within a distance epsilon are connected, we're only connecting the top few neighbors. This is the closest neighbors in this picture so this picture is just a representation of the network, the space in which that picture is is abstract, the points are placed in a Euclidean space based on what's called a spring embedding so we are taking the points who are either connected or not connected on the network and placing them so that points that are connected are close together and points that aren't close-ish. There's a spring constant between the points so that you naturally want points that are connected to be attracted to each other and you naturally repel points that aren't well this is, the visualization is not central to what I want to do, I'll show you on the next slide what I'm actually quantifying. This is just a picture to show you the motivation for what we're looking at I think you know better than most where this falls down but let me just skip forward in that case to the next part and if we take this network and then ask now points are connected if they are close in the Euclidean space of the embedding, we only connect each point to its k nearest neighbors, that means that if you have regions of state space that are relatively sparsely populated and you will have a similar number of connections so we are equalizing over the measure of the space if you like which does a few strange things but if we just look for a very simple property and that is in this picture if we take that graph and look at what small subgraphs we see, so how often do we see each of these different patterns of connections between neighboring states so if I have four nodes on the graph that are connected they could be connected in one of these six ways, if I count up how often each of these things occur and look at the rate of frequency of that then I see in this case one particular ordering of these different graphs and this is for noisy periodic systems if I do the same thing in the case where the underlying system is chaotic, what you notice is that the ordering of these subgraphs changes and this is a very simple consequence of the local topological dimension of your attractor because what's happening is if your attractor is locally one dimensional you can't see this occur, if it's locally higher dimensional then you will see this occur, does that make sense? From the data to the graph, so the data to the time series is standard embedding, having done the standard embedding we're in Euclidean space and then I look at the nearest neighbors to each point, in space to each point, I rank the nearest neighbors sorry, even if it's coming a long time back so I'm assuming stationarity for which neighbors in space are close there's a fixed number of neighbors here, that's the parameter, 4 in this case and so each point is connected on the graph to its 4 nearest neighbors now the reason for doing so, if we just put an epsilon ball around this and said ok we'll have connected everything in those then that's just recurrence plots but by doing this we're looking at something slightly different and when I get to the next part, I'm going to throw this away and do a different method of doing this which actually looks at the so once I've drawn all those connections and I've got the entire graph, I then look at all the subgraphs on that so take every vertical on that graph and look at all subgraphs containing that vertex and 3 of its neighbors so those subgraphs of 4 nodes could be one of these 6 different things only that it's one of the closest neighbors, exactly so you know now? I won't ask if you're happy with it but I'll ask if it makes sense now the method makes sense so this was the first thing that we did, we saw that we these different subgraphs and the question that we ask of course is does this happen this one here? so this is depending on what type of time series or what type of system I'm generating a time series from I then do the embedding, I then build the network, I then look at the subgraphs and you see different subgraphs occurring more or less frequently from right to left and from left to right is most frequent low dimensional chaos, if it's hyper chaos, if you've got more than one positive lap enough exponent it does something different and again it's related to the local dimension if you're on a high dimensional flow, you've got more degrees of freedom to see your nearest neighbors so that's the trivial case of just pure noise so that's just going to fill up your embedding space no, periodic flow and chaotic map it's these two here that caused the switch this may seem to be splitting here because it's just one transposition but on the previous page you could see periodic flow and chaotic map chaotic map and periodic flow is the same that one I'm not sure about, I think I've just copied the wrong line for that but this is different from chaotic flow this is one of the unsatisfactory things about this method and I'm sure you can tell me others but this is one of the unsatisfactory things about this method is that I'm doing a sampling so everything is essentially a map because I've got discrete states yet if the map is generated by a chaotic flow or if the map is generated by a chaotic map they do look different from each other the only important thing about these sequence of patterns is that if you ask yourself why would some things be more frequent than others and again to emphasise I'm talking about more frequent on a logarithmic scale so I'm counting up these things from a graph that I've generated but there's orders of magnitude difference in the frequency and the reason is what I want to get to on this picture but for example a periodic system where you've only got one degree of freedom then my nearest neighbours are on that same one dimension if you think about pure noise where you're filling up whatever your embedding space is then your neighbours can be in any dimensions so for example if you're in one dimension you can't have this structure because it's yep low order periodic orbits if we have a time series then that time series could be periodic of period equal to the length of the time series it could be anything going back to what I said at the start I'm concerning myself with what I can extract from the data so assuming that I've only measured this system for so long I'm already discounting like the possibility that it's a periodical but of order the length of the time series okay I think in the interest of time what I'm going to do is say this was the first way we were doing this to kind of motivate what we want to do now I'm going to skip forward instead and say well there's a different way in which we construct this which comes from some work that people have been doing looking at permutations of your points of your time series so the basic idea here is that everything I've shown you so far I'm connecting things if that's close in space and there are some real problems with that in some sense it would be nice to have the connections that reflect the dynamics so I'm connecting things if I'm evolving from one state to the next the other problem with the method I've just shown you is that it scales with the number of length of time series if each point is and embedding is a new point in your time series the longer the time series the larger the network it's kind of a little bit contrary to what we would like to get to if we want the network to represent the system we want to converge on something so instead if we think about somehow partitioning your state space and so the basic idea here is you've got your time series and instead of considering that war time series consider the order of the points within that short segment so by this I'm taking and the one parameter here is the window size I'm taking four successive points and what I'm saying is the first point of those four is the smallest the third point is the largest the second point is the second largest and the fourth point is the third largest so it's just the relative size of those points so I've chosen a window length I've got W factorial different possible permutations that could occur and I now say each of those permutations is a node on the network so I'm now looking at transitions between permutations now why might this be interesting if you think about the logistic map so here is the identity line logistic map and the second is the logistic map so the logistic map is f of x that's f of f of x and if I look at the possible orderings of points that can occur just in that simple one dimensional map these are the five orderings that I get one of them it's simply not possible to get out of the deterministic system because of the way that particular map is moving the points of that state space so now that's the way I'm going to represent the nodes on the network and if I go from one to the next so does that make sense? correct this is with this system you have a generating partition which you can generate easily in this system you can get to that generating partition in general you can't but you can always look at the coding of the points in this way so let me skip forward to that one so if we do this trick we generate these networks here for different types of signals so the number of nodes in the network is now a reflection of the number of permutations the window size that we've chosen looking at the system for longer and longer is not going to give us more nodes once we've sampled well enough to have seen what the system will do it's going to give us better estimates of the same probabilities between those sequences so we don't actually need to have the delay embedding we no longer need the state space we no longer need to be concerned with how far points how far apart points are in the embedded space because we're doing the simple encoding which in some very specific instances is a generating partition but which in general is not and doesn't need to be it's just a representation of the region of state space which can be got robustly so the key here is that I'm working from data and I want to be able to generate this in a way which is robust to noise to all those sorts of problems that we have and the first demonstration I'm showing you here is the first refuge of an experimental system when we're not very confident of the experimental system working in general we've got an electronic circuit so an analog to generating the data we can generate lots of data we can generate it very cleanly we can get reasonable representations of these pictures and then we can go and force that circuit through its bifurcation through the period-doubling bifurcation that happens to occur and then we can measure properties of the network that we've generated at different points in that bifurcation spectrum this is the largest sloppin' of exponent these three however are different so it's a large sloppin' of exponent estimated from the time series that I've generated from the system these are three properties of the network that change over the same bifurcation spectrum that's just the same picture just to show you this is actually the property of the network it's related to the out-degree of the node the mean variance in the out-degree of the nodes and you see it varying over the bifurcation spectrum and dropping down in the periodic windows which is not too surprising because it's drop goes from being chaotic over the observable time horizon that we're looking at to being strictly periodic but it manages to pick up all those periodic windows and more importantly it's increasing in a similar fashion to what the Lapland of Expandage is increasing sorry here it's a period dubbing bifurcation so it starts at a period one and goes to a period two period four so I haven't shown the part where it's just period one just because it's further away but if it's periodic it's still going to have the same sort of low value now let me just finish because we've already got a lot of questions so let me just finish with okay what can this do if I have some data that's more realistic than an electronic circuit what can I do if I have something where it's noisy it's high dimensional I've got limited observations of it so I don't really have a hope of estimating those dynamic invariants and all those other properties that you might like to have but generally can't estimate this situation where am I going we want to be able to measure something from the network which is useful and back to that permutation sequence those permutation sequences are the nodes on the network and I'm looking at transitions between those nodes given those transitions we can calculate these entropy like quantities which are the either the distribution of the sequences themselves which is nothing to do with the dynamics that I'm talking about here but just the frequency of the codings the transitions between those states which now reflects the transition between states in my space or a global average of that each of these measures has a single parameter which is the window size that I'm looking at which is just going to be a scale for your dynamics of your system and here I'm going to show you I'll show you two sets here so first of all this is some data that we recorded from patients in a coronary care unit many years ago undergoing transition from normal sinus rhythm through tachycardia defibrillation and okay if we compute these entropy like quantities from that network and then look at the distribution of those points for two different sets of the parameters we separate out most of those different rhythms that's good it's not too impressive because if you give the same time series to a cardiologist obviously they can tell you whether your infibrillation or tachycardia or normal sinus rhythm most of the time part of the reason it doesn't work entirely is that these are clinical ECGs so they are only that they are measured at a fairly low resolution this is 8-bit data a fairly low sampling rate just so that the people on the clinical ward can see what's going on it's not research data it's clinical data if I then do the same thing with a different data set this is called the fantasia data set it's recordings of EEG for old people and young people it might as someone under 42 I didn't make the classification so don't argue with me on that watching the movie fantasia and again the same the same trick build these networks look at these entropy scores for different parameter values so different time scales and you see these clouds of points which kind of separate young from old of course it's surprising that there's any separation at all here the signals all look the same some of the old ones are kind of closer to the young ones than others that's probably accurate of most of our experience not everyone acts their age in the same way some old people were more like the young people when vice versa what we can do is repeat this trick with different sets of their parameters and look for separation of two clusters and we do it in higher dimension we actually do get that separation and this is just a scatterplot of that variance mean three standard deviations and outliers for those two data sets I think I'm out of time so I'll stop there and finish on that slide and ask for any more questions thank you very much very nice talk