 Now let's continue the lecture of yesterday. Today we want to move towards what I call massively parallel spike trains. I would say more than 10 is almost massively parallel, but 100 also. And as you can imagine, this is so to say not straightforward to extend the methods we were using before. And I would like to suggest you a few which we worked out and which you may want to use for the data challenge. So to remind you, this is what I would consider massively parallel data. And the tasks you will have then for the next three days is, as I said yesterday, the data, the so-called data challenge, that you will have six data sets. And we give you this afternoon the features of these data sets. And your goal is to find this out, which is which. And we will have then a report on this, I think on Friday morning. And then after successful successful identification, you will get the real, you will know which is the real data, and you can also get the local field potentials to work with this on a neuroscientific question. Yeah. So we had this in the last courses kind of mixed, and this was not so successful. So, and we would also like you to form groups, teams, also today for solving these questions. And we have basically more or less for places where the groups can meet separately and discuss and so on. I think this is important to exchange each other. And you can already have this a little bit in the back of your mind with whom and so on. Then the group could go in this room. Another group could go in this room. One group could be here and one in the back, or you only form three groups. But I would strongly suggest to work in groups. More brains are better. Okay. And so we will now go into this massively parallel data. And a question which comes immediately up when you start thinking about this is do we have, do we deal only with pairwise? So these are four neurons. These are sketches. This is time. This is the four neurons in parallel. And now you can think of different kind of correlations between these neurons. So for example, in this case, these neurons always all have pairwise correlations. They are all pairwise correlated. And this may happen, but you may have also only triple correlations. You see this in terms of, it's a sketch, a very, you see that all, there are constellations, different constellations of three neurons synchronized, synchronized activity. And basically you have different types of triple correlations. Or in this case, all these neurons are firing synchronously together. And why do you want to know that? Well, I would like to know that because I would like to know which neurons are forming an assembly. So here I would say all these four form an assembly, but here I would say it's different groups of three which form assemblies. And I would like to distinguish this from a mere pairwise correlation. Or in other words, what I also want to say is in case you look just as pairwise correlations and notice that you find as here a case of four neurons all pair to pair, pairwise correlated, this does not necessarily mean that a exhibit all together a higher order correlation. Now I'm a bit sloppy with my definition of higher order because the mathematics, there are some approaches to do this, but what I mean is sort of say correlations, let's say of four neurons, which is not explained by pairwise or triple wise correlation. There are mathematical frameworks which we tried. Cumulants, for example, which are so to say mathematical way. These are kind of higher order features of data, but this becomes immediately so complicated. Even for a correlation of four, you have that terms and it's very hard for the amount of data we have available typically in experimental data to approach this in the sort of say strict mathematical way. This is why I suggest different methods, now it's becoming a bit fresh, I'm sorry, which are not that, how should I say, correcting for lower order correlations, but indirectly. Okay, so what I also wanted to emphasize that when you want to generate or design a method and you want to build on the pairwise analysis method, for example, unitary events or so, or no, this is something else. I wanted to say that if this, but I said this before already, that if you want to conclude on higher order correlations, you cannot do this from only pairwise correlation, you will miss it. But what you need to do as I said, higher order correlation analysis, and the difficulty is that scaling up, for example, the unitary event analysis to a large number of neurons leads to a combinatorial explosion of parameters. So if you want to see all different constellations of neurons across this 100 parallel recordings, you have two to the power of N, N equals 100 neurons, this would be so, so many, many, many different constellations. And for this, you simply don't have the data available. In addition, you run into a massive multiple testing problem. If you want, for example, to test for the individual patterns, we touched this issue already yesterday. This means if you ask many questions to one data set, for example, if you would be interested in testing for each individual pattern constellation, but have a, for example, a significance level of 5% or so, due to the number of questions you are asking, you would get anyway, 5% significant. And this, and the more, of course, the more questions you ask, the more significant results you get. This, of course, you don't want. So first of all, you want to avoid massive significance testing on the same data. And second, there are correction methods which then, so to say, reduce the significance level in extreme Bonferroni that you divide the significance level by the number of questions you ask. So for example, if you ask 1,000 questions and you have a significance level of 5%, then the significance level is so low that nothing will become significant. But this would be the super strict method, but there are also other ones available. And so we started about 15 years ago with development of statistical methods to analyze such massively parallel spike trends. Now, the first approach actually suggested to me by Wolf-Singer was, he said, why don't you just look at the population activity? And I started to think about this. And actually, it's a quick way of looking at data to get a hint if there is higher order correlation available in some sense. And then you can later on dig into this and focus and try to find out which neurons and so on. So this is based now on a paper I had actually my only paper I have with Moshe Abeles. So I can also provide this to you. So what we did here, so this is only simulated data to make, to really know what we are doing. So these are, for example, 100 parallel spike trains simulated as Poisson processes with a given rate. And in addition, we inserted into 20 of them, not in addition. We inserted into 20 of them. You see here these highly synchronous events and compensated the rates of the background, correspondingly such that all of these neurons had the same fiber and probability. If you randomize the neuron IDs, you can hardly see these events. This is something you should realize because it's not something that you look at it and you see it immediately. And what we mean by population histogram is to bin the time axis, let's say in one or five millisecond bins and sum the number of spikes across the neurons. This is what I mean by a population histogram. And then what you can see here easily, you have when the mere independent background, you get relatively low entries here, but here and then when we actually meet this higher order event, we have a high entry. Super. And if you have independent data, which you could in a, if you have a stationary data, you could simply randomize the spike times, then this looks like this. Okay. Then we thought maybe we should use, we should then look at the distribution of these entries and we call these entries complexity and we form the distribution of all these counts and compare it then to the independent case. We learned yesterday you can make your data independent in one or the other smart way. So this you can then do always and it's a simple computation. No, I don't think so. So what you now see is this is the distribution normalized. This is sort of say the probability to find a certain complexity in there. And what you mainly see here is that you have a peak there on low complexities, but the inserted, the inserted synchrony of 20 is not, kind of does not appear in detail. If you look at the independent data, this looks more or less the same by I at least. But what we thought we could do is to take a bin wise difference, a bin wise difference and then this appears like this. So here is actually what we expected somehow to see. These are, so to say, these core desynchronized events of we inserted 20, but what you probably see is that the peak is a bit higher at a higher value. Why is this some idea? Maybe I show you the dot display again. So in this 20 we had inserted these events of 20, but why do we get maybe sometimes 21, 22, 23? So these inserted events may by chance meet background spikes. Yeah, exactly. So this is why this is a bit broader. Now why does this look like this? Why do we have here a negative peak? Exactly. Exactly. So because this is normalized, there is a probability distribution, the spikes that are here have to be taken from somewhere or are taken from somewhere and this leads to a negative dip. And what I can tell you from experience is that mainly this is often very hard to see, but if you see something like this, you know there is something going on. There is some higher order correlation going on. Now, yeah, it's an interesting idea. Yes, yeah. I see what you mean. Yeah, it's a good idea. We can think about whether this is helpful and how we can use this. Yeah, that's an interesting idea and you are welcome to implement this. So I, yeah, and show me what you found and so on. Yeah, of course. I just want to tell you for this simple case, Poisson and so on and insertion and so on, there you can have and you find this in this paper, analytical description. This is mainly dealing with, with binomial distributions and products of convolution of binomial distributions. If you do it just to get the idea. If you reorganize this, this is what we actually did. We gave these neurons the same firing rate and what we did here is that we reduce the background activity by the coincidence rate. Yeah, to have all these neurons also the same rate. And now to make a calculation to derive what we would expect, where you can reorder them that you deal these segments differently. This is, this is basically the idea because it's independent of when it occurs in time. Yeah, it's just to give you an idea how you can approach this. Okay, what can we do with it? So now we come to this, to this issue of Chittor and Ditter. So let's assume you have data and you want to know, for example, whether, whether there is higher order correlation involved than or off what, what order. What you can then do is, for example, vary the bin size. Yeah, perform a complex, this difference of the complexity distributions for a bin width of two and three and four. And this is here now shown in a, in a color plot. Yeah. So, and what you can see here is that this positive negative from the beginning is well separated from this feigned entry here. And at some point, they kind of run together. Now, so you can try to detect the order of the correlation from, from the second line. If you actually want to also identify the temporal Chittor of your data. So if there is an imprecision, let's focus on this. Otherwise, it's becoming too difficult. I did not explain all the rest. So if you, your temporal, your data have, your data have a temporal Chittor. Okay. So if the group, you have, you have inserted a group here of 20, as we had before. And the data had different temporal Chittors like being synchronized with, within one millisecond, two, three and four. So now if we vary here, the bin width for the analysis, we will notice somehow if the second of the second peak and a kink. And the reason for that is the following. Let's assume you increase more and more the bin width. You start to collect more and more of your, of your higher order events somehow and by chance. And then at some point you capture, you capture the inserted events. And by further increasing the bin size, you are only collecting additional background activity. And this is why here, the slope is changing considerably. Meaning, if you are able to identify this, kink here, you can tell what the temporal Chittor is because you know when you have captured all the events and also the group size or the complexity of your data. Give it a try and let me know how it looks like and how much you can derive from this. Very easy. It's not so time consuming for computation and for the tasks you will get this afternoon. It may be helpful. Okay. This is what I said. Simple measure, easy to compute. You surrogate for implementing the null hypothesis. It can give you an indication of existent correlation and group size. And you can use a variation of the temporal, of bin size to provide the temporal Chittor. Now, now we want to get a step further. We want to somehow scale this up and identify the neurons that are involved in significant patterns. And I learned at some point from a computer scientist. He was talking about a supermarket basket analysis. I don't know if you know about that. So people somehow, the supermarket seemed to be interested to know what people buy together. So that they buy typically wine and cheese and some bread together. And this they use to place the product in the supermarket. That is easier for you or that you buy more. And we were able, together with, his name is not here because he is one of the last authors, Christian Borgelt, to make use of this approach to products in a market basket analysis to a spike train analysis. Because then I said, I mean, we have a similar problem if I consider each neuron as another product, so to say, as another identity. And then the set of products would be the set of neurons which are active at a certain time bin. And so we were able to apply and translate this approach, which is called, where is it, frequent item set mining to the spike train analysis, the synchrony pattern analysis of parallel spike trains. So effectively, this frequent item set mining is an efficient way of counting different patterns. Mining is just counting. When we have counted them, we still don't know whether these are significant or not. So this is the pre-step. And what we do then, in order to perform our significance analysis, we do the following. So from such a data set again, this is again a simulated data set with 100 neurons and 20 neurons being synchronized. We perform, we apply the frequent item set mining to it. And then we get a list of a lot of counts. And this we enter now in this particular histogram. It's a bit difficult to explain. So what we do is we categorize each of our patterns we found into their size or formally called it complexity. So the number of spikes involved in a pattern and their number of occurrences. So you may have a pattern of neuron one, two, three occurring five times. And I don't have this number of spikes because my bin size is small enough. Or if I do my bin size, make it a bit bigger, then I clip it. Because if you start to have different counts, you can do this. But then your patterns become really more difficult. Then it's not only neuron one, two and three, but neuron one having two spikes and neuron three having five and so on. Then your space explodes much more than here. But if you are interested in something like this, we need to think about how you could do that. But here we typically have one spike per neuron. Yeah. Now what we do is, so for example, we said occurring five times, right? So what is going to enter in this bin? In this bin, we count how many patterns there are of size two, of two neurons involved occurring five times. So we don't count how often a certain pattern occurs. But if a pattern occurs with size two and occurring five times, this adds to this bin. And the reason why we build up such a matrix where we pool certain patterns is because of this multiple testing problem. Because in the next step, we want to know if an entry in this matrix is occurring significantly more often than expected by chance or just by chance. Yeah. And we need to ask less questions about 10 by 10 or 10. But still, I will come back to this. We will also correct for multiple testing. Okay. Now, how do we want to do the significance analysis? Well, we do it quite similarly as yesterday. We generate surrogates from our original data and do exactly the same analysis on these surrogates. Exactly the same. Multiple times. In order. So in here, in this case, we we apply a little again. And what we then get is a matrix, which gives you sort of say the probability to find patterns with occurring five times composed of two neurons with a certain probability. And you see the dark part here means, well, they anyway occur by chance. Yeah. But if you are here in the whitish part, the probability to find them by chance is extremely low. Yeah. That's the philosophy. So basically what you can see is by chance, you get anyway patterns which are of low size and relatively low either low size and may occur a bit more often or vice versa. Now we compare the entries of this empirical matrix to this and ask for each entry. Is it occurring more of significantly more often than giving significance level of 1% or so? And also correct the entries with, I forgot the name, not Bonferroni, but another correction due to the multiple testing and then we get a result like this. Yeah. So now here we have read entries which show these bins contain data which occur significantly more often than expected by chance. Now we are not completely done here because what we inserted here were patterns of a complexity of 10 occurring that many times. So what are these others here? Yeah. So these others are patterns which are actually composed of other pattern we included plus a chance background spike or sub patterns of it. And actually we wanted to get rid of this and we wanted really to get back the one which we inserted. And for this we added a particular step where we asked ourselves, well, this has the highest probability to occur. Can we explain the occurrence probability for this times if this pattern is explained by the occurrence probability of this times getting a chance background spike as well, then we say this is sort of say a false positive in a certain sense that it has a coincidence between the significant pattern plus a background spike. So it's a conditional filtering what we add in this case. And when we do this, we end up then with our inserted pattern. But we had a discussion with Moshe the other day and he said maybe you don't want to remove these. We can discuss about this and maybe I come back to this point why we discussed that after. You need to first know which are the patterns which have a high probability beyond chance to compare this mutual conditional. And you would have much too many. Okay. No. Only then we say this, so we look into this bin and then you will notice anyway that you because there will be only one type of pattern composed of yes because by chance they don't occur. Yeah. So this is our experience. Yes. That's true. That's true. That's true. Yeah. It could be. Yeah. What was your question? No. We would only extract here a pattern composed of 10. Well, this pre-step, this frequent item set mining is only extracting the biggest patterns. Yeah. This is the pre-step. Okay. Just to give you a little bit of feeling, so we did quite a lot of calibrations of the method which we use when we do, when we develop methods, we test them for different scenarios with ground truth data. And here, for example, you see the effect of pattern occurrence for different firing rates, just to give you a feeling. So this is of course then also considered in the p-value in the significance test. You see that the higher the firing rate, the larger is this area which of patterns to be generated by chance. Yes. So we have tested this as well in homogeneity across neurons. And our impression was, or what we tested was that this is not affecting the method. Yeah. But you need to make sure that your surrogates have the same in homogeneity. Yeah. Otherwise, you change more features of the data than the one you want to test for. Okay. Now, I can show you an application. Yes, please. Different patterns were redundant. You meant this one. It's basically like this. If you have found a pattern, which is how I don't remember the color coding, if you have a pattern with has the highest occurrence probability of the significant ones. So for example, this one occurred six times. And the other had a lower probability. Well, but this has a lower chance to occur because it has only three spikes. You compare any two of these patterns, whether the larger one, for example, can be explained by the probability of this ten neurons times the occurrence probability of a background spike of neuron 28. Yeah. And if this probability, this multiplication explains the occurrence for this, that this occurs two times, then we say this is sort of say a false positive composed actually by the other one. And you can do this also for sub patterns. Yes. I agree, but sometimes you only have the longest, but also shorter ones, which then survive. But the probability to occur if you count a certain pattern's length is then relatively high if you anyway have a pattern, a pattern, a significant pattern. You cannot only remove this. This is a matter of the counting of the frequent items at my language leads to this. Yes. Because I don't know. Good question. I don't know in the moment. So I wanted to show you an application of this method. The method is called spade and it was published basically mere. This was the mere method and here we, this is the application to the experimental data. So we used, I think in total, we used in total 10 sessions of two monkeys in the task you got already introduced. Again, actually the monkey was supposed to do a different grip with a different force. And the experimental protocol was that the monkey initiates the trial start himself when he's ready to go. Then he gets a cue with the information given by the constellation of the LEDs lighted on either precision grip or side grip. Then he has to wait for a thousand milliseconds. And then he gets in with the go signal also the information of the with what force he has to pull then the object. Then he releases a center key. This is basically the reaction time and then moves its arm to the object. This is the touch of the object and then he pulls it and holds it in a certain position. We wanted to know whether, whether there are sort of say patterns occurring synchronous patterns occurring. Are they specific to different behaviors and so on. And what we did here is because the spade method because of this many surrogates is quite computationally expensive. So we could not do directly as for the unit event analysis, a sliding window approach, but we segmented the data into relevant what we thought behaviorally relevant periods. So around the start around the cue early in the delay late in the delay during the movement aligned on switch release and during the hold of the object. And these segments are all 500 milliseconds long and these are sort of say separately applied for the different behavioral conditions. Side grip high force, side grip low force, precision grip high force, precision grip low force. And for each of these conditions we have about 30 trials. So when we analyze the data we take for example the period around switch release of PG low force from all the trials and analyze this together as one data set. Meaning we have in total of one experiment of 15 minutes we have four times six data sets which we analyze according to the segmentation and extract for each of them the patterns if we find some. This is how we do that. Now this I told you already. Now here you see an example of one session what kind of patterns we found. This is from the whole session and what you see here now it's also you notice already patterns are you need to deal with visualizing them and so on. But what we see here already is that for example the pattern zero is composed of this neuron and that neuron. Pattern one of this neuron and that neuron. And pattern two is composed of that neuron and that neuron and so on. And what you realize already is that these patterns are much smaller than I assumed in my simulations. They have maybe two at most this one three spikes involved. And I will come back to this why this may be the case. But these are in general the patterns we found for example in one session. Yes. First of all you cannot necessarily conclude whether this is a monosynaptic. But to be honest I didn't look at across Wait a second. Why is this relevant? Why would this be relevant for you? So I think they I still hope that they are coming from a sinfile chain which is so to say being a cell assembly but that our sampling is not so good. I come back to this we had a calculation on that. Let me continue a little bit. So and when you look at when these individual patterns occur in the experiment you have now you have now so to say how should I explain the different epochs the different trial types and when the particular pattern occurred and was significant. So what you can see here already that these patterns typically were significant and occurred at different epochs and typically also in different in different trial types. We then we did a lot of statistics on these patterns which occurred for example this is monkey L monkey N. So what we did here is as a function of these epochs and here you see for example the number of patterns occurring in different colors for the different behaviors occurring in the different epochs and one thing which is quite obvious here already and this holds for both monkeys that they mostly occurred during the movement period and it's the amount of pattern seems to be more or less equally distributed across the different trial types. Also the size of the patterns does not vary so much this is now statistics across 10 sessions of each of the monkey. The size does not vary so much so it's also not particular. The number of occurrences of these patterns may be relatively high but they are also not specific and then we also looked here is a better display at the occurrence probability of these patterns as a function of the distance on the array. We come back to this. But no. No. I was thinking and so okay so just to give you an idea how probably it is to find significant patterns as a function of distance on the array and you see that for both monkeys the occurrence probability decays very much with distance and basically on a distance of about above 4.4 millimeters this is more or less diagonal you don't find any anymore. So it's a matter of distance which you can imagine that this is natural because of the connection probability in space also decays however faster actually and single unit single unit. Okay now we were also interested is there a certain how do they occur on cortex so to say. So to put them back onto onto the array and see when do where do these patterns occur and with which orientation and so on and here again for two monkeys here is the general probability of finding patterns across the array here we put for each pair of neurons within a pattern the orientation where they occurred yeah on the array and if you and what we found is that there is actually for the patterns a preferred orientation on the array yeah so for example they are preferably maybe lateral occurring almost identical almost very similar for both monkeys and what we thought but could not test yet that this may be related to what Hadzopoulos was showing and others with these propagating waves in motor cortex yeah because we know I showed you yesterday a slide we knew that or we know that significant patterns seem to ride on a face of an oscillation and so this may be the reflection of spikes riding on this of this wave of the oscillations sorry I forgot to tell three milliseconds no I the idea was I think I had a figure which was so showing this this hypothesis sorry I forgot to put it here um no the idea the idea would rather be this that if you have waves going like this ah sorry if you have waves going in this direction on the LFB that we see so to say spikes riding together on this that they are synchronous on on a so to say a high peak of this oscillation that's so to say the guests but let me go a little bit further we were interested this was a question before we were interested whether a pattern is specific to the behavior is a if if there were groups of neurons um particularly processing a certain behavior we would expect that they exhibit these patterns which are specific to behavior that's the philosophy and so we made um we calculated the specificity and it gets a value of one if a pattern occurred only in one epoch and in a particular behavioral condition or if they are distributed in different epochs behavioral epochs we give a specificity of zero and this is the analysis of it again the different epochs um the number uh the probability the the specificity of a particular of the particular patterns and for example this is pretty clear that that these neurons seem to be very specific to the precision grip uh to the grip type not so much on the um on the force level this is why when we look at the combination of of uh grip type and force type that this is going lower because this is reducing the specificity but during um during the uh the trial type we noticed that they are highly specific in particular in the later part of the experiment so it seems the different assemblies are active for different behavior okay that's our conclusion um however you could also ask well some of these patterns were overlapping quite strongly yeah for example there were comments by common neurons in these patterns um and we applied a kind of a cluster uh on cluster those and then looked at their specificity yeah so we were asking is this actually a bigger pattern do they belong to the same assembly but are different expressions and when you do this you notice that the specificity is going down considerably meaning uh these uh they are not coming from the same assemblies that's our conclusion here and why is the patterns size approximately only two for synchronous patterns well um we assumed a sunfire chain one sunfire chain in the volume below the array which are about how many one million neurons and sampled about one hundred neurons randomly from there in as these were the situation when we um when we sampled with a uter array and we assumed um we assumed a sunfire chain one layer of a sample from synchronous activity one layer of a sunfire chain so we assume that there are different that there is a sunfire chain and if you find synchronous activity you would expect that you sample them from one layer yeah uh or one group and um we assumed to have that many and um and uh that many per group so the then you can calculate that the probability of sampling three neurons from one group versus two uh is 300 times smaller so the probability to to randomly detect uh from such a group two neurons is much higher than uh larger ones on the other hand uh the statistics of spade is relatively conservative so um and another point is that um I wanted to do not want to express this so strongly but and I could give another talk on artifacts um and uh we don't know where they actually come from but what we did and I don't tend to have a figure now for this but I noticed um at some point different strange phenomena in the correlation analysis and then I asked my people to go back and look at the original time resolution of a 30th of a millisecond and make a population histogram for that and then you see high peaks in there but this can't be neuronal spikes have a duration of one millisecond at least so what we did is we could not find out what is happening on the setup and could not remove them so to say in terms of the setup probably it's some crosstalk but what we did is we first form did this um population histogram and removed all spikes which are in a synchrony of more than two of two and more in case to remove to remove these artifacts yeah and this may have contributed also to low complexity because we may have also thrown away good spikes but I felt better like this okay where are we okay I wanted to let you know but don't want to go too much into this but I wanted to let you know that we also extended the spade method to um two spatial temporal patterns meaning that you cannot only detect synchronous patterns but also patterns with a certain delay between the individual contributing spikes um I hesitated for a long time to do this because the problem explodes but we somehow after having developed spade we had some somehow the mindset changed and uh we were actually able to extend spade to this I just want to give you again I should have brought this figure before much earlier um so why would I expect to get spatial temporal patterns if you think that um at that uh cell assembly is somehow organized in this as a synthfire chain so Moshe Abelis uh invented this concept 89 uh 1982 and 91 there are two books which we have here as well the idea is the following given the connectivity we find in anatomy of the cortex you have as I said you have a high divergence from individual neurons to other ones and each neuron is receiving a lot of convergence from different neurons and if you now do this for groups of neurons in different layers you can define so to say groups with um strong divergence convergence between them and it was known and shown in in studies by Moshe and also others that if you stimulate the first group for example synchronously that this very stably propagates as synchronous activity through this network very stable um a synthfire chain is a very so to say a very strict model of that kind you can relax this also a little bit some people call that synthfire braid which is you don't require to have all these neurons firing synchronously to that uh that a receiving neuron is getting synchronous input but they could be sent out at different points in time but have compensating run times so you could assume different delays between the neurons which are then compensated by different initiation of the spike but still arrive synchronously at a receiving neuron and this concept was for example put forward by bean in stock or much more known unfortunately and he also didn't cite bean in stock by it's a giveage um they are of course you would also find spatial temporal pattern and preferably spatial temporal pattern because you have a delayed distribution here and you would not see any synchrony in that sense anymore yeah okay and um we were lucky um that we were able to extend uh the spade analysis to also detect spatial temporal patterns this took us about three papers that we realized that the approach we first took is actually equivalent to frequent item set mining I don't explain all this now but formal concept analysis in our application is basically equivalent to frequent item set mining and so we we could use also our whole machinery more or less identically but just feed in the data differently so instead of looking for synchronous events we first search for for some patterns and then reformat um these data by concatenating these of one window search window together and then we are back to search for synchronous events that's the rough idea and um however what we had to do in order to detect um spatial temporal patterns correctly um is that we had to extend this spade matrix to a 3D version because um patterns of different durations temporal durations have different null distributions and so when we when we pooled patterns of um let's say the same signature like three spikes occurring five times however of different durations this led to uh false positives but we did this we inserted this and um then uh we for when we only used the 2D spectrum we had problems of finding really these long patterns because they are more rare for a certain allowed duration than shorter ones and of course you have to build this in your um um your you have to consider this and then when we adjust to a 3D spectrum which is then number of spikes um their occurrence and the duration of the pattern if we then have these cubos and test for the individual um pick what is this now uh huh voxels then we exactly find back the patterns what we inserted i just wanted to let you notice that this extension exists i can already tell you that for the data challenge you probably don't need that but you can of course play with that um we also applied this in the same way to the data and indeed we found spatial temporal patterns you see here the result for one session of 15 minutes and we find the same phenomena that we find most patterns during the movement period here is their uh their complexity or the number of neurons involved they are now going up to six or so um um but a phenomenon and and i want to show you here one of these patterns which i like very much to see so this is for example a pattern one two three four of composed of five neurons and these patterns occurrences are aligned here on the first spike this is why they are on top of each other and then you see the second and the third and the fourth and so on and they have some temporal chitter because we allowed for three millisecond chitter of of such a spike in the pattern um i'm not sure if i understand so we apply for the search a window which moves it attaches to a spike and and then extracts whatever there is yeah then we do the the statistical analysis as before and the patterns we find are within the temporal precision identical neuron one is firing at a certain time so as i showed you here for an example oops so neuron one is firing then neuron two within the certain temporal chitter and so on yeah they repeat identically and they and what what you see here as colors is showing different trials because we wanted to see are they distributed more or less homogeneously across the trials they do yeah so this is one type of pattern and in this particular data set we even found we found 39 different patterns two of two spikes fifteen of three eleven of four eight of five and four of six these are occurring during a particular uh epoch in a particular behavior yes no i didn't i didn't but this method is available freely available open source an elephant so people could do this um i am a bit hesitating to apply to look at hippocampus data because i i and i'm in general hesitating of downloading data and applying it because i know you need to know much more about the data than just throwing a method onto it but i agree people found their these patterns and also are different um also slowing down and this type of things it would be this method would be extremely well suited for such analysis yeah one more thing i wanted to show you a puzzle we did not solve yet um now in for these patterns for all the patterns here in total were distributed uh or here on 10 neurons so only a small fraction of neurons actually contributed to this to these patterns and even more surprising and i really don't know yet what this means that this is now a plot like the unitary events this is one um one epoch aligned on in this case which release one neuron across multiple trials and this for the different 10 neurons and we inserted a square around the spikes which are involved in a particular pattern at a particular time so each color means a different pattern yeah so here for example you see there are only pinkish squares there somewhere there needs to be problems for this or here are only red spikes maybe with this one maybe another one but these many of these spikes here have multiple squares uh meaning an individual spike must be involved in different patterns and this is still uh kind of confusing for me and we need to find out what this actually means maybe we are actually dealing with much larger patterns and maybe we need to merge these patterns on top of each other and see maybe clearer an expression of sinfire or whatever assembly activity yeah um in principle i agree but um so this what however what i kind of had in mind is well i have an assembly and then this set of neurons are is composing the assembly and so i would see only patterns from those and then another set of neurons which is somehow i intuitively thought they need to be um separate yeah but obviously it more happens what i thought is that um they seem to multiplex just a second go ahead well you could try to and this is what people do reduce calcium imaging to kind of point processes time scale is different you need to be aware of but i think it should be worth to apply it there are more questions so what what's your point i think so which paper is it interesting okay that's interesting paulina i think i said this didn't i ah okay yeah thanks yeah they are both phenomena that individual neurons are for example this one this has not this has many different color spikes this seems to like like a kind of a half neuron being involved in different patterns but not the individual spikes but this and this has spikes involved in different patterns yeah i i i agree but this all does not i mean if you come i'm i'm long enough in this field to have experience this fight between rate and time temporal coding yeah and people the rate fill rate people are still very strong and this would not fit into their philosophy at all yeah yes this is what we think is happening and this is what we are going to do next to to merge them and to see whether these are actually composing a bigger pattern yeah can we go on a little bit i want to show you one more method can you still concentrate it's hard for me but okay then um yeah it's a good question we don't know yet it's a very good question because somehow you lose this perspective here because you need to um but yes you're right that's a good question whether there are neurons which are always sort of say starting a pattern or yeah yeah something like this okay so i i realize you are tired i want to then i show you just a picture that you get the idea what we also have in our package and um which you may have applications for and i will not explain this in detail to you um so we were having i have always in mind the sinfire model and if you were able to record all neurons from the sinfire model then you would probably see something like this so different neurons time and if such a sinfire chain like this is active and you record many neurons per group then you would probably see sequences of synchronous activity so here assuming you have a a more complete sampling yeah and if the sinfire train is activated again you would see this this activation of of these groups of neurons one after the other again and um um this is actually a simpler task in a way as looking for spatial temporal pattern spatial temporal pattern would be so to say a sparse sampling of this but here you have better chances to detect this and um a simple approach which we started to develop already 2008 um is that what we could do is the following really easy to understand um we take this time axis and again bend it in the potential width of these synchronous groups let's say three milliseconds and now we compare any any of these the contents of these bins whether we have overlapping neuron IDs yeah and in particular if i would have a bin here and a bin here comparing then the overlap of the active neurons is pretty high right and what you can do is insert this neuron overlap in matrix which compares any two times bins uh bins in time yeah so for example um um for example this here would be the comparison of this bin at t2 and this bin at t1 and we would find a whole uh strong overlap and if we had such a feature that we have propagating synchronous activity then also the next bin keeping these uh detecting these neurons and these neurons neurons would uh have a high overlap resulting effectively in diagonal structures within such a matrix with a high entry yeah that's the old this that's the idea behind and what we did in in this method um published in which we called asset um uh published as torret al is basically al is basically finding a way to automatically identify such diagonal structures in in such matrices this involves the number of steps but i um i this is just the idea i wanted to give you and you are welcome to apply this method as well but i will now skip the details okay um um discussion i showed you different ways of approaching such massively parallel data what one was the simplest using a population histogram looking at the complexity distribution and compare this distribution to the one of independent data it's easy to compute and it provides at least an indication of a presence of correlation uh a method i didn't show to you is cubic um which is um which can infer a lower bound of higher order correlations it just tells you seven then you know you have some correlation of at least seven then uh i showed you um uh the the method spade where we can detect higher order synchrony or spatial temporal patterns where you have the neuron ids resolved which are participa which are these patterns and also their time of occurrence and i hint hinted to you to the method uh for detection of sequences of synchronous events by asset for which holds the more neurons you observe the better is the detection and this is more or less what i wanted to tell you here is um um a kind of a review paper where you can so to say look at the comparison of the different methods and here i want to thank the people of my group thank you very much