 All right. Thank you, Giorgio. Thank you for inviting me and I'm very happy to be here. I think this is my first neuroinformatics meeting, so I'm really excited to see what's going on in the field. Now, I personally, that's a nice introduction, so I don't have to say any more about that, but I'll tell you some of my goals, my overall goals and how this fits into this meeting. I work with a lot of neuromorphic engineers. They're trying to make computer chips that are having the architecture and dynamics of a brain, and they need tools, they need applications on these chips. So that's one of our big goals in our lab, is to support this effort with tools that I'll talk to you about today. Then my own personal self-interest is making a robot that thinks like you do. One of the promises of these neuromorphic chips is that they will be very low power and small, and I hope on all sorts of embedded devices. So my overall goal or the lab's overall goal is to make like it says, a robot that thinks like you do. So today I'm going to talk about some tools we have developed to get towards that goal. One of the big bottlenecks is parameter tuning, and this has come up yesterday in discussions, and one of the talks yesterday I attended was about parameter estimation, and it's extremely time consuming. If you ever developed a large-scale neural network, spiking or otherwise, there's a lot of manual tweaking, and it's very tedious and it's very prone to error. So as part of our tool suite, we developed an automated framework to do this parameter search, and our simulator framework leverages off-the-shelf GPUs to increase the parallelization and speed things up, and some recent progress in evolutionary algorithms. Then one key guiding principle is using some theoretical work like the efficient coding hypothesis to actually guide this parameter search. So I'm going to talk to you about that framework, or that parameter tuning framework, but it's a part of a larger software framework that we call Carlsim. I should have said at the beginning. The name of our lab is Carl. UC Irvine's University of California Irvine's mascot is the Anteater. It's a long story. There's no Anteater's in Southern California, but it makes for a nice acronym, Cognitive Anteater Robotics Laboratory. So our robots are called Carl, and our simulator is called Carlsim. I guess if you want more information about it, it's freely available off of our website or off of GitHub, but I'm proud to say the last two versions have been nicely documented and provide tutorials. So we hope it's easier to use. It's scalable, extendable, it's very optimized, and it has a whole suite of APIs that make building large-scale spiking networks easier. Now, what we mainly support is the Izakavich Neuron, and all sorts of variants of spike-timing-dependent plasticity. I think that's been discussed over the last day or so, so hopefully I don't have to go over that. But what I'll show you, we're using spike-timing-dependent plasticity plus a homeostatic synaptic scaling rule. So that you might not have heard about. There was a talk by Claudia Klopath about this yesterday. So we can give the network certain firing target rates, so the neuron certain firing target rates, and then we can scale, synaptically scale all the weights up or down if that neuron is not near its target weight. If you just use STDP alone in our experience, you can get some, even though it's supposedly balanced, you can get some really runaway dynamics. So adding this homeostasis, it makes your networks behave better. But there's some interesting theoretical concepts that go with this STDP and combining it with homeostatic, see a homeostatic scaling. All right, so what's the first problem? Here's the task at hand that we're gonna, I'm gonna demonstrate this framework for you. So let's say I give the network like I give if I was studying visual cortex, sinusoidal gratings at different orientations. And my desired output is to have something like the bottom here, which is neurons that are tuned to specific orientations. But you want them broadly tuned with a nice cosine tuning function. So that's the problem. And instead of developing a network that does that, we made a neural network and we explored the parameter space. Now in this case, the network's relatively small, about 4,000 neurons. And what we're gonna just search for is the different parameters for STDP. So if you remember STDP, there's amplitude for two amplitudes, one for LTP, one for LTD, and two time constants for LTP, LTD. And then we're not only exploring STDP for the excitatory synapses, but also separate parameters for the inhibitory synapses. And then there's certain target rates that we wanna explore with the homeostasis, a SNAPTIC scaling rule. And so that gives us a network that we give inputs from the sinusoidal gradings. And then we have a very small pool of excitatory inhibitory neurons that hopefully converge to showing nice cosine tuning curves like simple cells in V1. Now what we're gonna use is evolutionary algorithms. So just to show hands, how many of you know what an evolutionary algorithm is? Lots of good. How many of you used evolutionary algorithms? Wow, all right, great. Well, as you know, you can search parameters if you have a given fitness function, and then you select the best from your population, and then you do some sort of mutation or replication and mutation so that you generate a new population. And there's lots of ways to do this, lots of different parameters to tweak, but you can use standard evolutionary algorithms to do this. And what's different about this is on CarlSim, on the GPU itself, we're going to run multiple, the whole population in parallel. So if we have a 4,000 neuron spiking neural network, we can run anywhere from 15 to 40 neural networks in parallel simultaneously. And do the fitness evaluation simultaneously. And so what that means is really the heavy lifting, all the intensive computation is done in parallel on GPUs. So then if you know evolutionary algorithms, the trick is picking a really good fitness function. So we decide to look to some theoretical ideas called the efficient coding hypothesis. And the idea of that is, brains really want to extract the most information from the environment, from the stimuli, using the least amount of effort. And so they want to span the input space and they want to be able to, if the input space is tight, they want to make sure they want to adapt to that, the input space is broad, they want to adapt to that. So you want to get neurons of fire nicely across the whole input space. And then the other thing is you want not that many neurons responding to each thing. So you want sparseness. And then in addition, you don't want neurons to be too correlated because if they're too correlated, then they're replicating a signal. So they're redundant. So you have to, so the brain, there it goes, is trying to do that all the time with stimulus. So what if we just made a fitness function based on that idea? And that's the fitness function for this model that I'm talking about, which has decorrelation, expanding the stimulus space and a sparseness parameter. And all of these things we're trying to maximize, or in this case, minimize, well actually maximize the way it's went over to actually get the most fit network. So what amazed me is in just a short number of generations, we got very good fitness scores, all right? And so this didn't take long, it's running parallel. So compared to a CPU, we could get up anywhere from 40 to 60 times speed up by running this in parallel. So it really speeds up the parameter search and the number of generations to do this is very short. But remember, we're searching the learning parameters. So we have to have a training period to let the learning happen and have a testing period to see if that learning was good for end fit. And so what you're seeing here is the change over time during training. So all this changing of the weights is due to changing a STDP as it's running along or learning through STDP. So it's not that we're shuffling the weights, it's actually a much smaller parameter space if you think about it. There's on the order of 20 parameters that we're searching, but we have to let it run for several seconds or minutes per stimuli. But like I said, it's done in parallel, so it really speeds things up. So this is a really nice demonstration that showed it's sufficient to use this method or framework to actually make biologically realistic spiking neural networks. And it was very nice that we're using a theoretical construct to actually guide this search. But these are toy problems. So what about a real brain networks? Can this actually scale up to real brain networks? So we are very lucky to have data given to us from Doug Knitz's lab at University of California, San Diego. And he was recording from an area of the brain called the retrosplenial cortex. And retrosplenial cortex is interesting because it has three types of spatial navigation and coding, route-centric, allocentric, and egocentric. And I'll explain what that is in a minute. But if you look down here, the task was having a rat run this W maze back and forth, back and forth. And then they would move the maze to a different position and have the rat run back and forth on this W. Now, if it's egocentric, then neurons fire every time it takes a right turn, for example. Now, if it's allocentric, the one on the right, it fires in specific places, or encodes specific places in the environment, like a place cell, but they don't really look like sharp place cells. But they can still, neurons in retrosplenial can still encode allocentric space. Now, something different is called route-centric. And that is, you might find neurons to fire on the second left turn of a path. And they call that route-centric. So they find in this interesting area that is connected to lots of areas like hippocampus, parietal cortex, and subiculum. They find in this area, these three types of classes of cells that are important for spatial navigation. So they gave us this data set. They recorded from about 250 neurons. And this is work by my student, Emily Rounds, in conjunction with Eric Siggy Scott and Ken DeYoung at George Mason University. And she'll be presenting this in the next few weeks at PPSN, Parallel Problems Solving in Nature, PPSN. I'll go through quickly, just some of the key results. So, Andy Alexander, Dugnitz's student and Dugnitz recorded from all these neurons. And while they're recording from neurons, they're recording certain behavioral metrics. So they're recording the angular velocity of the rat, the linear velocity of the rat, the rat's head direction, and the rat's position in space. So we took those behavioral metrics that they had that were tied to the recordings, and we put it into a network of about 600 neurons they record from 250 neurons, remember? And it had a 10% connectivity. And we did the similar method that I just showed you before. We searched the parameter space of STDP and homeostasis. And the fitness was just, instead of using the efficient coding hypothesis in this case, because we had the data, the fitness was just how close we match or how close we correlate our synthetic neurons to their real neurons. And we could run 15 neural networks in parallel on one GPU so we could really speed up this process. The first thing is we got really good fitness in a really short amount of time. And really good fitness is 5.5 to 0.6 correlation between the two neurons, between any two neurons. So in like 20 to 30 generations, we're getting really tight solutions. And I'll explain towards the end of the talk why I think this is happening. We have an idea now why it's so fast. But what was nice was you see in red our synthetic spiking neural network, spiking neurons in a real neuron that's firing and here's on the track left, right, left, right, left, left, right, right, left, right and so on. And so it's following the nice spiking of these real neurons. And this is just a mean firing rate of the spiking activity from their data set. So we got really good matches on an individual level. So what about at the population data? So we saw all three types of neurons in our data, in our synthetic data. We saw route modulated turn cells. We saw pure turn cells. We saw turn insensitive cells which are probably doing more of an allocentric concoating. And then if you look at the population on a whole, we see that we can do a reconstruction of the path. So let's say the rat runs on this WMAs, a whole number of trials. You can compare the odd trials to the even trials and say, can I predict where this rat is from one population activity to the next? And this is the experimental data and you can see you can predict very nicely where the rat would be from an even trial where it would be on an odd trial. And this is the synthetic data. So we could have a really nice match. So as a whole, these neurons are encoding allocentric space. Now if you take the, compare, do the same analysis, but you take when the track was in position A and the track was in position B and do the same thing, you do not get that. So it's actually specific to that space. And this is the real neurons and this is the synthetic neurons. So very nicely, we see that it's specific and selective to space. All right, so I think just, this evolutionary framework is quite promising. Not only for just different types of models, but also now actually replicating real data. Given a set of behavioral metrics and some spiking or any kind of neurophysiological data, we can actually use this to match and we can do it in a very efficient way. And it's not just matching the specific neurons, it actually seems to be capturing the population dynamics and the behavioral metrics of the real network. So we're very excited about that. And it's interesting that we can use what in evolutionary algorithms called an indirect approach, right? So a lot of people do what's known as a direct approach so you would actually evolve the connectivity of every single way, but that's a huge space. So actually leveraging what STDP and homeostasis is doing indirectly, you can actually get these matches, which is interesting. And then using GPUs in this and in our other framework really helps to speed up some of these simulations. All right, so I was kind of wondering, I was at another meeting just explaining Emily rounds this data and they were saying, well, how does it do this so fast? It doesn't make sense. Usually when you see evolutionary algorithms, you're talking hundreds to thousands of generations before you get a match. And it was odd to me. And then on a separate line of research that just came out in J Neuroscience about a month ago, another student in our lab, Michael Baylor, was looking at an area of visual cortex called MST. And so this is a totally different area or subject in a way, but I'll show you how they link together. So primary visual cortex has strong projections to an area called MT. So area MT has very strong directional selectivity. So cells are directionally selective to motion and also the speed of that motion. And then they have a strong projection to an area called MST. Now MST is kind of a mystery to everyone. It is originally, you know, there's strong evidence that MST is encoding the heading or focus of expansion as you move through space. But some of the receptive fields are kind of crazy looking. It's probably responding to things that are moving in space, in different positions. And then there's some really wild receptive fields that no one was really sure what's going on. So what Michael did in the lab was that, well, he had made a very detailed model using our simulator of this region, MT. But then he sort of took a step back, said, well, I want to do more of a theoretical model to see if I had responses like MT, what would happen in this area? So what he did was he took a bunch of features and a bunch of stimuli. So the features were the MT-like responses. So they were like motion directional tuning and speed tuning. And then the stimuli were just a whole bunch of flow patterns, including eye rotation. So how you move through space while your eyes are moving and eyes are in different positions. And then he used something called non-negative matrix factorization. So a dimensional reduction technique called, I'll keep calling it NMF, non-negative matrix factorization. What's nice about this is it's parts-based and everything's non-negative, so it's positive. So eventually this could actually go into a network if you could actually pull out the weights from this method. So it's similar to PCA and ICA, but it has those nice characteristics. And so the idea is to take the feature space times the stimulus space and then look for a set of basis functions that you could use to reconstruct this input. But you want the basis vectors or basis functions to be much less, much smaller, so you get a sparse encoding that can reconstruct the input. And so that's the idea. And he got receptive fields that look like this. He got, I won't go into, I don't have time to go into detail and look at the paper, but he was able to match a wide range of data using this methodology, which I was really impressed. Anything from MST in the last several years was able to match the receptive fields that came out of this. But what's interesting for our point is it fits in with the efficient coding hypothesis. So it's doing a sparse representation so that if you get an original image like this and you have, in this case, 64 basis vectors, which is much smaller, much reduced, you can reconstruct very accurately the response of this optic flow field. And if you look here, these are the basis vectors that are responding the most. It's hard to match it up, but trust me, they're picking out the big features of this particular stimuli. And so it's a sparse representation of this. And we're able to reconstruct and also predict where the focus of expansion is. And it didn't do just that. It also showed neurons or receptive fields that will be responsive to eye position, like you see in real MST and a whole bunch of other parameters. It turns out that cells in MST are not just doing one thing, they're kind of generalizing, but they're also doing it a very efficient sparse way. And then just playing with a number of basis vectors, you see the error goes down as you increase the number of basis vectors and there's a sweet spot, which is somewhere between 32 and 64 basis vectors. And then there's an increase in sparseness as you have more basis vectors. So it's really nicely chopping this problem up and doing it in a very efficient way. And I always forget to introduce who did the real work. So before I wrap up, let me just introduce the team. So this is my long time collaborator at UC Irvine, nickhill.nick. He's been working on this, Carl, especially the neuromorphic engineering portions of what our lab does. Chris Carlson did some of the original work for the evolutionary framework. Michael Baylor, I just talked about his work on MST into some really nice large scale networks on area MT. And Emily Rounds is the one that did the work with the retrosplenial cortex data. All right, so what do I think is going on? Well, first, I think we need these large scale, there was discussion about this. I think we need these large scale complex simulations to really, at least from my idea, get an understanding of the brain that leads to behavior. And especially for fields like neuromorphic engineering or my other field is called neuro robotics. We need to at least explore these dynamics and figure out what principles to extract from it. Now, what do I think is going on with the data I just showed you? I think the combination of STDP plus homeostasis is doing the equivalent of dimensionality reduction. I didn't really believe this. And then Chris Carlson and then Michael Richard, who was a postdoc in our lab at the time, buried this in when Chris introduced the homeostasis rule, had an appendix of this conference proceeding, a proof that said STDP plus homeostasis is equivalent to non-negative matrix factorization. I didn't give it much thought, but I said, yeah, put that in. But now I'm actually seeing it in practice. So I think that's what's going on. Well, that was what he did in that toy model, but with Emily Round's data, I think that's what's going on. I think that it's doing that dimensionality reduction on the types of inputs you get from the head direction system, the hippocampus system, and the parietal cortex. And you see the same types of neurons that are generalizing in a very sparse way, like you saw with Michael Baylor's work in MST, which was really nice. He used not our method, but he used the actual non-negative matrix factorization. I think if he had done the evolutionary framework, he would see the exact same thing, something we want to try out. So I really think that these principles of efficient coding are not only a way to guide the construction of spiking neural networks, but they're also very informative about what's going on in brains and how it's really trying to extract the most by doing the least amount of work. And that's a good point to stop. So I'll stop there. Thank you for your attention. Things that weren't quite being matched by your network. So for instance, when there was some high activity of the experimental neurons, your network didn't always exactly match it. Also in some of the other predictions, you could see that there was a structured difference. Yeah, so you look on the bottom right, for instance, the experimental, you can see there are big peaks which get missed. So do you think your error metric is really capturing what's important about these neurons? I do. I don't think you're ever gonna get a perfect match. And then why I think it's doing things properly is because the population dynamics match. Yeah, because I would never expect to get an exact match. It's apples and oranges. We're using as a cave it's neurons and they're getting different inputs. I was kind of shocked. And actually Andy Alexander who sent, the student that did all the recording was really shocked. I was like, I always was wondering if this was noise but there's a real structure there. So it's capturing the structure. It's capturing the population dynamics. And then what will hopefully come out very soon is we're trying to understand some other things. So we're doing manipulations that they couldn't have done in the lab or they haven't had a chance to do that we're doing to the network. And the early looking at the early results it's manipulating or changing the network dynamics in an interesting but plausible way. So I'm pretty confident about that. And I'm not sure how you could squeeze out like even a better match to that. So can I just ask, does the brain actually learn the way your simulation does? If you take a naive animal and put it into the maze for the first time, does the development of network dynamics parallel to what you see in your simulation? No, I probably not. I mean, because we're doing something different. So maybe with a simple cell, but that was a toy model that you're seeing sort of a development of orientation specificity. But with the other thing, the rats have already been very well trained. So they've run this maze a bunch of times before they actually hook them up to record. And so we're doing something different. What I like about the method is now we have a way if anyone gives us this data set we can try and replicate it and then we can do experiments in simulation. And then with putting on my robotics hat instead of having to make artificial networks I can actually make networks that are very closely matched to real networks and then do the kind of simulations I like to do with that.