 focus rather than the applications that we saw in the previous talk, more on the science side. I'll start by quickly describing the differences between neurocomputing and neuromorphic, but some of these differences were already nicely described by my previous speakers. Roman was saying how the field of neurocomputing dates back to the early 40s with McCulloch and Pitts and even to the early 30s. Neuromorphic computing is a relatively younger field. Mark was mentioning Carver Mead who coined the term neuromorphic in 1992 in a paper in the Proceedings of IEEE to refer to hardware implementations of neurocomputing algorithms. And it's a field that started in the early 90s, maybe late 80s, but now became very, very popular. And you can see that because there is a wide range of different neuromorphic computing architectures that are being now proposed in the community to implement different types of algorithms and to follow different types of approaches. So you can go from, we've seen also in this morning, super computing algorithms implemented on standards, computing architectures, or implemented on generic computers or graphical processing units like we saw from the celebral model with Tadashi. We've heard about Spinnaker, which is the sort of dedicated computing platform assembled using ARM processors. So standard microprocessors being done in Manchester with under the leadership of Steve Furber. Another example is high precision, but also power hungry analog circuits to simulate large scale neural networks being developed in Heidelberg under the leadership of Cahindt-Smeyer. We've heard about the IBM True North. IBM is also interested in pursuing more for practical applications, neuromorphic computing approaches, and they have a very nice, remarkable, very low power chip that is fully digital. It's not analog. And it's using integrated and fire neurons and asynchronous digital circuits. We've also heard from Mark the approach that Stanford is taking through Neurogrid. It's again a million neuron system implemented with analog circuits on a 16 chip printed circuit board. And I'm going to tell you more about smaller scale systems like we saw yesterday. It's interesting to focus also on small scale systems, not only large scale systems that are being integrated on single chips, but that also try to implement or emulate in hardware more details of the neural processes that we are interested in understanding. And the Institute of Zurich, of Neuroinformatics in Zurich, INI, is not the only one pursuing this approach. There's other groups around the world here in Australia, the Marx Institute in Italy, the National Health Institute in Germany, the CITEC in Bielefeld, and elsewhere in the world, just to give you an overview. So the neuromorphic term actually is being used for a wide range of different hardware implementations which have different application domains. So if we want to sort of keep with this classification, you can think of neuromorphic dedicated platforms for practical applications, for example, like the IBM chip. You can use neuromorphic computing platforms as a tool for neuroscientists to simulate your neural networks, for example, to implement Nestor neuron on dedicated computing platforms. And this would be sort of simulation or the approach that Carver Mead proposed in the early 90s and that I will try to show you also and that we follow in Zurich, is using hardware to directly emulate the physics of the neural systems that you're interested in understanding. This approach is the one that is following this idea of understanding by building. It's also what Dick Feynman, also at Caltech at the time, wrote on his board, what I cannot create, I do not understand. The idea is that we're not trying to build a simulation platform. We're really trying to use technology, VLSI technology, microelectronics and silicon, as an extra tool to understand the principles of computation in the brain. You can think of this as a tool in the toolset of a neuroscientist. So you can do software simulations, you can do electrophysiology, you can do imaging, but you can also build electrical systems that reproduce the physics of the real systems in order to understand the underlying principles of computation. And the important point is that if you follow this approach, it's important to use components that reproduce the physics of the processes being modeled directly. What I mean by that is that when we use electronics, we have to use them in a way that directly emulates the processes of neural systems. So if we look at transistors, this is a symbol of a transistor, sort of the basic fundamental unit in all of the chips that we make. Instead of using them as a digital fully on or fully off device, like you would do in digital, we use them in analog. Not only we use them in analog, changing smoothly the input and measuring the output smoothly, we use them in a region which is called subthreshold with very small voltages and very small currents. You know, hundreds of millivolts and femto, picot, nanoamperes. In this region, you see we have a semi-log plot. We have, that means that if we change linearly the input, we get an exponential output current. The reason that we get an exponential output current is because the mechanism of carrier transport is diffusion. This is exactly the same physics that is underlying the transport of ions across membrane channels. In fact, if you look at what Hodgkin and Huxley were measuring in the 50s, when they were trying to characterize the sodium and the potassium conductances, you get exactly the same plots. You get voltage differences on the x-axis, semi-log plot on the y-axis, and you get a line which saturates. You get exponentials. You have Boltzmann statistics and diffusion, both whether you're using electrons to flow through transistor channels, or you're using ions to flow through proteic channels. And now you can imagine, this was the insight of Carver Mead. If you had these analogies at the microscopic level with single channels, once you integrate many, many of these transistors together, you will observe analogies also at the macroscopic level. We'll have exponentials, logarithms, thresholding, all of the type of non-linearities and behaviors and dynamics that we observe in the neural systems by using electronics in this domain here. And this is something really particular that only a few groups in the world are pursuing. Like, as I said, in Germany, Italy, Switzerland, some groups in the States. So the idea is we use the transistors in this way. We put many and many of them in a chip because we can use very large-scale integration and we build neural networks. That's the principle here. So the transistors are being used to implement synopses and neurons. I'll show you examples of that. We can easily implement all sorts of dynamics and feedback mechanisms and plasticity. I'll show you those also. And integrate many of these components on chip. In the end, we end up with a silicon device, a microelectronic device that has massively parallel arrays of non-linear circuits that are all computing in parallel. These arrays are distributed by design. They're doing distributed computation. It was mentioned because these are analog circuits, once you fabricate them, even though the layouts, the design is the same for everyone and you just copy-paste like you see here, when you fabricate them, they will have variability. They're inhomogeneous because of device mismatch. And so these computational structures are distributed but also highly imprecise and variable. Classical engineers see this as a bug. Neuroscientists or people that want to reproduce or emulate neural processes see this as a feature because neurons are the most diverse type of cells that you have. And also the noise is an important factor. Noise is being now used as a feature for doing, for example, stochastic resonance or improving the type of computation that can be done in these types of neural networks. And also because we have electronics, we get noise for free. Instead of trying to fight it and minimize it, we actually try to exploit it for computation. Again, by design, these chips are fault tolerant. If some transistors fail, the synopsis will just stop working but because everything is working in parallel, it's not going to compromise the operation of the whole chip. Maybe performance will degrade but the chip will still be functional. This is different from what you have in your microprocessor on your laptop. If one single transistor is broken, you have to throw away the whole chip. So from the point of view of technology, it's a promising approach that might be useful also for improving technology in standard computing platforms. And again to emphasize, computation is analog. We're using microvolts and nanoamps. So we have really nanowatts. Extremely low power and extremely parallel. Once we go off chip, we exploit the best of both worlds. We can use digital, we're not against it. So what we do is we encode the address of the neuron that produced the action potential with a digital word and we put that on a digital bus. And then you can have a chip in which you have analog computation inside and then digital communication outside. And then you can connect multiple chips and take advantage of all of the digital infrastructure that the IT industry has developed in these, tens and hundreds of years. What is also important to notice is that we are, as Roman was saying, it's more and more important to realize that synapses are not just multipliers. They have important nonlinear functionalities and dynamics. And we can reproduce these dynamics in silicon really easily. So these chips are really massively parallel arrays of nonlinear dynamical circuits that have realistic synaptic and neural dynamics. And this is a really a big paradigm shift in computing. Computing in standard technologies is basically done following the Turing machine approach for Neumann architectures where you have memory outside and then computation in the central processing unit. In the chips that I just showed you, we have memory and computation co-localized. And again, we do have time. Time is not discrete. In these chips, time is intrinsic. I just took this word while Roman was talking and put it in the subtitle here. Time represents itself. This is what Carver Mead used to say. So we have continuous dynamics. We're not discretizing and everything is real time. So if you want to emulate one millisecond of neural activity, you have to wait for one millisecond and see what's going on in your neural electronic circuits. It's important to try to reproduce biological plausible time constants if we want to interact, if we want to build neural systems that interact with the environment. And the reason is that if we want to interact in real time, for example, for gesture recognition, we have to have time constants in our systems that are well matched to the time constants of the signals that we want to process. And if it's natural events like speech or motion, these are tens of milliseconds. It's not gigahertz like you have in laptops. It's really hertz or fractions of hertz. And this is something difficult to achieve with silicon. Silicon wants to go fast, so we have to slow it down. And again, it's something that not everybody is doing. Also people doing analog circuits, for example, in Heidelberg are not using real time. They're using accelerated time because it's really a difficult task. But if we use sub-threshold transistors like Carver Mead was saying, we can achieve these 10 millisecond or 100 millisecond long time constants. And this is just to show you that just by putting together, is it five, six transistors in a capacitor, you can really reproduce, emulate the physics of real synapses. Here you have some data taken from a chip that has this circuit. And you see that you have these decaying exponentials which are a really good match to the decaying exponentials measured from real synapses. So you have excitatory post-synaptic currents produced by electronics, and you have excitatory post-synaptic currents measured in real synapses. And you see you have exactly the same type of dynamics. You can change parameters on the circuit, for example, which determine the efficacy of the synapse if you change this voltage here, and then you get different type of outputs depending on the parameters. You can even derive the transfer function of the circuit. It's a circuit which is so simple that you can write down all of the equations of the transistors, you know, do Kirchhoff, and then derive the transfer function. And you do get a first-order differential equation, which is a really good match for synaptic dynamics. It's a really good model. And this is for one part of the synapse. You can think of this as a building block. You'll see you have a differential pair with two transistors here that are pulling current in one way and pushing current in the other. And you can put these building blocks together to build more complex structures, such as neurons, conductance-based neurons. So this would be like a multi-compartmental model where you have a leak conductance here, you have after hyper-pororizing conductance here, and then you have other circuits that implement, for example, sodium activation and sodium inactivation conductances or potassium conductances. This is just, again, using the analogy that Carver was saying that the single transistor represents a channel, or maybe a population of channels. It's nice to notice that by trying to optimize, minimize power consumption, minimize size, in designing these circuits, we ended up, the community, ended up with a design that is a two-variable system, which is actually identical to the adaptive exponential integrative fire model that Roman and Wolfram Gerstner proposed in 2005. Here we have a first-order differential equation with a non-linearity. This is exponential with a positive exponent. And this is a slow variable that is governing the adaptation current, which is this after hyper-pororizing current here. And if you take data from the chip and if you fit it with these equations, you see you have a really good fit with both negative exponent exponentials and positive exponent exponentials. This is really hard to get if you just have arbitrary fitting functions. You can put, just inject constant current into the neuron, measure what happens after it goes into the adapted state because the chip is subject to thermal fluctuation, you get Poisson statistics. It's a fully deterministic circuit, but it's a real physical system. You get stochastic behavior out of it. And you can change parameters. You also have parameters here, like the leak conductance, the adaptation rate, and so on. And by changing these parameters, you can obtain different behaviors. This is an example of spike frequency adaptation and bursting that was obtained just by having constant current injection and finding the right parameters for this particular set of data. So we have now circuits that we can use to implement synopsis, circuits that we can use to implement neurons. We can really build large neural networks, but the interesting thing that's in the title of the stock is plasticity. How can we put together these circuits and include plasticity to reproduce interesting computational properties? So I'm going to quickly go over the spike-based learning idea and framework. One of the most popular models is spike timing dependent plasticity, STDP. It was sort of shown in the late 1997 by Pooh and Markham independently. You can observe spike timing dependent plasticity in many different animal models, many different areas of the brain. And the classical curve is shown here. If you have causal relationship, pre-synaptic spikes produce post-synaptic outputs. You tend to increase weights, and if you have anti-causal relationships, you tend to decrease weights. This is the classical curve, but just the word of caution is that you can observe everything and the contrary of everything if you look at biological experiments. And in fact, if you go, for example, in electric fish, you can have anti-STDP. If you look at GABA-ergic neurons in hippocampus, the same culture that was used for this data, you have pure hebbian type of plasticity. And if you look at stellate cells in your cortex, you have anti-hebbian plasticity, just as a word of caution. But the idea is that you can implement learning just by using spike timing. And this was a very popular idea. Many, many models were proposed. And of course, there was also controversy saying this is not necessarily the most powerful algorithm that can explain the data measured in biology. Maybe it's an epiphenomenon and there are more elaborate algorithms that can actually explain this data and other data as well. This is what we followed. We followed the theories of Gerstner, Fussi, and many other computational neuroscientists that are saying spike timing is not the only thing. If you want to really understand and reproduce the data, you also have to look at other factors, such as, for example, the state of the memory potential of the postsynaptic neuron when you receive a presynaptic spike. And we were very happy to see these theories come up because they are really nice for VLSI implementation. It's really easy and efficient to implement these theories that come from theoretical physicists and computational neuroscientists in silicon because it doesn't require you to have very elaborate circuits. For example, the theory says you don't need to have 32-bit floating point precision for a synapse. It's enough to have two states, either an LTP state or an LTD state, on long time scales. In silicon, this is really nice because then you just need a flip-flop. You need a couple of transistors to achieve this. The other recipe is that, okay, if you have bisability on long time scales, you also need redundancy. You need many, many synapses to encode patterns, to encode variables. And this is easy to achieve in silicon because we have very large-scale integration. We can put many, many transistors on a chip. And finally, another important ingredient for plasticity is the stochasticity. If you have many, many synapses, it's important that you select a random subset of them when you want to perform LTP. Otherwise, all of the synapses just behave as one big single synapse and it wouldn't be very useful. And the stochastic mechanism that we use to do this random selection of subsets of synapses is the spike timing. Because we have Poisson spike trains, and if we represent variables with mean rates, for example, a variable is a three hertz number, every time we produce a three hertz Poisson spike train, this precise spike timing is different and it's random. And this is enough to allow us to have stochasticity in the learning to get this to work. And we can, as you saw the circuits for synapses and neurons with just a small number of transistors, five to 10, we can implement these spike-based learning algorithms. And then we can integrate them on large scale on chips. We've been doing this for many years since the early 90s, first at Caltech and then in Zurich. And so we've been doing many, many types of chips. The latest chip that we've fabricated is essentially a big memory chip in which the memory elements are synapses. They're these complex nonlinear elements that have biologically plausible dynamics and that have spike-based plasticity algorithms implemented in them. So it's a chip that has 64K, actually 128K synapses and 256 neurons. We don't have these 10 to the 11 neurons that people want to reproduce in hardware. We only have 256 neurons. The idea is that we're only looking for principles at the moment. And when we want to explore principles and understand the principles of computation, it's not necessary to have large scale simulations. These have been sufficient for us to do all sorts of very interesting projects, these numbers, hundreds of neurons and maybe thousands of neurons. I just want to demonstrate some examples of plasticity with you. So the first thing that one can try to do is to implement the most trivial type of learning which is the perceptron-based learning. In perceptron-based learning, basically you have, sorry, you have a single neuron which is receiving a big input from all of the big synaptic matrix field. And the spike-timing-based plasticity algorithms that we implemented actually reproduce Hebbian type of learning as well. So what we do is we initialize all of the synapses to have a low weight. Remember, these are bistable synapses. In the end, they're either high or low. And then we present Poisson spike trains that represent some pattern. So here it could be IBM, INI, or anything else. White pixels are Poisson spike trains of say 50 hertz or maybe 100 hertz and black pixels are Poisson spike trains of five hertz. When we present this to the input, these are converted into spikes. They go to the synapses. The synapses produce currents. The currents are integrated into the neuron. The neuron eventually fires. And then by looking at the timing of the pre-synaptic spike and the membrane potential of the post-synaptic neuron, the synapses decide whether they potentiate or depress. And because we were using Poisson spike trains, only a random subset of synapses potentiates. And if we do this, if we present a stimulus for 50 milliseconds, a small random subset potentiates. If we keep on presenting for another 50 milliseconds and more and more and more, more and more synapses potentiate. It's important though that some synapses do not potentiate because then we have resources left to learn different types of patterns. For example, if we learn IBM, we don't overwrite the INI pattern. So we have a neuron that is able to be tuned to different classes of patterns without using all of its synaptic resources. This is the Peri-Stimulus time histogram of the neuron before training the neuron, actually we reproduce this for all 256 neurons and before training the all neurons had a very low firing rate. And after training, they had a high firing rate. You see because of the mismatch I told you the variability, there's a huge variability also in the output of these neurons. This is after a few milliseconds, maybe 50 or 100 milliseconds of training. If you keep on presenting the same stimulus though, the histogram changes and more and more neurons get potentiated more, more synapses so the neurons become more strongly tuned. So you go from this situation to this one to this one by keeping on presenting input patterns. So this perceptron idea works really nicely. If we try then to do more elaborate type of benchmarks like we've heard about the handwritten digit recognition benchmark, then you'll see that this perceptron works but it's nowhere near close the state of the art in machine learning. So what we tried is we tried again to represent numbers with Poisson spike trains. We presented them to a neuron and then we trained the neuron to recognize three against all other digits. And the classification results, the accuracy, the performance was around maybe 50%, you know, just to give you an idea the state of the art machine learning groups fight over small percentages in 99.98 and 99.97%. So 50% is laughable. No one would ever even consider this type of technology in these type of benchmarks. But neuroscience and machine learning comes to the rescue because there are some theories from the mid 90s that show you, that show that if you have enough weak classifiers or crappy classifiers like the one that we have here and you put them together, you can improve by using ensemble learning techniques or boosting techniques, you can improve the classification performance just by combining many, many classifiers. And so this is what we did. We just, you know, we have 256 neurons in this chip instead of using one, we started using more and more tens and 20s and 50s. And we saw that the hardware follows the theory. The theory says that the error goes down with the number of elements or classifiers exponentially. This is really powerful. And so this is just to tell you that this technology, even though these proof of concepts, perceptron ideas are laughable when you look at a single chip, at a single neuron, you can actually achieve state-of-the-art performance using more neurons. And you can do this using one chip that burns a few microwatts and performs in real time. You know, in 50 milliseconds, you have your classification result. The 99.98 accuracy performance that machine learning algorithms achieve, they do that using GPU farms, which burn kilowatts, if not megawatts, and take up a room that's as big as this room here. So this is the difference. We're getting closer to, you know, the technology that is in our head by using these sub-threshold transistors. Another type of learning is, this was supervised learning because we were telling the neuron, you know, whenever you see three fire and when you see four, five, six, don't fire, we were giving it a teacher signal. Another form of learning is that you can do is by using Hopfield-type networks. We connected all of the neurons to all of the synapses. So we have a fully connected network in the chip. And then we just inject current into a subset of these neurons and we see what happens. It's just unsupervised. And you can imagine if you inject, for example, current into the bottom four neurons of the chip, you will have activity at the output here. Because of the recurrent connections, you have activity in all of the synapses of all of the neurons. But the synapses that receive input, presynaptic inputs, and have no post-synaptic outputs, they will tend to depress, just have been learning, or STDP. The synapses that receive presynaptic input and also have post-synaptic output, there is correlation, neurons that fire together, wire together, so they will tend to potentiate. It's just classical, happy, and learning in Hopfield networks, but we can do this with sub-threshold, non-linear dynamics in these chips. And you can stimulate other sets of synapses, and you will get other sets of synapses potentiated just with unsupervised learning. This is the data from the chip that demonstrates this. We have the synaptic matrix here. Black means low state, white means high state. We stimulated the bottom 64 neurons for one second, and the synapses started to increase randomly, and the other synapses started to decrease randomly. We repeated the same stimulation again for another second by choosing different populations of synapses. So this population of synapses could represent your mother's face, this one your father's face, and so on, they just represent some patterns. And as you keep on presenting over and over the same input patterns, the synaptic matrix starts to get its shape. Until finally, the synaptic matrix is strong enough that you have sustained activity. This is the property of the attractor network. Even after you take your stimulus away after one second, the attractor stays active for as long as you don't inhibit it. And here, when we stimulate different patterns, we inhibit the others. So we keep the attractors in its on state until we stimulate something else. And here, for example, in the last simulation, we didn't inhibit anybody else, so you see the attractor just stays on. This is classical Hopfield theory, but again, we achieve this with microwatts on these neurons. And these synapses, they have this stop learning mechanism which has been demonstrated by theoretical physicists to have maximum memory capacity in these Hopfield networks if you're interested, all the details are in these papers. So once we have attractors, we can use these chips to look at neuroscience. Again, the idea is that we have this as a tool in your neuroscience tool set. And you can try to reproduce or understand perceptible by stability, decision-making, working memory. So for example, we configure the chip to have two attractors, A and B, which were mutually inhibiting each other, and then provided some input. You can see when you provide input to the population A, if you look at the mean firing rate, you have a high activity. When you take the input away, you still have some sustained activity. It's slightly lower, but it's still there. As soon as you stimulate another attractor through the inhibition, it will kill the first one. You will have high firing rates during the presentation of the stimulus and lower sustained activity after you take the stimulus away. And you can repeat this on and on by having switching between these two percepts just by stimulating the different populations. Or you can kill everything just by having an inhibitory input. You can look at the dynamics. You can reproduce the gamma type. In fact, if you have constant current injection in both populations, you will have autonomous switching between the two populations. And this is what happens by just injecting the same amount of input both to A and B. One attractor will stay active until for finite size effect it will be switched off. The other attractor will become active, and so on. You can look at the raster plot coming out of the chip, or you can look at the mean firing rates just by doing the population averaging. And the switching dynamics really reproduces the type of dynamics that is measured in monkeys when they're looking at perceptual by stability. So to make a long story short, we have the possibility of implementing, of directly emulating the biophysics of real neurons in silicon by using these sort of unconventional domains, a subthreshold domain where we bias transistors in this domain. And now I showed you a learning chip where we have plasticity in the sinopsis. We recently implemented a multicore chip where instead of having a single chip, we have a core. And within the same chip, we have multiple cores. And then we can send spikes from one core to the other using the digital framework that I told you about. So inside the cores, we have analog computation. And then not outside the chip, but outside the cores, we have digital communication. And we're in this, you know, having connecting all these neurons to among each other would require a huge amount of memory unless you make some assumptions. And if we assume that we have clustered connectivity, like the type of connectivity that is measured in the cortex, then we can minimize the memory requirements and we can really have very efficient implementations. So for example, if you have a multicore and multi-chip system with just a few kilobytes of memory, not the petabytes that we heard about, you can really have networks in which one neuron can fan out to other domains like thousands of neurons in another core or other thousands of neurons in other cores. The only assumption we are making here is that we have clusters. We don't have all-to-all connectivity everywhere, but we can fan out to large populations in this clustered way. The chip that implements this was, we just received it back. It's again using a quite old process, 0.18 micron or 118 nanometer. But it allows us, because we have very large scale, we have very small circuits, to have 1,000 neurons per chip. And this could be one core. So these are actually four cores, but then you can integrate thousands of cores on a single chip and you can really have millions of neurons. Again, the goal is not to have very large scale systems. The goal is to have relatively small scale. We have thousands here, which allow us to explore different type of computational paradigms. As I told you, attractors, working memory models, learning perceptron models and spike time independent plasticity models. So in conclusion, the idea is that we can try to use these devices as an additional tool, in addition or in parallel with our MATLAB simulations and Python simulations and neurophysiology experiments to explore the principles of computation. And the things that we've been looking at recently are context dependent learning and processing with these chips. There are some papers, if you're interested, that show recent results. We've been trying to reproduce properties of working memory and decision making and trying to measure reaction times and see that the reaction times measured in these real time electronic systems match the reaction times of monkey in monkey experiments. And we have results also there. Or we've been trying to implement cognitive systems by using state dependent processing. By connecting many of these attractors together, you can implement sequences or finite state machine like sequences. For example, if you see a queue in the right part of the monitor in an experiment, then you have to suck out right, otherwise you have to suck out left. These type of cognitive experiments that are being made in monkeys can be completely reproduced without using any computer, just using the chips that I told you about. And we've been also trying to look at, because this is technology, not only neuroscience, but also applications. And of course, there is a lot of interest from the various companies Qualcomm, Samsung, IBM, which we are in contact with for autonomous sensory motor systems, embedded systems, these ARM processors on chips, and also for brain machine interfaces. These chips are extremely low power, so they don't heat up. And you can actually have chronic implants where you can try to decode real action potentials using electronic action potentials and really hopefully close the loop and have closed loop systems to augment or implement prosthetic devices. This is not a single work, it's a group effort. And there are many students in my lab that contributed to the work, Channing, Federico, Corradie, Hisham, Laurence, Mark, Jonathan, Dora, and Richard. And also it's something that's been going on for many years, as I said. And the collaborators, the previous students, I'm very proud now that they're professors elsewhere, are still collaborating with us. And they're at Columbia, at Samsung, Bielefeld in Italy, Irvine, England, and Residen. The funding organization that supports this is mainly European, but also National Swiss, Kistera. And of course there's the whole support of the Institute of Neuronformatics in Zurich. Finally the last slide, it's a slide of an advertisement. There is a workshop that we organize, it's called the Capocaccia workshop, Capocaccia Cognitive Neuromorphic Engineering Workshop. Every year we go there, it's in Italy and Sardinia, and we bring chips, robots, neurophysiologists, neuro-scientists, and we try to brainstorm for two weeks. It's really a working group, work group workshop, and you're all invited to look into it and participate. Registration is free, it's basically on a first come, first serve. And it's really an exciting place where computer scientists, mathematicians, physicists, and neuro-scientists get together to try to solve the problems that we are dealing with even here in this room. So thank you for your attention. I should also mention that electrical engineers are starting to become really interested in neuroscience, and this has been shown by the Proceedings of IEEE, not only was the Proceedings of IEEE publishing the first neuromorphic paper in 1992, but lately, if you look at the reviews, they have special sessions on brain-inspired computation. And there is a link that I can put on Twitter, but also here's the code. It's just saying search for neuromorphic in the Proceedings of IEEE, you get all the reviews. It's really interesting to see that companies and engineers are starting to look at the brain to try to understand how to build future generations of computing systems. Okay, thank you for that. Questions? That couldn't be included in the hardware implementation, but you thought it might be important. Yes, there are many aspects that we think are important, but we have to make compromises. Just like in software, when you're deciding if you want to do a large-scale software simulation, maybe you do integrating fire models, single compartment, or multi-compartment. So the chips that I told you about focus on plasticity and learning at the single sign-ups level, and we're using point neurons, but other colleagues like Shichi Liu and my institute and elsewhere in the world are trying to implement multi-compartmental models because they're interested in the spatial processing, the dendritic processing. And so the technology allows you to explore. So in my group, we're focusing on one specific aspect, but the community overall is following multiple approaches where the different ideas just get implemented in silicon. As I said, it's another tool. Instead of using Python or C, you use VLSI technology, and if you have some idea or interest in some specific topic, you can also implement that in electronics using this approach of sub-threshold transistors. Okay, thank you. So I'm wondering, in your view, what's the advantage of a spiking neural network has over artificial neural network? In terms of a computation efficiency or computation power? In terms of computational efficiency, the advantage is that when we want to build large-scale systems, spiking neural networks scale much better than non-spiking ones, at least when we're trying to implement them in analog VLSI. So analog VLSI is very useful because you can get really extremely low power, but if you want to have scalable systems, you need to find a way of communicating analog signals long distances. And it turns out, if you try to optimize for bandwidth and power, you end up finding that the optimal solution is pulse frequency modulation, which is what nature found by using axons and action potentials. So it's also nice to see that by optimization techniques through mathematics and engineering, you end up with the same solution that nature evolved using spikes and axons. Okay, thank you. Another question is, you mentioned in your single trip, it cannot learn to recognize the handwriting digitals and the... It can learn to... Yeah, the first step, you have only 50% success rate. Yeah, one single perceptron has really weak performance or happy performance. I see, so how many neurons in that single perceptron? One perceptron is one neuron. I just want your... But the chip has hundreds of neurons and if you put two chips together, four chips, you have thousands of neurons. So you can increase the accuracy just by using more neurons. Okay, thank you. No more questions. So then I guess we should thank you again and all the speakers of this session. Thank you.