 Ladies and gentlemen, may I have your attention please? So, we are gathered for a momentous occasion. Something that has been 20 years in conception and 15 years in construction. The spinnaker machine. This has been a staple part of the school for as long as I've been here. So this really is a milestone in the history of the school. And for many of us, this really seemed like science fiction when Steve first proposed it all those years ago. So today we have three distinguished speakers for you. The first of which will be Steve who will describe to you what this thing is. The second of which will be Sasha Albada over here who will be describing the neuroscience applications of spinnaker. And finally, we have Mike Denham on the front row here, Professor Mike Denham who will be discussing a spin-out company called MindTrace.ai that is exploiting the intellectual technology in spinnaker. So without further ado, I will be handing over to Professor Colette Fagan who is Vice President for Research at the University of Manchester who will say a few words and introduce our speakers further. Thanks very much Gavin. Welcome everyone. Gavin's just introduced me so I won't repeat that but I'm really delighted to be part of this event to celebrate spinnaker switch on and to share this historical moment. The spinnaker machine, I'm told, is revolutionary and novel computer architecture. It's designed for large-scale neuromorphic simulation and it's capable of simulating 1% of the human brain in real time. I mentioned that to my son this morning and he just went, wow, he's a 16-year-old just starting A-levels in physics and maths. It's also an exemplary of low-energy computing. Now, this pioneering brain-inspired computer platform has been 20 years in conception, 10 years in construction. The ambition of this project was so visionary and far-reaching that many thought it to be akin, as Gavin said, to science fiction at the inception. But it's delivered and it's delivered successfully on its vision ambitions and today reaches its ultimate goal, 1 million microprocessors will be activated at once and working in concert. That is fantastic. Such major scientific discoveries are not possible without creativity, a long-term horizon, determination, hard work and perseverance by a team who've been assembled to bring together the very best and to mobilise their skills and knowledge. And Professor Steve Ferber in our School of Computer Science had led this fantastic and large international project over the past 15 or so years. Now, of course, many others have been part of the team. Around 50 PhD and students and post-doc research assistants and together the team has produced over 50 articles including three which count among the very best world-leading papers and are highlighted in the Faculty's in abstract spotlight on world-leading papers on its web page. Of course, the team needs funding to bring such magnificent ideas to fruition and this has been possible here with long-term and large-scale funding from both EPSERC and the European Union to the tune of about 15 million. And we're delighted that we're joined today in our celebrations with a representative from EPSERC, Dr. Anna Angus-Smith. Now, this project, biologically inspired computing, epitomises the best of what we continually strive for at the University of Manchester. It's ambitious. It transcends disciplinary boundaries. It produces excellent science and creates research training and early career opportunities to build our next generation of scientific leaders and it makes a positive impact and contribution to society including through industrial collaborations. We're going to hear some of that. Artificial intelligence, machine learning and robotics are fast-moving fields no longer the space of science fiction. These technologies have enormous potential to make positive contributions to so many aspects of our lives as well as creating new risks and new ethical questions about the relationship between the human and non-human. But this whole body of activity and imagination and realising that imagination are at the heart of the strategic priorities for research and teaching at Manchester for the foreseeable future. They're central to our focus on industry for and data sciences which priorities in our current recruitment of 100 presidential early career fellows and it's a fundamental part of our recently launched digital futures interdisciplinary and university-wide network of 800 plus and growing academics. I am confident that Steve and the rest of the School of Computer Science will continue to make a significant contribution to our research and teaching in this arena. So I'm now going to hand over to Anna to say a few words and I shall see you all later at the parties and celebrations. Thank you very much for inviting me today. I'm really excited to be here and I know people always say that in a really platitudinous way but I mean it because I actually think it's really unusual that we get together to celebrate science, science that's been transformative over a long period of time and I'm not just saying this because I've seen the plans for the cupcakes with the champagne and the balloons but actually because it is really important that we do that we usually do these events when we're trying to launch something but actually getting together and saying it's really wonderful that we've done this research over a long period of time I find genuinely really exciting. So Gavin sent me a couple of exam questions when he asked me to stand up and talk probably because he knows that I ramble otherwise and the first one was how in the future will UKRI fund this kind of transformative, risky research over the long term? And the first thing I thought about when I looked into how we've supported Spinnaker over to be honest longer than I've worked at EPSRC is that actually we've done so through a number of different kinds of mechanisms both through our own funding and through EU funding and other funders and the important thing about UKRI is that we will not lose those mechanisms to fund transformative, risky research with long term potential. We won't because people like me think that they're important and we think that our job is to make the case to colleagues in the government about why it's really important that we do this long term research. However, UKRI also has a mission to deliver the government's aspiration of investing up to 2.4% of GDP in research innovation and it's doing that through additional opportunities and I started talking to Gavin about those earlier and I think I blew his mind slightly but actually I really would encourage you to look at some of those additional opportunities because they sit alongside this core funding that we've always had and we will continue to have for transformative, risky research and they give really great opportunities to all of you in the room and that links me straight into the second exam question which was talk about AI strategy, Anna. Roughly wasn't it? I think my response to that is it's a really exciting time for all of you who are involved in AI whether it's fundamental research or more applied research that aligns with the broad AI agenda as lots of people who know less about AI see it, frankly. Why is that? Because if you look at the government's industrial strategy white paper AI and the data economy is one of their foreground challenges and what we do when we're trying to bid into the government and when we're thinking about what they might invest in is they look at their own industrial strategy and they try to make aligned investments so we've already seen investment in up to 20 CDTs training students in AI. We'll do interviews on those next week and I truly believe we will continue to see additional investments in AI. I'm not keeping them from you what those opportunities are but you need to be aware that they will arise possibly on short timescales but they will happen and you need to be ready to respond to them. So it is an exciting time but I think you just need to keep an eye on the horizons for opportunities to do things like this. Now I'm quite aware that I'm standing between you and the actual exciting stuff. So I think I should stop rambling hence the two exam questions. Thank you very much and I'll pass back to Claire. Thank you. Give your hands up on the floor to Professor Steve Furber who's going to tell us all about this. So thank you very much for those generous introductions. Today is mainly a party we're celebrating achieving this milestone which we set longer ago than most of us can remember and I see people in the audience who are on the steering committee who completely forgotten we were trying to do this donkey's years ago. So it's mainly a celebration which will be happening upstairs in the common room and in the Atlas rooms where there will be suitable refreshments but you have to of course suffer before you can have fun. So I'm going to tell you a little bit about the history of the project where we've got to and then I'm going to call on friends and collaborators to tell you a little bit about how Spinnaker's been used outside Manchester or at least outside the university. The goal we set ourselves a very long time ago was to see what we could do if we put a million mobile phone processors and they're made by or designed by Arm in Cambridge if we put those into a single computer able to support real-time models of the brain and back at the beginning we realized that even with a million processors you only get to 1% of the scale of the human brain and that's with lots of simplifying assumptions and in practice we're probably a little bit less than that the same 1% of the brain is about equivalent to 10 whole mice and now with the sort of richer knowledge we're getting of mouse brains realistically we can probably still run a single mouse brain network on Spinnaker if we do the work to get ourselves there. Now how did all this come about? Well, you've already heard the 20 years in conception story and the origins of the project do go back to 1998 which my arithmetic is right it's exactly 20 years ago when by virtue of getting some funding from Arm in Cambridge we were eligible to apply for a small EPSRC grant under the heading of ROPA or realising our potential award now these grants are so far in the distant past that Anna is not sure of what they are but I can tell you the requirement here was that you had to have qualifying industrial funding and you had to propose a new direction of research so in the 90s my group was designing a wide range of asynchronous processor technology and we had to take this off in a different direction and I'd become frustrated at that point by the fact that although processors were very much faster than when I started playing with them in the late 70s they still couldn't do things that Brains found easy but I was also interested in associative memories and how you build those on chips and so we produced this proposal to design efficient VLSI architectures for inexact associative memories standard associative memories which kind of do reverse look-up compared with conventional memories are very brittle you give them the right input they give you the right output very small errors they give you completely meaningless output so the question is could we make that softer could we make memories that gave you approximately the right output if you gave them approximately the right input and the answer to that question as we pursued it basically amounted to however I looked at it we appeared to be reinventing neural networks okay so in this work all paths seem to lead back to neural networks so at the end of that I thought well okay if that's the case maybe I should pay some more attention to neural networks and see where that leads and I started looking at what was being done around the world in building systems because basically my group of computer engineers we build stuff at building systems to support neural network modelling this there's then quite a gap while we thought about this we discussed plans and things bounce backwards and forwards until eventually we came up with a sufficiently coherent set of ideas to bid for a grant and again you'll see this is EPSRC responsive mode and the title here is a scalable chip multiprocessor for large scale neural simulations and that if you like is the Spinnaker concept and we attracted this funding and started designing the silicon itself the issue of scale really came in with the next grant and this was the last ever ICT large grant because this was a scheme that EPSRC ran up to a point and we were fortunate to get the last one which was particularly well adapted to what we wanted to do and here we had the title biologically inspired massively parallel architectures computing beyond a million processes so you see the million process a target had certainly appeared by the time this proposal went in which was presumably some time in 2008 and that pointed us in the direction of the trajectory which we've if you like come to this major milestone on today and the thing I want to note here is although we now receive quite a lot of funding from EU sources we would not have been in a position to even bid to engage in projects such as the human brain project had we not built these foundations using UK funding from EPSRC now I don't want to take you through every possible grant because there are quite a lot of them and some of these are specifically Spinnaker related and some are broader with components of support for Spinnaker there were some grants here where Spinnaker was used as a tool to support other work you see there's quite a lot more EPSRC support on there but also it got us in a position to bid to join the human brain project and that's now our major source of funding going forward to develop a second generation machine which I'll say a bit about later and to continue improving the software support for this machine to extend it to a large sort of global user base and on route we've had some other European funding through the European Research Council and the little bit at the bottom which is small because it had to fit on the slide but not insignificant is the university actually provided an injection of a quarter of a million of capital because we set off building the million core machine with the target of delivering it on a build cost of a pound per processor and in the end it ended up costing about one million but one and a quarter so not quite to plan now I've mentioned human brain project funding and at this point I feel I ought to inject a slightly sad note because within the human brain project in the neuromorphic platform we worked very closely with a group at Heidelberg who are developing another large scale neuromorphic system which works on quite different principles which is shown at the bottom right and this group and the whole neuromorphic activity has been led throughout HBP by Karl Heinz Meier who very sadly passed away last week and I would like to acknowledge that a lot of what we've been able to do in HBP is due to Karl Heinz and his passing is a major loss he with Henry Markram were the two principal proponents that led to the HBP flagship being funded in the first place so he's a great loss and the thing I noticed that whenever he was talking about neuromorphic computing in public he would always give equal emphasis to the brain scales work at Heidelberg and the spinnaker work at Manchester a very balanced view and I think that's an example respecting the work of others that we can all follow so I wanted to note that now while we've been building this machine over the last ten years those in the business will have observed there have been parallel developments in machine learning applications and industry has developed very impressive advances in machine learning through the use of large scale artificial neural nets now artificial neural nets are not the same as the things we run on spinnaker spinnaker is intended to support biological research and therefore we model neurons that spike artificial nets don't spike they produce continuous output it may not sound like a very fundamental difference if you're not in the game but it's led to quite parallel but distinct development trajectories and these artificial neural nets are now everywhere if you talk to Siri on your iPhone or Alexa on your Amazon Echo then when you say something apart from the trigger which turns the speech system on all that you say is sent off for processing and then the interpretation comes back with the answer to your question to your phone so it's not happening in your phone it's happening on the far side of the planet typically and these are formidable networks the way these work is you show this particular netting image at the left hand side images have been quite a major area of development I think usually you show the picture of a cat and then this flows through hundreds of layers of neurons there's a classifier which comes out and says it's a cat or if it's not trained properly it comes out and says it's a dog and then at the right hand end you have to tell it the difference between a cat and a dog and feed that difference all the way back up the network adjusting millions of parameters until the network starts saying it's a cat and if you show it 10 million pictures of cats it'll become quite good at recognizing cats at the end of the process now for the artificial net my observations of the biological net are that if you take my 2 year old grandson and show him one cat he will recognize cats for the rest of his life okay so there's some fundamental difference between what's going on in these industrial nets and biological nets the problem is we don't understand how the biological net works now my comparison is a bit unfair because of course the google net that recognizes cats completely scramble brain whereas my 2 year old grandson starts off having had 2 years to build a model of the world inside his neural network into which cats fit rather neatly so you know it's not a simple comparison but there's certainly a lot of interest in industrial networks in learning from biological systems to reduce the training cost if you could train your network with 100,000 cats instead of 10 million it would dramatically affect the economics of those industrial processes you see this network is characterized by picture at the left, information flows to the right mainly the information flows in one direction in these nets although there is growing interest in a little bit of feedback but that's hard so there's not much of it and it's trained through this mechanism called back propagation which is sending the errors back up the network if you look at the biological system the picture is quite different this is an abstract picture of Cortex and what you see is information coming in at different places going backwards and forwards flowing all over the place it's a much less simple picture and this is the kind of thing that we're trying to understand with tools such as Spinnaker in the human brain project the biology uses spiking networks which means all they do is so often they go ping so all your thoughts while you're sitting there thinking deeply about what I'm saying or possibly thinking deeply about the refreshments you're going to next all those thoughts are patterns of pings flowing around between the neurons in your brain it's the best of our knowledge which is not complete so they're spiking neurons the Cortex has two-dimensional structure and it's always sparsely connected the current solution favoured for industrial networks because they're densely connected are these GPU engines this is the flavour of the month from NVIDIA this is a GPU so originally designed for graphics but it's very good at matrix operations so the dense connections that typify industrial networks fit here well when they learn how to use sparse networks as biology does the GPU will go into history this box consumes about three kilowatts and we're learning how to operate at increasingly low precision to get more efficiency out of these but it's still a fairly power hungry device if you want to model biological nets then the state of the art is to use a supercomputer now you can get reasonably efficient support for sparse connectivity, sparse matrices and you can communicate spikes around but now for one of these big machines your power budget may have gone from kilowatts to megawatts if you're prepared to accept a certain degree of simplification down onto what are called neuromorphic systems and you've heard Spinnaker described as one here there's a very strong emphasis on localising the memory and computation so that the data moves around much less because that's really what burns power you can get real-time networks running not on megawatts but on milliwatts so you see we've got a sort of scale of about nine orders of magnitude that you can explore while investigating this sort of system and clearly there are widespread applications of artificial nets there's much less direct exploitation of spiking nets but there's growing interest in the big companies and start-ups playing or investing in this space so this year Intel announced the Lowihe chip which is a spiking neural network platform very impressive, IBM's had true north throughout as long as we've had Spinnaker and then start-ups there are spin-outs from Zioric looking at spiking-based vision sensors and here in Manchester we'll hear later from Mike Denham with Mindtrace who developed seed funding for working in this area so there's industry interest but there is what we're really short of at the moment is compelling evidence that this neuromorphic technology can actually deliver in commercial applications it's clear it can deliver in brain science but it's not yet clear that it will deliver in commercial applications anyway what do we do we made a chip there it is it's a square centimetre of stuff about 100 million moving parts on the chip it took about 5 years and 40 man years of design work to put that together one of my claims for microchip design as a research area as you get more PhDs to the square millimetre than in most subjects which is clearly an important metric we designed this chip between about 2006 and 2011 so it's been around for quite a long time and it's put in a package with a standard industry memory chip and we can then tile a two-dimensional surface with it now this isn't supposed to be a technical lecture it's more a celebration but what's on that chip well there are 18 of these microprocessor things somewhere there's quite a lot of memory but the key innovation in Spinnaker is how we connect all this stuff together and the thing you can't do with a computer is make the number of connections between neurons that you find in biological systems so that was the focus of the Spinnaker design that's the red bit in the middle it's the router every time a neuron spikes on Spinnaker it generates a tiny packet which flows around the system this is like the packets that flow around the internet only these are very small internet packets carry kilobytes of data Spinnaker packets carry a few bits of data we can then route these to many thousands of destinations so it's not just going from A to B it's going from A to a thousand B's and it has to arrive in a small fraction of a millisecond to deliver the biological real time that we're looking for so that's the key it's the only thing you need to know about Spinnaker really the rest is just lots of processes in the box making the stuff is fun academic research projects we get our memories from Micron I don't quite know where they actually make them but they're headquartered in Boise, Idaho they send them across to Chengdu to UNISEM the Spinnaker chips themselves are manufactured via Euro practice which means going through Leuven in Belgium they're manufactured by UMC in Taiwan who then send the chips to UNISEM they assemble them in these small packages and then the packages come back to us and then we send them to Norcott there are several people here from Norcott where are you Pete oh they're over there this is the Norcott bunch they play a crucial role in Spinnaker because they get the PCB's made and then they assemble all the components onto them and this is a non-trivial PCB to assemble because it's got a very large number of connections to be made reliably and all the boards you see there's one over there and there are 1200 in the machine and there are several others lying around these have all been through Norcott's hands and we've been very happy with the collaborative way they've worked with us on building this stuff for Spinnaker so we have these chips there's what's inside the chip I've never seen it if you go to the assembly line which is a clean room so they won't let you in and they stick the Spinnaker chip down and then they stick the memory chip on top of it and then they wire them together with tiny gold wire with these machines that work at fabulous speed and the astonishing thing is they can do this extremely cheaply so we can then put those on a circuit board there's one down here I'm never quite sure which side of this board to show because that's the pretty side but all the flashing lights are on the back I'll leave it that way for a bit and then we can take these boards and we can assemble them together in this large machine which is in the machine room upstairs and when we go up for the celebration for the refreshments we're organising tours of small parties to go and see the machine there'll be some fairly evident sign up mechanism when you get there the machine room itself is not enormous so we can really only take about a dozen or we'll take 20 in at once I'm not sure anyway so if you all want to go we'll manage it but it'll take quite a long serial operation to organise that we build these big machines we committed to deliver a half million core machine to the human brain project by the end of March 2016 and that's been online serving remote users across Europe and in fact around the world they're all submitting jobs so we've run quite a lot of jobs that's the good news, the bad news is they're all tiny so we have a huge machine and nobody's quite worked out how to use it yet that's the next challenge but that half million core platform is now upgraded to the one million cores and that's what we'll be offering through the human brain project going forward what can you say about the machine well one of the statistics that terrifies me is that if you count up all the chips the memory and the spinnaker processing chip there's about 10 square metres of active silicon area inside this machine okay I've attempted to draw that roughly to scale to this cartoon man you know when you think of microchips you think of tiny things like microchip the size of this front area here it has many moving parts with the machine this scale it'll never all work at once and therefore we've had to develop technologies that allow the hardware to accommodate faults and allow the software to understand how to use the machine whilst avoiding the bits that aren't proving reliable a lot of engineering has gone into understanding how to make this machine appear reliable to its users but we're pretty much there and the machine operates pretty reliably both for external users and we have quite a lot of users in-house now so that's the big machine now we come to the high risk bit of the talk because this is where Andrew is going to come out and we're going to attempt a live boot of the machine and the program that we've developed to do this does interesting things it doesn't take too long we're not switching it on because that takes quite a long time if you just pull a big switch and turn it all on the entire Manchester goes dark so we're not doing that instead what we're doing is we're bringing the machine up and Andrew is running on this laptop and what's going to happen? the first thing I'll do is press this button and we'll start the boot process so all of these red lights that you can see up here are the ethernets of the machine there's 1,200 of them now we've sent the boot image into one corner of this and what's happening is it's spreading out the boot image across the whole machine so what you can now see happening here is we're speaking to each of these ethernets saying are you booted yet? we've now received the image so they're not fully booted, they're just getting ready to speak to all the other chips so you can see the way the machine communicates here as well because it obviously goes around the circles you can see it goes faster in one dimension than the other so now it's reached a point where everything has received the image and what's happening now is all the chips are communicating with each other really fast so they're sending messages saying we're here, we're here and they're all finding paths throughout the machine a way to communicate with all the other chips across the machine hopefully in a minute it will then finish that process and everything will go nice and blue is it going to happen? it's okay, this process does take a little bit of time and you can see the number in the middle is the number of processors reporting for duty so now what's happening is it's now counting up the processors on each chip how many processors do you have and we're adding on to this number in the middle as it goes up you'll see some black squares appear in this diagram which is basically where as Steve says this machine isn't all going to work at once all the time so we have to have parts of it that aren't working and so sometimes we have chips that aren't responding to us at any one time with any luck this will still go for a million so live demos are high risk activities we did have a here's one we made earlier back up that wasn't it but that wasn't it I can assure you that was happening live on the machine that you can go up and see later on now we built the big machine it's not the only machine around in fact we've got about 100 spinnaker systems in use around the world with other groups we started off loaning them then we ran out of money to make boards to loan so we started selling them and you see that we've got pretty good global coverage I guess we need a bit more interest in South America I'm not really optimistic about Antarctica but we've got coverage everywhere else from Auckland and New Zealand to some boards in Boise, Idaho which is where I said micro on a base but it's not micro on that I've got the machines I don't know what's going on so with the little boards, the four node boards you can do very localised mobile robotics work with the big boards such as this one here you can get to the scale of a small insect brain so you can model something like Drosophila that's just the scale of the machine it's not easy to get to build a model of Drosophila but in principle that's what you can do with the million core machine here which will support a mouse and what can you do with this machine well you're going to hear more about this later but we've worked with our HBP partners at Eulich and Sasha's going to tell us more about this model after my talk we've built a model which can be run on a supercomputer and also on Spinnaker and we can compare the results and check that we get numerically reasonable outcomes because if you run a very large number of processes in parallel you've got to have some way of knowing that the results are sensible that's 77,000 neurons 285 million synapses the synapses the connection from one neuron to another and we'll hear more about that in a little while we've also done some more abstract problems so one of my students has built a stochastic spiking neural network that solves constraint satisfaction problems in which the most familiar example is Sudoku and there you saw a solution emerging that was one I recorded earlier by the way in case you're wondering and so this network uses a kind of stochastic annealing algorithm it's very analogous to what you can do with a D-wave quantum annealing machine in terms of the class of problems that you can sensibly address with it so these are abstract problems and then of a more commercial application nature we've got some preliminary work now on doing building very sparse networks that implement conventional deep networks and here's something that has a 96% accuracy on MNIST which is reasonable for the size of network here but it's only got 0.6% connectivity so we're pursuing this idea of extremely sparse networks much more like biology than the industrial networks we haven't stopped with Spinnaker 1 within the HBP we're developing a second generation chip in collaboration with a very good silicon team at TU Dresden in Germany because of advances in process technology we can go from 18 processors on a piece of silicon to 160 and this allows us to deliver about 10x performance and efficiency improvement most of which comes from advances in process technology so we're designing it to go with the memory you have to think very hierarchically about these big chips so we're putting four processors in what we call a quad processing element and then tiling 40 of these onto the chip and that's in mid-process at the moment the first chip with this layout on has come out of fab, we haven't seen it yet it's still in Dresden but we should be getting our hands on it and starting to play with it in January so the work is still going on I'll sort of wrap up here just to summarize you've heard this about three times already Spinnaker's been a long time in conception, different millennium when we started thinking about this building it for a long time we set the million processor target quite early on we set it, I had no idea I would be drawing my state pension before we got there but that's a different story we have machines with groups around the world we're supported by the human brain project on the software side and developing the next generation there's growing interest in what this kind of technology might do for industrial AI though as yet no compelling demonstration that it's going to take over the world in that space but there are a number of neuromorphic platforms with different trade-offs between efficiency and flexibility and Spinnaker is at the most flexible end because our neuron and synapse learning rules are all in software so we can change them easily so it still seems to me that it's a very good way of developing research into this space if you don't know what you want you want a flexible platform to work out what you want when you know what you want more efficiently some other way now I've done most of the standing at the front so far but lots of people have contributed to this work and since it's a very long list I've taken the nearest I can do to Hollywood techniques and I hope I've got this right what I was going to suggest is because this takes a little while to get through will everybody who's worked on Spinnaker in the past or at the present stand up please so you see it's a lot of people it's not quite the entire audience but anyway the achievement we're celebrating today is the achievement of the people you see before you so I think we should give them a round of applause okay so I've said my bit now we're going to hear from a couple of activities that have used Spinnaker outside the Manchester group and they will tell us their experience nicely I hope so so the first speaker is Sasha van Albaarder who's from the Eulich Supercomputer Centre and I'll hand over to her your applause was missing now please give a hand to you so good afternoon everybody my name is Sasha van Albaarder I'm from the research centre in Eulich I'm a computational neuroscientist and I've had the pleasure of working together with the group of Steve Furber on a project in which we ported a neural network model that is at the same time tiny and humongous to Spinnaker so this work took place in two great European projects starting with brain skills and later the human brain project now what is this huge little circuit that I'm talking about it's a model of all the neurons and synapses in a square millimeter of mammalian cerebral cortex amounting to roughly 80,000 neurons connected via about 300 million synapses and the reason for considering this circuit is that it forms a generic building block for cerebral cortex as it captures most of the local synapses because the local circuitry has a range of on the order of a few hundred micrometers and because most of the synapses onto cortical neurons are local this captures the majority of all the synapses and thereby the circuit is largely self-contained the cerebral cortex is organized into layers and the model describes each of these layers with two populations an excitatory one so the neurons that increase the firing probability of their target neurons and an inhibitory population with neurons that decrease the probability of firing of the target neurons and what this model does it relates to the network structure, the connectivity to the network dynamics and to keep things simple we model each of the neurons with the identical intrinsic parameters so that we really focus on the influence of the connectivity on the dynamics this connectivity is given by a layer and cell type specific connection probabilities but apart from these specific connection probabilities the neurons are connected at random so there's no further spatial structure an external stochastic input to all the neurons represents the non-model parts of the brain and the model was originally implemented using the neural network simulation software NEST to run on a local compute cluster in defining the connectivity of this circuit the model integrates the knowledge from more than 50 experimental papers and it's thereby able to account for the type of neural activity observed in the waking brain where the neurons fire action potentials in an asynchronous and irregular manner you can see this in this so-called raster plot or dot display the neurons are ordered along the y-axis and the x-axis represents time and each dot is a spike of one neuron the spikes of the excitatory neurons are in blue and those of the inhibitory neurons in red and one feature you see so the the asynchrony because you don't see any vertical stripes here and if you look at spike times of one particular neuron then they occur at irregular intervals also the inhibitory neurons tend to spike at higher rates than the excitatory neurons in the same layer despite the identical parameterization of the cells and traditionally the higher rates of the inhibitory neurons were attributed to different intrinsic properties but this model shows that it can also be accounted for by the network structure and a final experimental observation that the model is able to reproduce is that the firing rates differ across layers with layer 2-3 having the lowest rates and layer 5 the highest rates in our project with Eulich and Manchester we set out to bring this model to Spinnaker and it quickly turned out that this was a bit of a challenge because the properties of the network differ from those of neural network models that had previously been simulated on Spinnaker in particular our model had shorter time constants so that we needed shorter integration time steps and also a larger number of synapses per neuron and this causes the input rates to the neurons to be very high and that could be a problem because if the receiving neurons are still busy processing other spikes then a spike may not be delivered on Spinnaker but our colleagues here in Manchester especially Andrew Rowley were able to solve these issues to run the model on six Spinnaker boards which a quick calculation will show is still less than 1% of the full system as it is in place now and in a comparison between nest simulations on a time grid nest simulations with precise spike times not restricted to grid and Spinnaker we found that the statistical dynamical properties of this model were equally well represented on each system and this means that the fixed point arithmetic and the asynchronous update of Spinnaker were valid design features I understand that the new Spinnaker system will support floating point arithmetic making it even better able to represent biological neural networks such as this one and I think one important aspect of this collaboration was that we computational neuroscientists and the neuromorphic hardware developers here in Manchester understand much better now what each of us means what each of us does and what the challenges concepts and solutions are so that these insights from each side can flow into the simulator development into the traditional and neuromorphic simulator development alike now what makes these particular results interesting well as I mentioned the cortical microcircuit can be considered as a sort of building block for cortex because it represents a large percentage the majority of the synapses impinging on the neurons making it largely self-contained and we also know about Spinnaker that it has a smart way of rooting the signals between the neurons so that in principle the communication only grows linearly with network size and these two factors combine mean that we could in principle take a whole lot of these building blocks and fill all of Spinnaker Spinnaker's 1 million cores with microcircuit building blocks and simulate in mouse brains or a small portion of a human brain in this study we also characterize the performance of Spinnaker and Nest in terms of speed and power consumption and when plotting the results of these measurements we chose some really nice colors but a big conundrum arose which was what to call these colors and the early idea of Marcus Diezmann was to call it's difficult to see but the mapping face to call that orange and to call this brown but I thought that this required a bit of a stretch of the imagination and I proposed magenta for this phase to which Marcus replied there is no discussion about magenta in this country this is defined by German telecom and he had a good point and he hopefully went on to suggest alternatives namely salmon for the mapping phase and raspberry for the data generation phase and reading this for some reason I suddenly felt like having a nice meal so I thought this was quite a good combination and so I said I'm starting to get hungry and consequently we kept this gourmet selection in our caption and whether due to our culinary offerings or other factors there has been receiving a surprising amount of attention for a paper with such a long title already more than 15,000 views which brings us into the 97th percentile of all frontiers papers even though our paper only appeared in May this year and also the media picked up on our study and it was featured not only on the University of Manchester and Research Center Yelig websites but also on Science Daily and on the American radio show Science Friday and because of all this media coverage I even got an invitation to speak at a TEDx event in India about Spinnaker so now I in no way agree with Trump's attacks on the media but it does indicate that a little bit of distortion is taking place because now I'm suddenly seen as a Spinnaker expert when all I've done is to perform one study in collaboration with the actual Spinnaker experts here so what's more it appears that due to our study we are suddenly very close to curing Alzheimer's disease if it's written on the same page as the soap opera spoilers then it must be true so in fact when I investigated the evolution of views of our paper I discovered that the sudden jump in views coincided with the appearance of our paper against Alzheimer's website so correlation, causation who will tell and we can also learn from the media that Spinnaker is not in fact a neuromorphic supercomputer but a mere brain inspired laptop which has been tested for vitality effectivity among other things to incorporate studying and other issues and that it does so via the trade of alerts between neurons but at least it's a laptop with one million cores so I think that's no mean feat and I mentioned before that the microcircuit we simulated can serve as a building block for much larger networks no longer big little circuits but truly big circuits in Ulich we have already started thinking in this direction and we have developed a model of all vision related cortical areas in one hemisphere of the brain of the macaque monkey consisting of two circuits this model contains four million neurons connected by 24 billion synapses and it's currently implemented in Nest and runs on a supercomputer in Ulich and since a single microcircuit obviously was not enough of a challenge for our colleagues here in Manchester our next project as part of the human brain project is to port this model to Spinnaker so I warmly congratulate Steve Furber and his group for creating this million core neuromorphic laptop and I look forward to our continued collaboration thank you thank you very much Sasha that was that was great so if you fancy finding a strong chair somewhere and sitting down and asking us to put our million core laptop on your lap what does it weigh about five tons yes good luck the cortical microcolumn workers was very challenging for us and really pulled the state of our technology forward and we're now using it as a benchmark for tuning the software when the paper was written loading and running the cortical microcolumn model took about eight hours it now takes about eight minutes it ran on six boards with a 20 times slowdown from real time and Dr Leicester on the front row claims he can now run it on less than one board in real time so we've seen sort of since the paper was printed about a fact a hundred improvements in both the run time and the execution efficiency we haven't yet seen Dave demonstrate his hundred execution efficiency but you agree Dave you're going to do that okay anyway we should move on and the next speaker is Mike Denham from mindtrace.a oh he's there, yes good still in the room okay Mike I'll hand over to you thank you very much there's the usual sort of thing with the pointer and the red one okay good afternoon everyone I'd like to start off by adding my congratulations to Steve and the Spinnaker team which we saw quite a few people in the audience for this amazing project and the success of being able to achieve this million core machine is absolutely amazing and I'd just like to say that this is largely due to the sort of inspiration which Steve has provided over the last decade or more I've known Steve for for many years and he's been an inspiration to me as well in the area of neuromorphic computing computational neuroscience and to see this machine growing and this opportunity to actually really build serious computational neuroscience models in a scale which represents the human brain or a sub part of the human brain I think is truly amazing and I'd also like to say how happy I am to be here today to celebrate with the team and thank you for the invitation I'd like to introduce you to our company mindtrace.ai we've been in existence my co-founder here a committee at Imova since January 2017 and as Steve mentioned earlier around about almost exactly a year ago we were successful after a lot of hard work to get venture capital funding for our seed investment of £1.5 million and that's been an inspiration for us to use Spinnaker and know that we have a platform on which we can start to investigate how this type of architecture can contribute to the important area and the growing area of machine intelligence so what our aim is is to take a brain inspired approach to developing machine intelligence our strap line which we use a lot now and this has come about really because we believe that the current state of machine intelligence although as Steve mentioned earlier it's made remarkable progress over the last five years in particular in image and speech, recognition natural language processing and so on that the current technology what we think of as deep learning or deep neural networks are matching what we think of as the human capabilities of of intelligence the need to train on tens or hundreds of thousands of examples for instance is something which we don't do and it's very rigid and fragile in transferring learning from a trained situation to a novel situation on which it hasn't been fully trained in contrast as we've also heard humans learn new concepts from just a single or a few examples what we call or becoming known as one-shot learning they generalize robustly and accurately to transfer knowledge into new domains and we can use those learned concepts in a much more richer way for action, imagination, creativity and so on to give you an example of what we mean by one-shot learning this is an example taken from a paper by Brendan Lake and his colleagues including Ruslan Saliputinoff who leads the AI group at Apple and Josh Tannenbaum which probably you've heard of at MIT a very renowned guy so what this is you take a single handwritten character from a set of alphabets and you provide a set of candidate characters for matching and the simple question is what is the best match of this character with the candidates well we can do it quite quickly that's the one which is the best match so think about how you did that you probably think well what I did was I looked at that example and thought how was it built how did I draw that or how was it drawn and then look for something which was drawn in a similar way so what effectively we're doing is using an internal model a generative internal model of our understanding of the world and how characters are drawn and how they've built up in other words how we could take a character and pass it into its constituent components and the fundamental component of this is this idea of this generative model my interest is mission is to provide machines with human levels of intelligence make machines think we believe that the component parts of that primarily are to be able to do things like one shot learning to be able to continuously learn from not go away and do a big learning exercise in the cloud and then come back and do inference and then go away again and have to do another learning exercise to smoothly transfer existing knowledge into new domains and to make goal based predictive autonomous decisions in new and unexpected situations and that there are many many applications out there of such a technology if we can achieve it which will take the world of AI machine intelligence into a different category so what is our particular approach which we're adopting in mind trace well we're trying to exploit the synergy between brain inspired algorithms and my background is about 40 odd years in academic computational neuroscience studying the mechanisms in the architecture of the brain and combine that with this wonderful neuromorphic hardware which is rapidly developing now and of which Spinnaker is a major current example so one of the things which we realize that we need is to build these probabilistic generative internal models and combine them to keep neural network models of learning and inference which are inspired by this architecture and processing mechanisms of the brain and deploy them and this is the important bit on event based asynchronous many core brain inspired computing hardware we're firmly of the belief that if we want to go in this direction we need the sort of hardware platforms and computer architectures that the CEVS group has provided through the Spinnaker computing system because it allows you to do things in a way which we can't do on things like GPUs no matter how powerful those GPUs become they still have a fundamental restriction in terms of the ability to accommodate this type of architecture with all its feedback connections and so on and in particular one of the most important things which is growing now in the area of artificial intelligence and machine learning is the issue of energy when we started thinking about this maybe two or three years ago talking to people in Silicon Valley they were all saying well energy is not so important now energy is becoming a top priority energy efficiency with the growing emphasis on the use of machine learning at the edge of the cloud so the opportunity which platforms like Spinnaker provide in terms of low energy is absolutely crucial to provide the sort of applications that we need from mobile to the edge what we're doing initially is to develop in the company what we call our technology demonstrator or proof of concept this is a fast low energy machine vision system which uses end to end event based processing it takes a dynamic vision sensor which many of you may have heard of which is basically a sensor which is not like a normal camera which is frame based but has a set of independent pixels which report only motion in other words contrast change that pixel and it does that within microseconds so it's producing a stream of events and those events are then processed by our algorithms using the neuromorphic event based computing platform of Spinnaker so here's the general picture so we have a real world image an object of interest the sensor is a dynamic vision sensor and then we process that using combination of our brain inspired algorithms with the brain inspired neuromorphic event based computing system the application we're targeting initially is something called autonomous emergency braking which is fast becoming one of the most important safety systems within modern cars it's advantages over existing systems which we claim for our system will be that it be fast with microsecond latency doing selection, tracking and recognition of vulnerable road users through this end to end parallel processing system it has a high dynamic range due to the nature of the DVS camera very low energy due to the nature of the algorithms and the computing platform and a very fast response time and this performance target is the new car assessment program which is being run by the European Union which is in relation to what we call vulnerable road users and in particular looking at car to pedestrian and car to cyclist impacts which are one of the most frequent incidents which happen and lead to very serious injuries and very large it's a very important issue to actually address in terms of safety and security in vehicles our aim is to achieve a significant impact on that field and provided by providing this fast accurate low energy solution and substantially reduce pedestrian and driver injuries and loss of life ok so this is us this is our board we are very fortunate to having a chairman Hossein Yosai Hossein Yosai was previously the CEO of Imagination Technologies and he's very kindly taken up our interest as the chairman of our board I mentioned my co-founder and you'll notice here as a member of our board but most importantly is these guys sitting down below here many of whom I think everybody probably is in the audience and that's our team of researchers and engineers and software developers and they're providing the energy and the enthusiasm to deliver what I've been talking about today so thank you for your attention I'm sorry I couldn't go into any technical details in other talks but you'll understand that commercially we have some difficulties in talking about detailed technical issues so thank you ok so I think that ends the program of presentations I'd like to thank Sasha and Mike very much for sort of complimenting my talk on the history of Spinnaker to give some examples of uses of the machine and we very much like those to expand a lot of resource now so we're looking for big challenges for the future but I think the program now demands that we give you the opportunity to ask questions and I think Gavin is going to manage this somehow and I think you can direct questions to me, Mike or Sasha as you see fit, are we going to run it? Yeah, it could be good for us Can I ask a question? Thanks Graf You mentioned that you built a hundred of these Spinnaker machines around the world, is that right? Yeah, little ones Some are smaller than this, some are a bit bigger but there are not a hundred million core Spinnaker machines around the world My question was going to be you've got a hundred of them but why not a tiny one? Thank you Well thank you for presenting such a beautiful project since I'm first doing PhD in cyber security, I'm just wondering for such a big project how to cover the project back does it go under any cyber security testing to ensure such chips are secure enough before going to a real remote place? I'm pleased to report the Spinnaker machine itself has absolutely no security mechanisms whatsoever Of course it isn't physically connected directly to the internet there is a host machine that manages it and the host machine has normal levels of security but I don't think Spinnaker is a real security risk at the moment if anybody can work out how to make it do anything interesting in that space Please let us know, yes Yes, I'll sign up for a PhD I was just going to ask you The architecture does have some limits but they can all be circumvented we set the million core target really to pick a very big number so that we had to address scalability head-on from the outset but we did for a million core there is a sort of internal address space which is addressing chips in its 16-bit so it was a kind of 65k limit but I'm sure we could get around that if we had to the neuron identifies the descent around the machine or 32-bit so potentially there's a 4 billion neuron limit but again actually if you use the number locally in one area you can reuse it in another area so that's not a hard limit I suspect we begin to have the real time if we made the machine 10 times the size the other limit as you'll see is it fits in a fine size room and there is only 100kW of electricity available in that room so under 100kW of cooling the machine if you work it all very hard which is a challenge which I set you then you can get it up towards 100kW although normally the university's electricity bill you'll be pleased to know actually the other problem we have I asked Jim for permission to give him a microphone give him a microphone Dave wishes to make a comment I asked Jim when we could come up with a mega-block machine for the next generation so the subscription would be placing underneath this room and probably we'd have to pay the electricity bill so those are two constraints which are worth mentioning in this context Have you got any applications outside the neural office space well so the machine looks a bit like a general purpose parallel computer except the cores of course are very small so I mean we do run non-neural on it we have a model that we use for debugging the machine early on which is just a heat diffusion model which is doing a very simple algorithm at each node and there have been some other things we run large scale Markov chain Monte Carlo generation on the machine you can do other things but you have to ask yourself the question as to whether it's the right machine to do if you have a problem which is elegantly expressed as a large graph where all the nodes are doing relatively simple things and there's lots of small scale communication going on then there's a reasonable chance we can support it but it's certainly not a general purpose computer there's one thing it does better than a high performance computer which is model real-time systems of spiking neurons almost everything else it does worse can we have any more questions perhaps on the neuroscience side are you curious hi okay no I think just probably more about the human brain project are you trying to understand how the human brain works or are you trying to recreate the decisions of the human brain mind oh oh there is a supplementary to this oh so so far we have not looked into decision making and we would like to understand how the human brain works thank you question actually comes to mind is if you want to replicate decision making how do you differentiate between the decisions that were led to say Trump or Brexit and um how do you describe to here's a here's a Ponzi fraud scheme and what else so so I think this impinges on you know often given a bit more time I talk about Turing's views on human like artificial intelligence and why it's proved so much harder to build that kind of intelligence into machines than Turing many people since him expected and I think that's because Turing made the mistake of thinking that human intelligence was based on logic and there's been quite you know there's half a century of symbolic AI which is based on that assumption which turns out you know to be interesting but not as a way of understanding human intelligence because I think human intelligence is sometimes the opposite of logic you've quoted some good examples I think just some curiosity right contrary to the other smaller machines around the world you have in one room all these small versions of the small machines that you try to work in unison I wonder besides thinking of a bigger brain or a human brain is there maybe an interest on trying to make each of the little ones individual agents and make them work not as a single computer but as a group of small insects to investigate emergent phenomena in you know basically because you have all these computers in the same place and you can make them talk as individual entities but is there any interest on that certainly that's a possible way of making valid use of the big machine there are more ways of making valid use of the big machine than just building a single big network to run on it so one of my students Ed he's here, he's a wave somewhere Ed has sort of built a genetic algorithm framework which runs hundreds of jobs hundreds of smallest jobs on the machine simultaneously so you can use it for running lots of independent jobs but if you want to investigate swarm intelligence and you want to model each of those intelligences as a small network then that's certainly again something that you could use the machine to do there's a vast number of things one could do and life is finite so we can't pursue all the interesting options but if you, I mean the machine is available if you want to do that I saw Carol Yes, I'm going to ask a question on behalf of shy colleagues which is why did you call me Spinnaker? Yes, okay choosing project names is an interesting question Spinnaker is a sort of compression of spiking neural network architecture and if you're observant you'll see that we always spell Spinnaker with two capital ends in the middle so it kind of it's not an acronym but it's a kind of compression and it makes for a nice pretty logo because of the sailing association you see pictures of sails even on balloons these days apparently so that's the origin of the name I mean I like to find a name for a project which transcends the particular funding regime in which it started it's a name that can basically eat funds for the rest of time that's because actually I think individual three year projects are not long enough to really establish a visible brand and so getting a reasonable name and I think in the 90s we had amulet for our asynchronous processes and that covered a whole range of funding sources and Spinnaker has actually become quite well established as an internationally recognisable brand in the rather sort of small sphere of neuromorphic computer people but it's it gives it recognition that transcends whatever particular project it's being funded through at the time maybe I have a follow up question for you then have you already been contacted by people from the world of sailing no although it does mean if you go searching for Spinnaker videos on YouTube you will learn quite a lot about sailing on the way to finding what you're looking for there are many attempts to estimate where we'll be able to simulate the human brain functionalizing would you like to make an estimate of when you'll be able to simulate a mouse brain on your machine so I claim the machine has enough capacity to do that admittedly with relatively simplified neural models so biology is extremely complex and you have to abstract a lot of that complexity away to get something you can compute sensibly at scale so there's a lot of simplification going on here within the human brain project there are already exists a complete mouse brain model now it's very rudimentary and when Mark Oliver Gewaltig was here describing it he said think of the first attempt to draw an atlas of the globe they drew this thing some bits of it are recognizable some bits are a bit wrong and Australia was completely missing because they did it before Australia being discovered and I think the mouse brain model is probably at a similar state of rudimentary development but it is a model that we can eventually run although there's quite a lot of work required to change the way it's constructed to make it suitable for running on Sminneker I mean the mouse brain is just under 100 million neurons and the current representation of the mouse brain if you imagine the connections between those 100 million neurons you can think of a matrix which has 100 million rows and 100 million columns and then the connections are drawn into it that's rather hard for us to deal with we have to somehow cluster those groups and break them down impose the kind of hierarchy that we like in our models and then it might be possible and Dave is still shaking his head take a moment to run sorry can I just expand on that so what's the inputs and outputs on your machine that's going to be the inputs and outputs of course this is an important question when you're brain modeling is that for most purposes brains without inputs have no purpose and just sit there and quiver the standard approach to this is to try and embody the brain in some kind of body embody in a body does that make sense and again the human brain project there is a virtual robot mouse which is actually quite realistic so potentially you could embed this virtual brain occupying these enormous racks and couple it to a model of the mouse now that is easy to say I think very challenging to do and of course then you connect the virtual mouse brain to the virtual mouse robot and if the result likes cheese you're getting somewhere but virtual cheese yes okay what does virtual cheese virtually smell of I don't know so embodiment is an important step for the small brain regions then you can find what data is known about what the inputs to those regions look like and what the outputs should look like and the micro column model that Sasha talked about a lot is known about the rest state of the cortex basically that was a rest state model but in HBP we have theoreticians who are interested in adding functionality to those models and understanding what they do other than sit there and quiver so you can do it locally but often there's a fundamental problem if you take a bit of brain it's a black box you don't know what the inputs are you don't know what the outputs should be so how do you know the black box is doing anything sensible it's quite an important question to ask yourself regularly pretty much yeah see with the speed coming up with 160 processor on each chip would we be able to model one meaningful portion of the brain like real cortex no you might you might be able to get something like a small insect brain but even that's pushing it a bit I think now I think you need a few of those a few spinnaker 2 chips to get to insect brain ideally the spinnaker 2 chip will have the capability of one of the current boards like that but in fact that's only ideally in less than perfect circumstances it's a bit less than it's about a third of a board so what we'll be talking about 160 cores 250 neurons a core that's 40,000 40,000 neurons Drosophila is about 100,000 neurons so you're talking about less than half of Drosophila Drosophila is not very bright by the way you have to go up to about 850,000 neurons to get to honey bees and they're quite clever and in those chips if we make this bigger machine would we be able to make some portion of it well it's about 10x so we're currently 1% on some very idealistic assumptions we're a bit behind that so I think with a similar size machine made out of spinnaker 2 would be a few percent of the human brain of several mice getting funding to build a big spinnaker 2 machine is going to be interesting who should I look at here so with that talk of spinnaker 2 and the future I think this is an appropriate timeout to say thank you to our speakers today and say one final big congratulations to the entire spinnaker team for the past and present as well as seeing and having the vision to bring this to fruition so we give them all a big round of applause