 All right, welcome team and good afternoon and good morning in Almedin. We're live streaming the event as well and good day everybody for those of you who are connecting remotely and I'd like to welcome all of you to the second seminar of the year of our distinguished speaker series and Really, it's my great pleasure today to welcome professor Rafael Uste Rafael is leading neuroscientist and professor of biological sciences at Columbia University He has pioneered optical methods to measure and modify the activity of neural circuits of the cerebral cortex on the quest to understand how brains work Rafael Uste obtained his MD at the University of Autonomous in Madrid and his PhD from Rockefeller University and was a postdoctoral student at Bell Labs He joined Columbia University in 1996 and is currently the director of its Neurotechnology Center and co-director of its Cavalry Institute for Brain Circuits in 2011 Rafael led a small group of researchers to propose the brain activity map precursor to the US brain initiative and in 2016 he helped coordinate the launch of an international brain initiative He's presently involved in establishing ethical guidelines for neural technology and artificial intelligence what he refers to as neural rights Rafael received multiple awards for his work most recently sharing the Eliason global leadership prize Today, he'll present the use of optical methods to selectively image and manipulate the activity of neural populations in 3d in vivo and to alter behavioral choices. I Had the good fortune of visiting Rafael at his lab at Columbia University Earlier when late last year and I can tell you that it deeply impacted me I mean he impacted me for the possibilities and the capability of the science and the technology But also because of the consequences Of what this is all gonna mean to each of us and to our society So I could not be more delighted to have Rafael here today and share his pioneer research with our IBM community and Following this seminar actually Rafael is going to participate in the AI ethics board That we have in IBM to discuss this very topic in more detail and discuss some of the implications and actions that we can take and collaborate together So Rafa the stage is yours and welcome Thank you Dario and they're here for inviting me and delighted to be here So the story today starts with magnets and this is something that many of you are probably familiar with so I'm talking about ferromagnetism and you know that Magnets have these very weird property, which is that if you split them apart into individual atoms the atoms are not magnetic But if you put them together something happens and the system becomes magnetic So this is an example of an emergent property also known as a collective property Which is a property that by definition is not present in the individual elements of a system but emerges when the system is put together and Magnetism a hundred years ago gave a lot of headaches to physicists because they said well How can you get something out of nothing if there's no magnetism in the atom? How can you get the magnetic material until this? Undergraduate student Ernst Ising proposed a very simple mathematical model And this is the critical equation of the model in which he describes energy of the magnet And you can imagine and stand energy at the propensity of a system a physical system to change so you have a lot of energy you're likely to change and he defined it with this negative integral of the sum of the Spins of each pair of atoms times the coupling coefficient this term J which essentially measures how strongly couple atoms are with each other and The higher the J if these spins are coherent The larger the term which means the more negative the energy which means the more stable the system is going to be So armed with this equation he predicted this existence of these states of matter the spin glasses in which Parts of the material becomes magnetic because they have coherence spins that actually are in a way self-generated these territories Emerge an emerging property of the interactions of the atoms. They're not Magical they just come from the interactions of things. So So this is an example of a classical example of an emerging Property that has been understood scientifically But now if I draw your attention to neuroscience and you probably know that we don't understand how the brain works But there's a lot of things that make Sting that the brain is the mother of all emergent systems because if you want to build an emerging system What do you do you build a system that has many particles many units as many as you can and then you Enhance all these interactions because the emergence properties depend on the interaction You make them as strong as possible and as rich as possible and guess what if you look at the evolution of the center nervous system nature it's Systematically increasing the numbers To the point that in our in our brains. We have approximately 86 billion neurons Okay astronomical numbers of neurons and the connections and the connectivity of the brain is still Unknown but on average our neurons are probably each connected to about another hundred thousand other neurons So it's not an all-to-all connectivity, but it's it's a huge number of connections. So Well, that's exactly what you need if you're playing these emergent property games So it's very likely that the brain is Optimized for the generation of emergent properties And as I told you it's critical for science to understand the brain for three reasons one is because it turns out that this organ happens to be the substrate of our mental activity of our minds and as humans we define ourselves by our minds We are the ultimate cognitive animals So essentially everything that you are your identity your thoughts your imagination your perception your emotion everything comes from this generation of Activity of these neurons that you have in your skull So we understood how the brain works. We would actually understand Who we are for the first time from the insights and typically will understand what is a human? What does it mean to to say that we're humans? The second reason has to do with medicine. You probably know from personal experiences with friends and family members That mental or neurological diseases have no cure and it's the dark corner of medicine and they Afflict the large proportion of the population which is increasing as the population is aging and The reason we cannot help these patients because as doctors we don't understand the path of physiology Of the system or the nervous system, which is when the the physiology the normal function interns wrong It goes is defaulted and that's the path of physiology So if you cannot understand the physiology good luck with the path of physiology, there's no way you can solve the problem So that's another reason to understand the brain and the third reason Probably you realize that working at IBM that has to do with the secrets of bio inspired algorithms that are in there because brains not just of humans but of all animals are computing all kinds of sophisticated optimization problems with Hardly any energetic cost that in the our brain is the wattage of a small light bulb No, and and we can we can achieve feats of computation that are that are unheard of for for technology So there has to be the ways in which brains compute have to be Critical to invent new technologies in the future so Why don't we understand how the brain works? So it turns out neuroscience for the last hundred years has been anchored in a theory in paradigm, which is called the neuron doctrine which assumes that the unit of Structure and function of the brain is the individual neuron and it was actually called doctrine because It was a religious belief. There was something that it had to be true So it wasn't a hypothesis. It was a doctrine and Pioneers like Cajal and Sherrington using methods that reveal the structure and the function of individual neurons Started a wave of research that continues to this day in which people have been taken apart brains one year at a time Describing how the neurons look like like these beautiful images from the Cajal drawings To recording the activity of individual neuron with electrodes Sherrington was the first person to develop the method to record the activity of individual axons of individual neurons and What we've been doing for a hundred years is to record the activity of one neuron in one animal and Correlating that with a behavior of the animal or in one patient and correlating that let's say with a pathology of the patient for example But if this is an emergent system To do that. It's a little bit like trying to watch a movie in a TV by looking at a single pixel The image in a TV is an emergent property of the pixels It's built with correlations in space and time and color of the individual pixels and by definition It doesn't exist in the pixels themselves So this is another example of emergent property So so imagine the foolish national trying to understand something like this by looking at single pixels unless you capture The pixels all at once you won't be able to see these interactions So because of that People already a hundred years ago start to think that maybe the near-inducting was wrong And it was actually someone in Cajal's lab Lorenta. Don't know who's studying these these circuits Which Cajal called the impenetrable jungles where many investigators have lost themselves And he was he was speaking about himself because he could never figure out the logic of the connectivity of the serial cortex He never break the code there So like a good disciple Lorenta did the opposite of what he was told and spent his whole life studying these impenetrable jungles And he came to the conclusion that most of the connections in the cortex of vertebrates Were recurrent loops were feedback connections. This is something that he called chains and He imagined that the whole purpose of this is to generate what he called reverberations Which essentially means that you have a circuit motif There'll be an example here like this one here on the top right Would you have a neuron that's receiving input from the left? It's getting turned on and it's passing that information to another near on the right But it's also sending these feedback loops of connections that activate other neurons which activated itself again So in a way by doing that, this is not a trivial Design Principle because when you do that you can generate the state of activity Which is independent of the input because it's self-exciting and What Lorenta thought if he argues that the purpose of the whole nervous system is to do that That it's all wired like that to generate these internal states that he called reverberations The reason this is important is because if you can do that then you have something inside your head Which exists independently of the world? It doesn't need sensor input to get turned on It can be activated independently and if you can do that you can imagine how evolution could then use these states of activity these endogenous states as Building blocks and as symbols for things in the physical world And I'm from that point on instead of manipulating the physical world You manipulate the model you have a model in which these are pointers that are symbolizing Things that happen outside. So that was the critical insight that Lorenta put on the table And this was also picked up by Turing who came out with the same conclusion like this is all a feedback business So this is an example of an emergent property because it's not doesn't exist in the individual neurons You need the neurons to connect to themselves to generate these intrinsic states The same idea was picked up by hep who Quoting Lorenta Changed the name and instead of chains reverberating change he called these things neural assemblies But the idea was identical you have a group of neurons in this case are the notes of this graph and the Edges are the connections between the neurons that are all connected to themselves in one way or the other So that means you can turn the whole thing on and Then it remains on regardless of the input and What had did propose the originality of head is to argue that these things will build automatically If you link these neurons through a synaptic plasticity rule The hep case is the famous hep rule that two neurons would Neuron that fired together will wire together So if you have near and are firing together because maybe they're responding to the stimulus outside and Eventually they can work together to the point. They can self-excite Themselves independently of the outside stimulus and again at that point you essentially are off from the physical world And you can start building a mental world So going back to Magnetism John Hufffield in 82 publish a very influential paper in which he took the IC model Whole sale and he applied to neural networks He argued that if you have a neural network in this case These little triangles are the neurons and these are the axons and if this neural network is connected in a feedback fashion in the ideal case where the connectivity a graph is complete and If these connections are symmetric he argued this is isomorphic with the IC model And he defined a concept of energy in this case is not thermal energy, but computational energy, but again it captures the the tendency of a physical system to change And he defined it just like I think did with the negative term The these here are the activity of the each pair of pre and post synaptic neurons and The T's are the J's the coupling coefficients in this case are the synapses So if these neurons are strongly connected The and their activity is congruent this you can have a very big term here a very very negative term And he predicted the existence of these Spin glass like states in the nervous system, which he called attractors Which would be stable states of activity with a group of neurons would be firing in a stable or semi-stable Duration and He took this model and said if you can do that well He predicted that this would happen the minute you have these conditions and Said well that you can do that and then you can build yourself a universal computer because you can Build a system of attractors stable states in the dynamics of the neural circuits You can in fact understand the entire neural computation as an example of classical mechanics and these stable states could be Imagined to be the solution to computation or a memory So if the this Topo map would represent all the potential Activity states of your brain. Let's say and imagine this case you have these two valleys These are the attractors and we're looking at the energy Define as I told you with this icing question and then Imagine that your activity at any given time is like a ball rolling through this landscape No, and as he gets close to the valley It gets attracted it drops down automatically to the bottom. That's why these things are called attractors So this way of computing has the property of pattern completion. You can arrive to the computation or to the memory With partial information you don't need to specify the entire path So this was the Hopfield model and I was lucky that I overlap with John Hopfield when I was at Bell apps He was we were in the same department. I had many lunches with him and I came from the new indoctrin Camp in fact my thesis advisor was the great right great scientific grandson of Sherrington, so like straight straight through and and Listen to John talk about these things a lot. Oh, yeah, I mean you completely this the brain is not completely connected The synapses are not are not Symmetric and said no no no they don't have to be completely connected You can achieve the same thing with a random connectivity and this is the synapses are not symmetric You cannot prove the map, but that doesn't mean that this doesn't doesn't work like that. It's just that our math is it's it's poor Doesn't mean that the brain is not doing it. So so I told him well, how can we prove you wrong? Okay, well you have a new theory you have to falsify it and he laughs It's going to be difficult because you have to record from every neuron in the brain to map all the attractors First and then you have to be able to move the ball to any point in the landscape and then predict test what's going to happen so anyway, so I kept that in my mind and then many years later actually the This has resulted in the brain initiative and I'm not going to talk about that But the brain should if essentially it's the research program to develop methods to do exactly that to map the activity of Everything in the brain to see the entire TV screen and to be able to put the ball wherever you want to be able to manipulate the system In a multi-dimensional space so they can actually achieve any position in this dynamical landscape So I'm going to spend the rest of my talk telling you what we're doing in my own lab at Columbia To try to advance that that program to explore if there's emergent properties And this starts also with with chemistry turns out that when I was a bill at Rockefeller working with Larry Katz We found out Well, how can we measure these properties? So we have to put electrodes in every cell and this is not going to happen You're going to turn the brain to Swiss cheese. So we decided to use optics to use light turns out that using High affinity calcium keelators you can make them fluorescent by attaching them a fluorophore and When they buying calcium they change the spectral properties Which means that if you make the spectrum of this Die, you can tell how much calcium there is and Working with Larry Katz at Rockefeller in the whistle lab We found out by chance that you can use calcium indicators to label Every neuron in a neural circuit in this case is a brain slice So this is a brain slice of the mouse cortex every one of these white dots is a neuron It's labeled and they're alive and they're labeled with this calcium indicator The calcium indicator gets into the neurons and then you can take images with a camera or microscope and See in real time the concentration of calcium in the neurons Which was interesting if you care about calcium, but what was really lucky Is our chance discovery that if we took images of the calcium concentration in these neurons over time This neurons seem to flicker the calcium concentration could increase and decrease and This has to do with the fact that whenever a neuron fires an action potential So this is an electrical recording of the activity of a neuron as a function of time and this Lines are the famous spikes or the action potentials the electrical signals all or none that neurons Generate if you measure calcium at the same time every time there's an action potential There's an increase in the intracellular or free calcium concentration, which means that we can take movies. This is a process movie every Dot is a neuron 10 minutes in the life of a piece of brain of a mouse visual cortex and whenever these neurons are activated they turn red and The reason is because they fire action potentials and they change their calcium So through measuring calcium, we can indirectly interrogate who's firing one and This is 500 neurons in the brain of a mouse the mouse has About a hundred million neurons. So it's a little corner of that TV screen of the mouse But in that little corner we can see every nearer. So it's the beginning of the program to essentially watch the entire TV screen and Explore if they're emergent properties So I was also lucky that when I was a Bell Labs. I work with Winfrey tank Who invented two photo microscopy? So two photo microscopy uses comes out of physics Uses a non-linear absorption of photons by excitation of fluorophores by infrared ultra-fast photons that have two Advantages for microscopy of living tissues one is because it's infrared light it penetrates deep into living tissue you want to image with Regular microscopy The brain it has the optical properties of a glass of milk. So you cannot see but with infrared Yes, you can go in and see and the second property is the non-linear excitation generates Point-spread function essentially a focal point Which is narrowly a restricted in space To a tiny little area So that means that you have like a magic one that you can use to focus the light deep into tissue Without being dominated by scatter and these are two fundamental advantages, which again when we work with on this We didn't know that we're going to be that great just one of the things that that happened luck so in deputy and But nowadays Many labs in the in the country in the world are using two-photon calcium imaging to measure the activity of neurons with these methods in living animals So going back to the the gist of the talk how about these emerging properties? They emerging properties in brain tissue So using two-photon calcium imaging we started to look first in brain slices Just pieces of the brain taken from the cortex of mice We can keep them alive for a few hours Label them with calcium indicators used to photo and see who's far in there And when we did that we noticed these big changes in fluorescence in this case they're plotted negatively, but they correspond to this action potentials and You can build plots like this. This is what we call a raster plot where in the y-axis you have neurons So every y-value is a neuron and x-axis is time and This is a graphical representation of a movie like the one that you just saw when you have the neurons firing in This case every black dot is whenever the neuron fires at least one action potential using this calcium imaging method and You can see these lines of of black dots that run down And this is a coherent activity by a group of neurons that fars together During the period of time for reason that we don't understand So this is something that we call ensembles, but this is the same idea that people have Defined with other terms you could could have called them chains attractors It's essentially coherent co-activation of a group of neurons It's an emergent property because you would have never seen that if you record from individual near You have to know what the other neurons are doing and realize that when you fire you're part of a group So we found this discovered these ensembles in these tissues and But then and they had very peculiar shapes some of them were grouped So in this case is an example of the neurons in red are the ones that belong to one of these ensembles In this case, they're a group near each other in this case They're Form a layer or little column most of the time the groups of neurons that form these ensembles were spread out through the T-shirt through the slides No, so they were located in different parts and they were somehow no too far together and This is in brain slices spontaneous activity. We weren't doing anything to the tissue. They just did that Continuously is we couldn't stop it Now because it was in brain slices many people thought well This must be an artifact of cutting the brain taking a piece of brain out of the of the animal But it turns out we've done the same experiments in in living away behavior in mice and we see the same thing So on the top you see a mouse that's awake. His head skull is attached Head fixed to the microscope. He's running on this as spherical treadmill and he's watching this TV screen where we're showing him some visual stimulation and Simultaneously, there's a laser and infrared places that you don't see that's going through the skull and We're taking data like this is the raw data coming out of the microscope This is the two-photon microscope These are the neurons they're labeled with calcium indicators And if you look at the road it that you can see how these neurons are flashing These are these action potentials and you can now build the this is the analyzed version of that movie Which with color in red the news are half statistically significant changes in fluorescent corresponding to this action potential So this if you look at this movie you see these groups of new and fine together See how the news are not firing individually the firing groups. That's the ensemble. That's what we call an ensemble In fact, I bet you that in your brains right now. That's what's going on and this is exactly the challenge of neuroscience How to decipher this coherent patterns of activity and Understand how they relate to your thoughts to your memory to your how they build your mind This is in a in a guess. This is what neuroscientists the the holy girl of neuroscience to take Understand this this code But the goodness that we can actually measure this in a way behaving animals and long behold as I told you We see the same thing we saw in slices This is the way we analyze this data with a computer program with detect the position of every near in the movie For every near we compute the fluorescent intensity as a function of time here and then using an algorithm We estimate the probability that the news has fire and action potential as a function of time And these are these spike probability plots And then we build a raster plot like the ones that I just showed you earlier in which every line is a Neuron as a function of time and you can see how this raster plot is dominated by these vertical stripes Which you can see when you collapse the activity of all the neurons in this histogram This is a percentage of near that are active as a function of time You see how the activity of the cortex of this mouse at least in our lab in New York Is dominated by the little earthquakes and these are the ensembles This is what I'm talking about this small group of neurons about 6% 10% of the near that for reasons we don't understand them they far together for a little bit and then they Someone else first together and it's essentially this jumping between ensembles that we are finding This happens when you show the animal a visual stimulus like the one that you saw in that movie It's an example of raster plots for great things where we showing the animal this high contrast gratings and the The ensembles are painted in red in other words the action potentials that are statistically belonging to an example with very rigorous conservative criteria are Coloring red and you appreciate that the majority of the spikes are part of this ensemble if we use a criteria which is less rigorous less conservative then practically all the spikes are part of this ensemble and that happens also If the animal is watching a movie that resembles the natural scenes that mice Probably are watching the while we use BBC Documentaries of nature. This is an example so that they represent the right spatial and temporal frequencies of natural images and this also Turns on all these ensembles, but look at the top case. This is spontaneous activity with the light off Either the animal is in the dark or the animal is looking at the gray screen Homogeneously gray screen that doesn't change and notice how this hole was also filled of these ensembles The brain is not turning off if there's no input it continues to fire and it continues to fire with these groups of neurons In fact, if you analyze the position and the properties of these ensembles and their spontaneous activity versus the one That are generated during evoked activity You find that they're the same Okay, so this is an example of two ensembles From a movie of a mouse the neurons in red are the ones that are part of the ensemble This is an ensemble at some point and a little few seconds a minute later. This other ensemble happens But then if you analyze the data on the spontaneous activity of the same mouse when the mouse was in the dark you find That this ensemble has happened before and that's the one in top So the neurons that have the green circle are the ones that are repeating between this spontaneous ensemble and this evoked ensemble Ensembles actually never repeat exactly. There's a liquid quality To this activity so that when it repeats you can see a core of near and they're still The same and that's how you know that's repeating But there's always some new near and that comes in and some other neon that goes out There's some of plasticity in this type of representation But our hypothesis is that we can identify these visually evoked ensembles in this spontaneous activity So the idea is that when the cortex builds a response to a visual stimulus is using as building blocks Patterns that are already there It goes back to these older Hypothesis from Lorente and and head And how feel know that we have these attractors already there and they are doing some function And then when stimulus comes in we line them up in particular ways But we don't come out with a new type of representation. We essentially are using this internal building blocks So this raises the issue of What are these attractors doing these ensembles doing and to test that For us the dream experiment is something that we call to play the piano, which is the following you make movies Like the one I show you where you take You capture these ensembles in blue on the top and then you activate them artificially and Playback as if each neon was a piano key and you play back the same melody that you've seen That you heard by watching these movies and then you ask a question. What happens if you play that back now? If their ensembles are important and we play this back something's not going to happen If there's some sort of noise or epiphenomenon of the statistics of firing in the brain then who cares? No, you don't play back. It's like playing back noise So to do that experiment we needed a way because these ensembles as you've seen are intermix in in these cortical territories So you see how? There's one ensemble another one they're all mixed together near and are near each other can belong to different ensembles So you have to have singles of precision and turn on one ensemble without turning on the other and that's That's something we achieve with with the spatial light modulators building a two-photon hologram in which we can write words with two-photon lasers With a face device This is the four-year equivalent of that word and generate the two-photon word and project that into the brain This is a digital hologram the student who pioneered this Nicolenko is brilliant guy He decided to paint pictures of kaha Get the face picture and project it down into the brain I see if the brain would care that there was a light with kaha's pictures on it That's what the graduate students do when you're not looking around So this is our piano to turn on ensembles and then we also use an obscene a protein that we express into the neurons and This activated by light and makes the near on fire So this is an example of two neurons that are expressing this obscene C1 v1 and with the holographic Piano we can turn on both neurons simultaneously. They're near each other in space. This is the worst possible scenario They're only 20 microns apart And we have to selectivity thanks to the nonlinearity of the photon excitation and this digital holography of turning One neuron on or the other with no crosstalk so we can really play the piano So we started to play the piano In this experiments using putting a mouse In the microscope and two lasers one laser To image the calcium. This is the top imaging path and another laser through the SLM To generate the holography to play back those patterns So we capture the pattern first and then we compute the face mask Put it in the SLM and turn those neurons on in the same order in the same Precision that nature has played it so in the Control experiments at the beginning we decided we're going to play the piano with our elbow Okay, trying to activate many years at once. These are very difficult experiments You have to go through the skull of the animal. You need two to four lasers. The whole thing is really complicated so So in this experiment Luis Carrillo who was a postdoc in the lab decided to stimulate the neurons on the left side of the image All at once just as if he's playing the piano with the elbow and this is the calcium imaging Results and and alone behold whenever he turned on the laser He got all these neurons on the left too far. So he was happy not like I can play the piano successfully with Activating all these neurons at once in one half of the field and then he comes to me and said Rafa Do you remember those this parameter that I was turning all these neurons at once? Yeah, sure. What happens? Well, guess what these neurons are still firing together After I stop stimulating them So it turns out that he discovered by chance that if you activate these neurons 50 to 100 times They bind together they glue together up and they become an ensemble Artificially so and this they start to fire spontaneously Even if there's no stimulus, okay, and that happens even a day later Okay, so this is not a minor change in the reprogramming on the circuit This is already there and still there a day later. So these are the calcium traces of these neurons and The the dotted line represents one one of these spontaneous imprinted ensembles is firing So so this showed us that these ensembles can be created when we activate the neurons with our piano and Then Luis said, okay, let me this is a gross experiment and activating all the news at once Maybe that's I know it's an artifact Let's just play the piano in the piano one year at a time and then he started activating the news one by one and this example he's turning on this little neuron here and Whenever he turns on the laser, this is the calcium trace of that near and then you on fire So he was happy that he was playing the piano one finger and and that was working But then when he played a piano with one finger in the group that he had imprinted before Very often when he played one neuron the entire group came together. So he did pattern completion So this is an example of a pattern completion experiment The neurons in pink are the ones that belong to an ensemble and that near in 25 is the one that when he Plays it it trips the whole ensemble. It doesn't happen all the time But it does happen. So these are an example of the raster plots These pink lines is when the laser is on and it's turning on near in 25, which is this one here in the middle This guy here, so So you turn it on here and nothing happens around here nothing happens But look what happens here you turn on and boom bingo You start to get all these other neurons to turn on and they are the ones that belong to the same ensemble So we hit ourselves directly with pattern completion We build these in samples artificially and belongs to us and they have pattern completion This looks like a picture-perfect attractor like the the whole field model and this we think that the way We do in pattern completion is that we're activating these neurons together and Somehow their connections are getting strengthened Maybe through heavy and rule and then we turn on one and we bring them everyone else This sort of illustrated here in this diagram In which you have a period you can think of this as memory storage. You have a period in which you imprint By imprinting the ensemble really imprinting a memory and then we read it out by turning on only one of the neurons So this pattern completion was very robust it was there even a day later So this is an example of the similar experiment. We're turning on the new name in blue Tripping on these ensembles when there is a star We bring the animal back to the animal facility We test the animal the next day and turn on this new 19 again and bingo. We get the ensemble back So we don't know for how long this pattern completion Occurs is still there, but it's probably in many days. It's a long-term change in the in the circuitry Okay, so that means that these ensembles are not just some sort of fake Result from imaging you can build them So they're physical things that you can actually generate and they have these properties of pattern completion Which are not consistently or some of the key properties that theories had identified in these models of emerging properties But we still don't know if they're they're good for anything if they're used by the brain to donate computation We have to do this piano experiment. I was telling you about it essentially go between the behavior measurement Manipulation and see what happens to the behavior this close this loop closed loop So the behavior we chose is to show the animal Stripes of light in this case vertical stripes and whenever he sees them the animal licks I don't know if you can appreciate the animals licking because he saw these vertical stripes of light and we show the animal horizontal stripes of light We train him not to leak So this is what we call a go no go task and we're showing the animal one stimulus one visual stimulus of the other The animal behaves in one way or the other so in his behavior. He's telling us what he saw This is a very robust train in in one week or two we train the animals to close to 90% success and Simultaneously, we're imaging with calcium to photo calcium imaging in this awake behaving experiments the activity of these neural So So this is the experimental design on The top we show him these vertical stripes of light in blue and that turns on this blue ensemble of cells that correspond to the Vertical stimulus and the animal licks In the middle we show him a horizontal bars a Different ensemble which is in between interspersed with this go ensemble Turns on the green one and the animal doesn't leak. This is the no go and then the bottom experiments like We don't show him anything we turn on our piano and we play back these patterns to see what he does So this is the first experiment with it was to activate the no-go ensemble So in this plot is just one example of one experiment in which the blue bars is when the animal licks The short red bars is when we show him the ghost stimulus So in the top trace every time there is a ghost not every time but many times when there's a ghost stimulus The animal is leaking the tall red bars is the no-go stimulus So when we show him the no-go bar the animal doesn't leak The same animal the same stimulus and we're playing with our piano the no-go ensembles and these are about 10 neurons again in the brain of a mouse that has about a hundred million neurons and When we do that and turn on these neurons the animal stops leaking altogether So we're blocking the leaking by activating the no-go ensemble And this is illustrated here. Maybe it's a complicated slide, but maybe look at this the performance of the mouse This is the behavioral performance the leaking. This might start with 70 to a close to a hundred percent success in leaking for a visual stimulus when we Showing the visual stimulus with playing the no-go ensemble. There's a significant decrease in the behavior We're controlling the behavior and making him stop leaking or leaking less By turning on this the wrong neurons. Let's speak the next experiment is okay now we're going to turn on the right neurons and We're going to show him the right stimulus But we're going to lower the contrast of the stimulus so it becomes really hard for the animal to see that there is in this case This vertical bars of light No, he's the ones you show were high contrast and a percent contrast if you lower the contrast Not even you will be able to tell the difference if there's a bar there or not So we lower the contrast we show him the stimulus the animal doesn't leak because he doesn't see the stimulus and Then in the bottom. We're showing him the same low contrast stimulus and turning on the go Neurons and look what happens the animal now suddenly seeing what he didn't see before It's essentially leaking to this low contrast stimuli It's quantified again in this complicated slide, but if you just look here This is the performance of the animal with low contrast That's why the performance is very low about 50% and if we activate this go ensemble we increase the performance significantly The final experiment is No stimulus we turn off the stimulus we turn on our piano and Then we activate the go ensemble and to do that We use pattern completion. So we turn the neurons that we know our pattern completion neurons So we try to do with one we can do it with one so we did it with two So we're activating literally only two neurons in the brain of this mouse with optogenetics with this holographic device In this case this this two neurons near nine and eight and Whenever we turn on our hologram They fire not all the time this case. They only find nine only fires. I don't know why this is biology They both fire nine is more reliable and most of the time nothing happens, but look what happens here These two guys can trip the ensemble the go ensemble and what that happens the animal leaks. That's the star This is the position of these two neurons the position of the neurons in the go ensemble and the quantification I think you should look at this plot B the performance of the animal when we activate these two pattern completion neurons and We recall the ensemble goes from 20% to 80% So he's not seeing Anything in his eyes. We're just turning on the two key neurons that are tripping that go ensemble and that generates the whole behavior If we quantify that behavior is identical to the behavior that he had when he was seeing this This goes stimulus no difference in the delay to leak the velocity which is leaking the duration of the leaking For all we can see He interprets this as a visual stimulus to leak So in a way, we're putting in hallucination in his brain by turning on these two neurons again two neurons in a hundred million Neuron brain and we presume that this pattern completion of turning on one ensemble is Generating a whole avalanche of pattern completions that is propagated through the rest of the brain until he moves the mouth and leaks So when we're doing this experiment, we notice in one lucky case that there was a mouse in which the go ensemble Turn on spontaneously Okay, so now the animal wasn't looking at the screen. We weren't we didn't have our piano on this go ensemble this case Just lit up spontaneously and in every time that happened the animal leaked So this is an endogenous activation of that ensemble Don't know why but it correlates causally with the leaking of the animal So that's it in terms of what I wanted to tell you so in summary I think we are finding emergent properties of brain circuits as predicted by theories for a hundred years And that these cortical activities are nice in these groups of news these ensembles or attractors that are spontaneous states I didn't show you but individual neurons can belong to different ensembles. So they have a little bit of a combinatorial Quality to them and the temporal spatial temporal patterns are not identical. So they also have this flexibility and That they can do pattern completion Which I think would be critical to understand how the brain works if you think of it as a system of pattern completion The visually walking sample resembles spontaneous ones the ensembles can be imprinted and recall for several days So this is an example that a physicist would call a phase transition with generated phase transition in the circuit by activating a group of news together These ensembles are necessary and sufficient for visual behavior. So they're not an epiphenomenon of the circuit They're actually causally related to visual behavior and our hypothesis in red here is that these ensembles are a unit of perception so here I want to distinguish between what you could call a sensation which is the activation of your sensory system by by let's say in visual stimulus or Sensory stimulus and your perception which is your interpretation of that sensory stimulus In this case we can dissociate both things. We have experiments that I show you where there's activation of these internal states without any External stimulus we can do that artificially or they happen sometimes for the genius spontaneously So that's why we think that we're dealing not with sensation We're the perception and then once you're in perception you're essentially into memory. This is it's very hard to distinguish when you see something between your perception or your Memory of that object when you're looking at it your grandmother, for example, are you looking at her or are you looking at the memory? your memory of her so So I think this could be a essentially right in the line of fire set out by people like Lorente in a way Turing and Happen Hopfield that are these intrinsic internal units the one we should concentrate on because they can enable us to take Apart that edifice of the of the mind and just to finish how this is shows a very close link between the methods and the paradigms the people that pioneered the single-near methods kaha and Sharon turn at the same people that proposed a new endocrine in which you can have a new single-near method to everything is a single near Now if we expand the palette of methods like the brain if he's doing and then you develop methods to watch the entire TV screen of the brain then you can start seeing these emerging properties and start getting a new paradigm coming I Wanted to highlight that this was a lot of work done by mostly a lot of people. I would highlight Joan Miller and Luis Carrillo with the two posts of who did all the critical work on the last decade in my lab on these ensembles We have a relatively sizable team were funded by federal funding sources and hopefully About to sign a contract on a IBM Columbia data science Institute contract to For a conference on brain computer interfaces. I have a conflict of interest a pattern that has to do with this piano business the holography stimulation with using two for the light and I wanted to finish with a quote from My mentor in England Sydney Brenner who died recently who argued that progress in science depends on new techniques new discovery new ideas Probably in that order. Thank you