 Thank you for the nice introduction. I've got two different microphones here. So wave them away back if you can hear me. Excellent. OK, yeah. So what I do, I'm once known as a computational neuroscientist. So what that means is I make computer and mathematical models to try and better understand how the brain works. And in addition, I think the brain needs a body. And we'll talk about why that's so. So I do a lot of robotics work, too. And as Rita said, I'm professor in the cognitive sciences department and computer science. Just to give you an idea, because you guys are all incoming freshmen or transfer students, is that right? Yeah, so give you an idea of the courses I teach. I teach in the psychology department. And the main course I teach for the psychology department is to fulfill the experimental psychology requirement. And it's called cognitive robotics, where we go through a bunch of experiments programming. I don't know if any of you have ever used the Lego Mindstorms. That's what we use to program Lego Mindstorms. People understand sort of what I mean by embodiment and then also learn how to do some programming. And typically, I've had students, there's a limit because we only have so many robotic kids. But typically, I have a few students. We make room for a few students from departments outside of psychology. The other course I teach is a graduate level course called computational neuroscience. And that's really detailed mathematical models of how a brain works. And although it's a graduate course, usually there's a few undergraduates, typically in their junior or senior year, that sometimes take this course. So just to give you an idea of what I teach, today I'm going to be talking more about what our lab does for research. And in general, we're trying to model how the mind works. And it gets into this idea of synthetic methodology. So what does that mean? That means the best way to understand something is to build it yourself. So the mentor or mission statement is understanding through building. And primarily, I'm interested in how the brain works, how the mind works. So that's why we're doing it. But there's also some practical reasons for doing this. Current artificial intelligence systems or robots are quite brittle. If you look at them, they don't show anywhere near the capabilities of your pet dog or cat. So there might be practical applications in making something that was truly intelligent in the way we think biological organisms are intelligent. And so there's going to be in the next decade or less a bunch of neural-inspired applications in computers that are made along the architecture of how the brain works. And these intelligent machines show lots of improvement in manufacturing, homeland security, health care, all sorts of applications that I can't even think of yet. But I think this is a new wave, and it's really an exciting time when there's a merging of brain science and computer science right now. Okay, how many of you know what Moore's Law is? Anyone? Wow, all right. Moore's Law, I think it's Gordon Moore, who is the founder of Intel that makes most of the chips in all of our computers. And he had this law or idea that every so many years, every couple of years, the amount of processing that the chip on your computer can do doubles. And you see this exponentially increasing curve. That seems to have held for all these years when we had computers. That we're always doubling the amount of computation. And so a guy named Ray Kurzweil, I don't know if anyone heard of Ray Kurzweil on the singularity? I will, there's some philosophers that are almost sonnistic, and they're thinking that once we get the computer chips that can actually process as many out of computations per second as the human brain, then we're gonna have human brain intelligence. And he calls that this point here, which is at 2050, actually in 2023, the singularity, where machines will surpass humans. And as you see in his idea, machines don't always surpass humans, they get bored with humans, and then they populate the universe and yada, yada, yada, and they become intelligent. So it's not a bad story. They don't take over us like the matrons or anything, but it is a lot of thought. And I personally don't think it works that way. And I'm gonna try and make a case for that computation alone is not the key to intelligence. Okay, so how many of you know who these guys are? Watson. Watson, right. And so that was a really impressive landmark that IBM created a computer system called Watson that actually beat these, let's see, Canada grads who were the reigning champions in jeopardy. And it shows what a lot of people thought was true intelligence. So do we think Watson is truly intelligent? Well, I think there's a problem with that approach. And that Watson really is an intelligent, it's actually, and this cartoon actually drives on the point that Watson needs a whole bunch of computer scientists to program it. Watson needs its own power source. You know, behind the curtain here, there was a whole bank of computers that was sucking up a lot of energy. So we, it's really not autonomous, it can't do things on its own. Someone had to actually hear the question and then type it in so that Watson could understand it. It's a very important and impressive display of intelligence, but I don't think it's truly what we would call biological intelligence. So what makes, I think, biology intelligent? Well, this is sort of our mission statement in my lab and a few other labs around the world. The brain is embodied and the body is embedded in the environment. So what do I mean by that? It means that the brain doesn't work in isolation. It's not like a Watson computer chess program. The brain is really tied to the body. The body is really telling the brain what to do. And what do you want to do? Well, you want to survive in the world. So that's your environment. So there's a close coupling between brain, body, and environment. And I think someday we can get really, truly intelligent robots and systems, but I don't think you can ignore this fact that the system has to do something in the world and actually has to be on its own in the world and be able to be autonomous. So given that, what's the most important thing about how the brain works? Does anyone have an answer? Well, I have an answer. After many years of studying and talking to brain scientists, I think that it's the anatomy of the brain. So it's how the brain is structured. So both the body has a structure. We have arms and legs and we have senses like eyes, nose, and ears. The brain is quite structured to take advantage of that. And the brain is a beautiful architecture. And this is how I got through neurobiology and neuroanatomy when I was a student, using the human brain coloring book. But the brain has 100 billion neurons. And this is the human brain and has roughly 10 to the 15 connections between those neurons. So a million synapse, a synaptic connection in the brain. And if you unfold the brain, it rains quite wiggly and coldly. It's like a large dinner napkin in size with just a few millimeters thick. And all of that is doing all the processing that we take for granted. But that's how we do intelligence, cognition, mind. It's all going on up here, but it's closely tied to the brain. And it's all very structured. So if you look by an accident of evolution, I suppose, the eyes are in the front here. And they go through a lot of brain areas to get to the first level of cortical brain processing, which is back here in the back of brain. This area in red, the parietal lobe, takes that information and does a lot of spatial things with it. And it also does a lot of abstract things. It's very important for mathematics. This area in blue down below it is the temporal lobe that's very important for our hearing, but also for our long-term memory. And then the yellow part in the front, which really differentiates us from other mammals, is the frontal cortex. And that's very important for planning, decision-making, and a lot of important behaviors that we have. And then this area down here is the cerebellum. And that's got almost as many neurons in that area as the rest of the brain. And that's very important for fine motor control and dealing with precise timing. Now, actually, how many of you have taken a class in neuroscience and biology, isn't it? Okay. How many of you know what a neuron is? All right. And you know that synapse is a connection between two neurons. Okay. Yeah. So I don't want to give you too much of neuroscience lesson. Or if I say something that you don't understand, please ask a question, because when I'm talking with all my colleagues, we throw around a lot of jargon. Sometimes I even forget I'm using it. Okay. So my colleagues and I have put together a book to really capture both the brain structure and then also using robots as a tool to tie the body to the brain structure. And this is my friend and colleague from Japan, Hiro Wagatsuma, and myself and a bunch of other people around the world who contributed to this idea of building neuromorphic and brain-based robots. And you see this beautiful robot that was developed by some colleagues in England that has whiskers. You can see those lines are actually whiskers based on how the rats typically are moving around in the dark. So they don't use the vision very much. They use the whiskers to fuel around the world. So over the years in talking to my colleagues, we've come up with a set of design principles of what you need to build a brain-based robot. And so obviously you have to incorporate how the brain works, some model or simulation of the brain that has both the dynamics and the structure, the neuro-anatomy. And then brains, unlike Watson, brains don't get all the answers ahead of time and put them in a store. You have to figure them out for yourself. The world doesn't come with a bunch of labels. There's not a label that says this is a chair or this is a lecture room. You have to figure that out for yourself and put it into categories, sometimes without any supervision. So we need a way to do that. And obviously, I think the brain needs some sort of physical instantiation. So it needs to be in the world. It needs to interact with the world, touch it, feel it, and be able to act in, push the world around it and kind of close this loop so it can interact with the world. And finally, we need a way to adapt. So brains are quite adaptive, we learn. That's why you're coming here to go to school. You're always learning and your brain is always adapting to what you learn and what the environment, how the environment changes. So we need a means to do that. And then putting all that together to see if it really works. What we do is we compare it with a biological system. So we see if our robot has the same behavior as a biological system. And then we see if we look inside the robot's brain is the brain of the robot doing the same type of signals that would if I actually could stick a wire in someone's brain and record from that area. All right, so given all that, here's sort of a past history of some of the robots I've worked on. This one here on the upper left is doing a visual experiment. And this is actually a tough experience coming to the side between different colored shapes like these red and green diamonds. And you look at all these little flashing lights around him. And without going in detail, each little pixel is a neuron in his brain. And so the brighter the pixel, the more active that neuron is. So it gives us a really great tool we can record unlike with a real animal, we can record from every single neuron in this robot's brain throughout its lifetime. And then see how its brain changed over a bunch of learning in different experiments. And this is a classic experiment called a binding of visual features. On the upper right is a robot we made that was modeling an area of the brain called the hippocampus. And the hippocampus is an area that's very important for long-term memory and in rats especially for navigation. And this robot was doing the equivalent of rat famous rat maze task called Morris water maze. We designed it without the water though, so the robot wouldn't fry up. And you see if you're watching that after a bunch of trials, now the robots learned where it is in the world and how to get to this spot here in the environment. And that spot you can't see. So he's taken us on our robots outside. So this is a robot we designed that is modeling the area of the brain called cerebellum which is very important for precise motor control. So this robot would initially crash into things like the walls and the bamboo there, but after a while it learned to predict when it was gonna have a crash and make a nice movement not to crash. And this robot was really cool. It not only went forward and turned, it had what's known as omni-wheels, so wheels within wheels. And it actually could stain or slide. So it's actually a different, difficult motor control problem because it had so many degrees of freedom the way it could move. And then finally several years ago when we started all this together, we built several robots on the same way scooter platform and had teams of our robot is this guy here, and that's his teammate. And we played a series of matches against Carnegie Mellon and we got a Robocop soccer. And this put together ideas on how the brain does vision, how the brain does decision-making and playing against conventional artificial intelligence robot. And did quite well. I think we won every match that we played. Because these robots are so expensive and kind of dangerous, we got 300 pound robot balancing on two wheels. The league went to funk, so we're raining, sigla, soccer, and robot shoes. So what is the word? Hey, that gives you an idea. The final goal of final match. Give you an idea of the kind of things we do. Some of them are more practical, some are indoors in a lab, some are outside. So I'm gonna talk about what's going on in our lab currently. We'll see how much I get through, but just give you an idea of what's driving the research in my lab and one particular aspect of brain-based robots that we're interested in. And that has to do with the value systems or adapting behavior when an important environmental event happens. And in brain terms it's called neuromodulation. So brain neuromodulators can really have a broad effect on how your brain works. And they're very important for just regulating how you get through the day, but also making important decisions that in animal events critical for their survival. And this is your brain science lesson for the day, is the neuromodulatory systems are actually old in brain terms. They're sub-cortical, so all this folding and wiggly stuff up here is the cortex or neocortex. And below that is the, hope I can work this thing, is sub-cortical or under the cortex. And these areas in color are the different neuromodulatory systems. So the upper left is the cholinergic system, and that's using the brain chemical acetylcholine, very important for attending to things. The upper right is the dopaminergic system. Dopamine is very important for wanting things, how you deal with rewards and punishment. The lower left is the locus serulis. That's an area that's also important for attention, but a different type when something surprises you. Do you respond to it or do you ignore it? And then the lower right is the raffae nucleus. That's the site of the serotonin system. And serotonin is very important for mood disorders. Prozac is a serotonin reuptake inhibitor. It's very important for how you deal with risk and cost and harm. Now you notice these little, these things are small. So I said there's 100 billion neurons in the brain, human brain. These are on the order of tens of thousands of neurons and maybe up to a million in the case of the cholinergic system. So very small, but if you look at those arrows, they project all of the brain. So they have a very strong influence on what your brain's doing. And without going into too much detail, I tried to put together a wiring diagram of how these things are connected. And this is actually not complicated enough to capture everything. But the main idea is areas of the brain that we think are important for cognition, like the preferral cortex, the hippocampus, amygdala is very important for your emotional state. All of these areas actually project down to these neuromodulatory areas and they modulate the modulators. And so that's a very important aspect. So you have sort of cognitive control of your mood and of how you're gonna deal with the ward and you can predict when these things are gonna happen. And that, especially mammals and primates, that becomes a very important tool for planning ahead. Okay, so a little more brain science and then we'll get to what the robots do. So the neuromodulatory systems seem to have two different modes. So one is tonic. So what did I mean by that? So if you record from these brain areas I've talked about, you'll see that they'll fire what's known as an action potential. So a neuron communicates by having a very brief, one millisecond spike. And that's how they communicate information from a pre-snatch neuron that has a spike. It goes down a wire called a maxon to a post-static neuron. And then that post-natch neuron gets that event and decides if the issue was spawned or not. So when the neuromodulatory system is tonic mode, they're firing once to six times a second, not much. And the organism is kind of distracted but it's still doing its behavior. It's not sleeping. When something important happens, these systems have a brief burst of spikes that within 10 to 20 milliseconds, they fire at a rate of four or 50 times a second. So very briefly they go bup, bup, bup, bup. And when that happens, the organism is quite on task and it tends to whatever happened in the world. And then becomes decisive. And there's a trade-off in mathematical systems called exploration versus exploitation. And this is very important. So a lot of times you want to exploit what's important in the world and take advantage of that. But you don't want to get stuck always doing the same thing. So there's an advantage to not always doing, exploiting what's going on in the world, just exploring, being curious, trying something new because the world's always changing. So if you don't get out of that sort of what mathematicians are called local minima, your chances of survival to a changing world are very slim. Okay, I won't go into this too much deep built just to see what a brain recording looks like. So here's a neurodinergic neuron, a don't do anergic neuron. And this is how neuroscience is typically plotted. So you see that there's tonic activity, something happens, it starts firing much more rapidly, very briefly, and then goes back down. And these both plots show that they respond when the animal was actually doing well at the task, they had this phase of response. And they also show that this time course that's different stream, basic and tonic. The other thing that happens when neuromodulatory systems are basically active, they sharpen our senses. So the plot on the left shows that putting the neuromodulator neuron, an effort on areas of brain for vision, it actually sharpens your vision. And the right one shows if you put another neuromodulator serotonin on a neuron that's processing sound, it actually sharpens its response to sound. So it has a way of actually sharpening our senses. And that's important. When you have to attend to something, you really wanna focus on what it is and discern it. So it seems like it's a brain's way, not only to be trade off between I gotta be real decisive or I gotta be curious, but also when I'm gonna be decisive, I have to really sharpen my senses. So it seems like the brain's way of doing that. And then one thing which I'll come back to in engineering terms, that's called increasing the signal to noise ratio. So what does that mean? So let's say I'm in a noisy environment, there's a party going on, I'm trying to have a conversation with you, and but there's a lot of different conversations going on. We're very good at picking out that one conversation on many. If you just type a recording back, it would sound like a mess. It's very difficult for an artificial system to do that. But we do that by sharpening our senses, sharpening our hearing and sight to what we're trying to concentrate on and filtering out the rest. So we're increasing the amount of signal and dampening the amount of noise in the environment. And this seems like one of the brain's ways of doing that very important function. Okay, there's an equation and we'll move on. And that equation actually is a very simple model, a somewhat simple model, I shouldn't say simple, but an interesting model that allowed us to show that there's a mechanism, a mathematical mechanism that can capture this decisive movement. So let's say I'm choosing between door number one and door number two, and I can't choose between them. And then at time step 50 on this plot, I turn on my neuromodulatory system and suddenly I make a decision, a very decisive move to choose door number two. And that's what this shows and this plot on the right of all pretty colors shows that that actually holds over wide range of values and I've tried the mechanism that I think is going on the brain versus other mechanisms proposed by mathematicians and the brain mechanism seems to be the best one. Yeah. This might be a little off topic, but have you heard of the movie, Linnoclas? Yeah, I just saw it a few weeks ago. Okay, so do you believe that it's possible using 100% of the brain to heighten your senses and your skills and what not? I think there's ways to heighten the senses in the brain. Actually, we are using 100% of our brain. You know, when I was a grad student, we were always taught and no one knows where the statement came from, the statement like we only use 10% of our brain. Actually, we use all of it. The thing about Linnoclas that a little bothered me, but they got to it is there's drugs coming out to heighten brain senses, but there's always a downside to them. The brain is very nicely tuned, it's amazing. Especially if you try and build one yourself like I've been doing, it's amazing how well it does under a wide range of conditions. But it's a very delicate balance. So you throw that out of balance, you enhance something, but you can get very adverse effects. And you see it in the, oh, I won't get away with it, but let the movie for those of you who haven't seen it. But it was an interesting movie in that concept. But we are using 100% of our brain. There's certain things that are more active at different times. They've even done studies and found that the brain, when you're just sitting there doing nothing, they put people under our brain scanner that there's tons of brain activity when you're just doing nothing. But yeah, there's ways to heighten the brain activity both chemically and both through experience and constantly using your brain. I firmly believe in the use it or lose it, especially as you get older, that keeps your brain very active and working well. Okay, so this is our working model. We have sort of a theory of how neuromodulation is exploiting environmental events and then exploring new behaviors. And we're looking at these four different systems in the brain, serotonin, which seems to be very important for how we deal with cost and risk. Dopamine, which seems to be very important for how we seek reward. Norepinephrine, which seems to be very important for how we deal with surprising oddball events. Amacetylcholine for how much effort will put in attentional effort when there's a noisy environment. And there's a trade-off where when you're in this tonic mode, you're exploratory and curious and that's not a bad thing. But when something really important happens and one of these systems has that phasic response, you better respond to it quickly because it's important to survival. So then you exploit that information. All right, so now, we've got our model idea, we've got our theory, and we wanted to test it in our lab. So our lab is called the cognitive and theater robotics laboratory after all. And our robot in the lab is called Carl. And there's our first version of Carl on the left. And I got here in January 2008. So I got here in January 2008 and with Brian Cox, an engineering friend of mine, we put together the robot Carl, which is actually a simple robot. It's got a camera so I can see. It's got a bunch of sensors so I can feel. You can't really see it, but it's got a whisker so I can also use it to do it. But what Brian did, Brian was quite creative to make this robot called Disco Floor. Turn it off so I don't have to yell over the VGs. This Disco Floor is actually really nice. It can flash one of six different colors. And we can program it in any way we want. And also the floor can speak to the robot. So it's got like a remote control system on your TV. So if Carl is over one of those panels, we can signal to Carl that, hey, you're on a panel that's good or you're on a panel that's bad. Or, and so in that way we can get Carl rewards from the environment through his exploration of this room. All right, so we use this in a bunch task. I'll show you two, two different robot tasks with us. And we put together a model of how we think the neuro-modulatory systems affect the brain. And I won't go into the detail, but in these ellipses there are a bunch of neurons. So this is one of my small and old. It has about 10,000 neurons. And all those arrows are synaptic connections. But they represent thousands upon thousands of connections between neurons and the two ellipses. So this has like 10,000 neurons and over a million connections between the synaptic connection between the neurons. And a lot of those connections just like the real brain are plastic. So they change as the robot experiences the world. And they pretty much follow a rule that's called neurons that fire together wire together. So if I have a neuron that's firing and sending message to another neuron causing it to fire and they're coherently active, then I'll make that strength stronger so that this neuron has an easier time causing this neuron to be active or fire. If I have a neuron that's out of sync with this guy, I'll make that connection weaker because there is no reason they should be running together. And that's the way the brain does things. On top of that, these neuron-modulatory systems, when something important happens and these two guys are firing together, then it really makes that connection strong because that was something important. So that's all put into this model. And Carl has two different things to do. It has to treat anything. It has to explore the world, like I said. But then it has to learn what's costly and that's triggered by a serotonin system. And we just arbitrarily picked red as a costly thing for a debonair. And then what's good or rewarding. So every time he's on a green panel, we get more reward. All the other colored panels, we got no signal at all. There's no value to those things. And on the left, you'll see Carl's behavior. On the right, you'll see the neural activity. And like I said before, each little pixel is a neuron in Carl's brain. The upper row are areas in his visual system that are responding to different colors, green, cyan, magenta, red. On the bottom are action areas to either find, approach something, or run away from something, flee. And then this is the dopaminergic system and this is the serotonergic system. So Carl just will wander around looking for good and bad, see something red, doesn't like that. So he's learning that red is bad, backs away. And then shortly after red, it turns green. He remembers, hey, green is good. And he goes up and he'll stay on that panel as long as it's green. He's learning that green is good. As long as he's on there, he's getting that reward he likes. And the other thing is, if you notice, originally the green was actually driving the behavior. So he saw green and he remembered green is predictive of something good. That triggered his monetary system, dopamine, which said, hey, that's rewarded and singled to the whole system. There's something rewarding out there. Ignore everything else, let's go to the reward thing. And then when it turned magenta just now, said, well, magenta means nothing to me. So he ignores when he moves on. Just to break this down, you see, at the beginning of the video, the panel went from red to green. So he has a decision, do I stay or do I go? And once he realized it was green, it drove his dopamine system. And this is that singled to noise idea I was talking about. When the dopamine system came on, it turned off all the distracting colors in the environment. So cameras are really noisy, so there's all sorts of colors in the world. So he just ignored all the other colors and is concentrating on what's important to him, which was the green in the room and approached it. And then conversely, this is when he saw the red panel, wasn't sure what to do, that his serotonin system came on, and then he said, oh, red, I got to pay attention to red. So he always keeps his eye on the red panel, but he sort of backs away. It looks good actually when you're in the lab, because it looks like he's almost weird. He's keeping his eye on it, but he's nervously moving away. So this nicely showed that these systems were necessary for doing proper behavior or making that choice to exploit the environment if something important is around. And it also had this signal to noise ratio that I talked about. It's really an increase in attention. So it's forcing the system to attend to what's important and ignore all distractions. And we think that now we have a nice working model for how that works, how that's going on in the brain. And it has very important attentional aspects. And one thing I won't go into is what you could do with these systems, not only can you record from all the neurons in the system, but you can also cut out parts of the brain. So we've done what's known as simulating lesion studies and matched it to data about organisms that are lacking as part of the brain or have this brain part cut. And so we get nice agreement, but we have lots of control. We can just turn off parts of the brain and then turn it back on later. Okay, let's see, how am I doing on time? This is one section we did. We'll go through this real briefly. Just getting an idea of a collaboration that we do with people who are actually working with real brains and real animals. And so I'm working with a group at University of California, San Diego, Doug Nitz and Andrew Achiva. And we're trying to understand two of these systems, the norepinephrine system, the coloneric system, how do you do uncertainty in the world? So there's expected uncertainty. So how many of you have ever gone to Vegas? Oh, wait, I didn't think any answer had gone out. But anyway, you're gambling, you sort of know there's odds. So you have a certain level of expected uncertainty when you're playing a game. Different games have different levels of uncertainty. If the game or this situation is quite noisy, then you really have to concentrate harder. And so you increase your attention. Cedocolins seem to do that. But let's say the rules of the game change suddenly. Something suddenly changes. That's really unexpected. It's an unexpected uncertainty. And that gets flagged by the lotus rule system or the norepinephrine system. So we have two different systems actually flagged with two types of uncertainty that are important for our day-to-day activities. So we worked with the UCSD friends. We came up with a task that we could do on both rats and robots. And basically, you have a ring of lights. And the rat put in the center of the ring, or the robot put in the center of the ring, and one of those lights flashes. And then the rat has to run out, hope it knows where that light flashed, and then come back to the center, and then we give them a cheerio. Rats really like cheerios for some reason. So they'll work hard for cheerios. If I poke the spot right next to where the light had gone off, it doesn't get a cheerio. If it just sat there, it doesn't get a cheerio. So we can change the distribution of where these lights are going off. So I can make it so just as one light is always going off and there's not much expected uncertainty, or I can make half the lights going off with centered around this light, there's a lot of expected uncertainty. Then suddenly I can change the center of the lights to maybe down here. And that's a change in unexpected uncertainty. That was a big surprise. Rats do very well at actually figuring this out. So one of my students, Mike Avery, made a model of this system of how the brain areas will work. And there's just a simulated recording that actually matches nicely with the real recordings from these rats. And then actually this is the behavior. So about 60% of the time our model was choosing the correct light when it went on and off. About 30% of the time it was not getting it right. And about 10% of the time it just was sitting there not knowing what to do. And actually this matches really nicely with the performance of the rats that we tested. So he's very nice in all the behavior. And now we can do like I said, those artificial lesions, the brain areas that would be hard to do with a rat and predict which of these areas is contributing to which part of this behavior. Okay, and the final study I want to go over is a human-robot interaction study. So typically in movies around the United States we always have the robots being evil. Terminator, Matrix, you name it. But there's a lot of friendly robots out there. And especially if you go to Asia, the culture is much more entertainment. The robots are much more part of the culture and friendly robots. But here's a picture of some friendly robots that a couple from US movies. And we're very interested in how, because robots are gonna be part of our lives, so how are we gonna make them interact with us? And so we thought, well let's look at something called economic theory, because that's a study if you go into economics or some branch of mathematics, that's a study of how we deal with things where they have both payoffs and costs and risks with it. And so there's a bunch of games and they're well studied. Maybe we're gonna see the movie Beautiful Marnie. Yeah, and the guy's name was Nash. Well Nash was, he won the Nobel Prize in economics for developing game theory. And there's a thing called the Nash Equilibrium which is a mathematical study of how these games sort of settle into a particular spot. So this is one of the games that actually is a Nash Equilibrium too. It's called the Hawkdup Game. And essentially it goes like this. Let's say in the world there's a resource of some value and if I want that resource I have two options. I can fight for it, that means I'm a hawk, and when I fight the terminology is I escalate a fight. Or I can share it, which is cooperate and that's called display. So I have a choice between escalating or display. Now let's say you're playing against me. If I wanna fight for it and you fight, we're both hawks, we get in a fight and we both get hurt. Sometimes you get seriously hurt, sometimes you get a scratch, so we both get hurt. Let's say I'm a hawk and you're a dove, you wanna cooperate, I just take the whole resource, I get everything for myself. Let's say I'm a dove, I say I wanna display, I wanna share this, and you display, then we just split in half. So I get more points if I fight, but I lose if the other guy fights, but if I play nice with you, we'll actually always get something positive just not as much. So it's a nice trade off between competition and cooperation. So we designed this to work in our lab and set up a display of Carl's panels. And when the light is magenta, that means the resource is open, and when it turns green or red, that means we're fighting for it. And if it turns blue, that means that we're going to share. And so that's the one the person at the computer is seeing. And then to the right, Andrew, my students, Andrew Zalabar is the one playing and Derek Asher is there pointing to Carl. And so you see, Andrew has turned the panels red. That means Andrew's gonna fight for that resource. Carl is thinking about what to do. And when Carl gets over those panels that are a different color, then Carl will change the panels to what he wants. And so I think in this case, Carl looks at the panels, seeing that Andrew escalates a fight, says, hey, I'm not gonna let you take this resource for your own. And Carl turns it green, which means he's fighting back. And then the resource is open, and Andrew decides this time he's going to share by turning the panels blue, and that means he's gonna share and he's doing a display. And so now Carl will go up and decide to do the same thing. And this time they'll cooperate and share the resource. So they go on to play rounds of games like this. And we made a model of the brain areas. Again, dopamine energy system, serotonergic system of how a brain might be choosing between these costs and rewards. And then we tested this model against in different environments and against different opponents. So like I said, a fight is a chance to actually get just a scratch or get seriously hurt. So in an environment that's harsh, where there's a very high chance of getting seriously hurt, like this lower row, Carl learns to actually play nice. So Carl tends to cooperate when the environment is harsh because it knows it's gonna get seriously hurt if it fights. And vice versa, Carl will tend to be much more aggressive in a world that the chance of getting seriously hurt is small. But Carl plays against different types of strategies. So some is a person will pick a fight 25% of the time that's statistical. Sometimes it's a person who does what's known as a tick for a cat strategy. So if I fight you, then the next round, you'll fight me back. If I say I'm gonna cooperate, the next round you'll cooperate. That's tick for a cat. Another strategy is win, stay, lose, shift. So if I do an action and I get a positive payoff, I win, then I'm gonna do that action as an extra round and so on. Soon as I lose, then I'm gonna switch my actions. So that's the shift, win, stay, lose, shift. And notice Carl picked up that if it's a tick for a cat player he's playing against, that as soon as he starts fighting, the guy fights back. So it's not to his advantage to fight someone who's aggressive, so he learns to cooperate. So very nicely it responds to both the type of opponent he's playing and also the type of environment he's in. And I won't go into too much detail but we can do those simulating lesions. If we lesion his serotonin system, we get a very aggressive Carl. Tends to fight all the time. And that's important for the next part of the story. Okay, so luckily you guys weren't here last summer because we were recruiting subjects for this study and this was a really tough study. We probably won't do it again. We'll probably do other studies that aren't so invasive. And maybe you'll participate in. But this is known as an acute tryptophan of lesion study. So how many of you know about tryptophan? Tryptophan is in a lot of your foods yet and it's an amino acid and you only get it from diet. Well tryptophan if you do the chemistry is a precursor to making serotonin in the brain. So we make our subjects do a low protein diet the day before they come to our lab and then they come to our lab and we give them a protein shake that either has tryptophan or doesn't. And I can drink just about anything and I can move down this shake. I don't know how, it was so nasty and so bitter and so chunky. But we found eight brain cells to do this. And normally that we had to take their blood before they took the shake and after to see if actually their tryptophan levels did drop and they did seriously drop. So we found, we had eight people take this. We had 10 but two actually vomited after taking the shake. It's in the milk from the study. So I'm very impressed that we found people that would do this. And they played against Carl in these games, a couple of games, well I'll show you just one. But they played against Carl in a series of games. They played against Carl when they had their tryptophan depleted or when they had full tryptophan. It was blind to them and they played against versions of Carl. So sometimes Carl was that aggressive version. Sometimes Carl was that more normal version. And the main thing we found after doing all this was not the tryptophan levels and not whether you're playing Carl or a simulation but whether they thought Carl was playing fair. So when Carl was, when people were playing normally, a normal cooperative robot, they were tempted to do that Wednesday lose shift strategy. And when Carl became quite aggressive and fighting, then they switched. They stopped doing a Wednesday lose shift and they started doing tick-for-tack. And we thought that was very odd because when you do tick-for-tack and you're fighting back, you're actually getting less. It's not to your advantage, you know, economically. You're not getting as much payoff or money from doing that. And what we think is that the reason people are changing their strategy is they thought Carl was treating them right there. So it was a form of retaliation to Carl. So they thought that, hey, even though I'm gonna lose this round, I think you're treating me unfair and I'm going to retaliate back. And people respond, we're looking at the individual differences. That was the main thing over our eight subjects. People actually have other individual characteristics the way they're approaching this game. But that was an interesting thing. And something we have to think about because Carl has no means to retaliate. So what is that? Something I wanna put in the next version of this model and then what is the brain area for retaliation? So that's gonna take some investigation. Okay, so just to wrap up, we're kinda getting you a flavor of what we're doing in our lab. Some of the questions we're asking about brain science. And we sort of are putting forth a theory about this trade-off between exploration and exploitation and how different brain chemicals can have an effect on that. And we think there's a practical reason to do that, not only for understanding the brain, but also building a system and I can do this kind of trade-off, I think would be quite beneficial for our artificial systems. This is a team, I barely do any work anymore. I sort of just watch what these really talented and wonderful people do. I talked about our robot. So Liam Bucci is our engineer in the lab. He keeps the robots running. He's making a whole new set of robots. So he's wonderful to have. Mike Gehry, I talked about his work with the Uncertainty Project. Derek Asher and Andrews Aldebar did the Human Robotic Interaction Project. And the other guy that I didn't have time to talk to today, but there's a whole other aspect of the research in our lab where we're trying to make a computer that's more like a brain. And then for the human subjects, I've never run a human subject. I was trained as an engineer. So I really need help from some really great cognitive scientists, Lisa Brewer and her student, Brian Barton. And then finally, the mission statement, the brain is embodied, the body is embedded in the environment. So I really believe that if you're gonna understand higher brain function, you can't take the brain out. It's not working a vacuum. It really is closely coupled to what's going on in the world, what the world's giving its senses and how you act upon that role. And I think this idea of brain and spider robots is a beautiful, powerful tool to understand how the brain works. But I think it's also gonna lead to one day, the type of brain models, one of the type of artificial systems that we'll call truly intelligent. And it'll get us to this guide that's on the right, artificial robots that actually is what we call truly intelligent. So thank you if there's any questions, I'll take them down. Thanks for your attention. Yeah. We had a question about a trip to science study. I've been working at Brewer's lab and we've heard that there were issues with the IRB. Okay, so you're feeding this chunky solution to people that's gonna complete a chemical and they freaked out? I guess I'm just wondering. Well, I don't wanna go into the whole IRB of this, but the institutional review board. And the idea of the IRB is to make sure that these studies are safe. And what they really had a problem with is you don't wanna ever done anything like this before. So they really know where to do it. You only do brain imaging, you will do other types of study. This study didn't fall in any category. And the main thing was that, and the main other issue, which we made a lot of control, I didn't go into, these people had to be very, we had to screen them very carefully to remove disorders and interview them, to make sure no one's on drugs, you wanna add history. And it was very invasive. So we think they actually helped. We had to build a very long, I don't know how many-page protocol, what we have to do. So looking back, and given that that was my first, it was wonderful. Lisa Brewer, her lab days, really worked on the protocol. But looking back, saying this is the first study involving robots, humans, and a very invasive treatment, there's probably a little too much to bite off on the first try. So we're gonna try and simplify things. And also cut down the number of subjects we could have. So once we're got around to what we're doing, and they, blood and the dying everything, we couldn't find the subject. We really liked the numbers we were on 30 subjects. Sure. You had, about the pit attack strategy, and this reactionary, what part of the brain do you think might be working with that subject? Or do you think we're just strictly from a chemical? Do you think it's lindit, or amygdala, or how do you think you're deciding to- I don't think I have to think about that. Yeah, limbic amygdala might be very important. Areas of the brain, such as the anterior cingulate cortex, which seems to be doing conflict resolution, is probably important. Certainly, orbital frontal cortex, which is very important for reward, I think is important. And all those systems I just named are highly innervated by these chemicals that I've talked about. And all these systems I named project back down to those systems. So it seems like lightly targets. But I have to live, I mean, there's- It could be a complicated interaction. Yeah, it's always a complicated interaction. That's the way of brain science. We're gonna brain scientists, neuroscientists, and computational people are gonna be busy for many years, because there's so many open questions. And any time you look for a simple clear answer a few years later, you realize it's not clear. So, yeah, we'll be investigating it and probably coming up with some crazy ideas. So if you had two of the exact same robots, they come through a test to see how they learn. But the only difference between the tests are that one stimuli, there's like a spike away. So if you only like variables of time, would they learn differently? I'm glad you asked that. We've done cloning experiments, because we can. And what I did, I had a, one of my robots I didn't show, used to play with blocks that had patterns on it. And I actually could give them different words, same blocks, but in a different order. And they learned differently. And then they also, you looked at their brain and their brains were structured differently. The learning in the brain can change the structure. So it really gives you this notion that there's definitely a nurture aspect. Because these are simple systems. We still saw the different characteristics. And then the robot you saw that was doing that maze test, we saw individual characteristics. So there's lots of ways to solve that task. You can go directly to the spot in the room. But a couple of robots just would go towards one wall that had a color. And then just go right past the spot and then bounce off, let's say the red wall and land on the platform. And we realized that that was actually a strategy as optimized. So out of 10 robot subjects, they all, they learned three different strategies or more. So you see individual characteristics, just having a noisy environment, even though we try and control it, you see some that are better than others. So we see some that are really good performers, some that are really persevering at something, some that are more adaptive. Yeah, over the years, I've just realized that the environment really does shape how we work. And for that reason, I have to do, what people who do animal studies do, I can't just do one robot in brain. I have to do multiple subjects just like an animal study and look at the populations. Let's thank Professor. If anybody has a programmatic question.