 I'll tell you today a little bit about something that we do in my lab. I'm a professor at the National University of Singapore. I'm a neuroscientist by training. And what I work in is to try to understand how the brain works. Normally, the main line of research in my lab is related to understanding what makes humans and other animals intelligent. And we look at these at the level of neural networks in the brains. The thing I'll tell you about today is something that we leverage this knowledge to kind of apply for brain-machine interface. And I'll give you a very light paintbrush of the type of things I will be doing probably for the next, hopefully, hundreds of years in this area. At least my best guess I'll try to give you. So first of all, what is a brain-machine interface? It's any technology that enables the communication between the brain and the machine. Brain, broadly defined. Machine, broadly defined. I'll give you some examples of machines. Okay, so before we go into any boring technical detail, I'll tell you, I will use Hollywood to help me with imagination here. So what kind of things can we do with brain-machine interface? All right, so most of the work that is done around the world today is related to restoring lost functions. So if you become blind or you lose your sense of hearing or you get a spinal cord injury or something like that, you're losing something, whether it's a sense or the capacity to move. So because today most of this research is funded by biomedical research companies or government, this is the main focus today. What do we want to achieve? So this is someone, let me, okay, so this is a movie called Robocop, in case you haven't seen it. This person lost his hand, so he's an MPP. You cannot hear? He's playing the guitar. Anyway, so there is some fancy brain-measuring technology in the background here, and he's able to move his robotic hand in a way that basically is useful for his life, right? So in this case, he can play the guitar. And it's very touching, apparently. Anyway, so this is something that we're not too far away from. And this is, I'll show you where we're at. But why stop there? Why stop with what evolution or God gave us? We don't need to only have two arms and two legs. Maybe we should increase our capabilities. So this is an example from Spider-Man. This is Dr. Octopus that has, in addition to his two arms and two legs, he added an additional four limbs, because why not? Our brains are big enough to do this kind of thing. So that's fine. We could also imagine other scenarios where we can perhaps control remotely robots in a way that, in a two-way interaction, so what is called a closed-loop brain-machine interface where you get sensory input, in this case vision touch, and you can also control. The things that I'm showing you now, and this is another example from The Matrix, where it's the same as avatar, except that instead of controlling a robot in real world, you can control it in a virtual world. And the principle and the nine, all of this that I'm showing you is the very same, which is ways to interact with your brain, inject information, in this case from the senses, and at the same time, control something in the world. So we do this all the time. We have an ongoing brain-machine interface right now which is interfacing with our bodies. Our eyes are sending information to our brains. We are moving our muscles, which are robots, right? I mean robotic devices. Nothing much more smart about a muscle than about an actuator in a robot. So, and the examples I'm giving now are all kind of like the lowest level example. We're dealing with sensory and motor systems. These are useful because they're easy to observe, easy to empathize with. I will not tell you anything about the other things, but most of what our brain does is not about sensory and motor control. It's about thinking and about experiencing pain and pleasure and memories and all of this. All of this is fair game for brain-machine interface and eventually we'll get there. My guess, within the next 100 years, we should get here. But what are we doing today? Okay, so as I mentioned, we need to measure brain activity and in the time I have, unfortunately, I will only be able to talk about different methods that we have today to measure brain activity. And if I have some time left, I will tell you briefly about methods to manipulate brain activity. Okay, so normally when we're talking about the brain, what we want to do is extract information from it, right? We want to know what you are trying to do and what you are perceiving. And the brain is a structure that is localized. So different parts of the brain are processing different types of information. So when we are thinking about extracting information, we want to optimize two different types of resolution. In the y-axis here is a temporal resolution. If you are acquiring data very fast, that's good, because our brain works at a millisecond scale. Your thoughts, your movements, these are all generated within tens of milliseconds. So that's the time resolution that we want. In space, we want to be able to have a good spatial resolution because two adjacent parts of the brain may be doing very different things. So we want to know where the activity that you're measuring is coming from. So in this plot, if you are along this side of the plot, you will be very good, good temporal and menstruational information, and less information in this other side. So this is all that we have today to measure brain activity. Okay, maybe not all, and giving a few of it, but these are the main methods that we have today. In green here are the non-invasive methods, meaning that we don't have to cut skin to, or cut anything to measure brain activity. And in red are the ones that you have to put things inside your body to measure brain activity. Of course, we would want to not have to cut anyone to do this. So the first, most of the work is done on this area. But unfortunately, you can see, this is a quadrant that has the least information. If we wanted to have the best information, that would mean going to individual cells because we're assuming that there are good reasons to assume that individual cells are the ones that are conveying most of information. And for that, we need to insert needles in the brain, and I'll show you a little bit about that. But first, let's see about non-invasive techniques, what we can do with it. And everything I want to tell you about today is related to controlling the robotic hand in the first week. If we wanted to do that, I will tell you that these techniques are not good enough. For other things, they may be good enough, but not for this, okay? So let's say EG. So the EG, in case you're not familiar with this, it's a very simple technique, very old. It just involves putting an electrode in the head on top of the skin and measuring electric fields that are produced by the brain. And the analogy that sometimes people give is something along these lines. If individual neurons in your brain are people in a stadium that are watching our football match, your EG would be similar to a microphone that is a kilometer away. So you will be able to hear the people talking, but you won't know that Bob is very hungry and he wants a hot dog. But once in a while, someone will score. And then this guy will be like, okay, something happened. So what EG is measuring is this kind of activity, the synchronization of millions of neurons in the brain. And whenever this happens, you see certain oscillatory frequencies with high power and you say, okay, something happened in the brain and you try to interpret that. But we may be missing a lot of information, right? We're missing all of the things that these people are saying. Okay, so this is an example of, it's a bit old now, but I'll show you newer results. So this is an EG cap. It's a Japanese group a few years back. And what he's trying to do here, he's given an instruction, it says right hand there. So now this person in the cap starts thinking, right hand, right hand, right hand. He's trying to control the robot's right hand. He's still thinking, right? Now the computer is collecting all this noisy EG data. He's still thinking, he's still thinking. All right, eventually he will get it. All right, there we go. Now he got it, the information had enough, there was enough information there for the decoding algorithm to decide, okay, I can say he was right and not left. This is the simplest possible way of decoding something. You have two alternatives. One has to be the case. You're trying to decide which of the two is the case. So you can see that even though this was a big, it was important at the time, it just, it's not practical, right? You cannot be waiting 10 seconds to do something. So this is a bit of a busy slide, but never mind, this is a meta analysis done this year. And looking at all the studies that have used EG to control lower limb control. So to allow patients with tetraplegia or some form of spinal cord injury to be able to work again with an exoskeleton. Because it's a simple control, right? You all you have to do is say move to move and that's all these patients need. But unfortunately in a rating measure that they have, all of these methods, all of these papers, using the most sophisticated methods that you can imagine, they just don't do very well. So in conclusion today, we can say that EG-based systems as of last week cannot be used for real-time control. And we have good reason to believe that this will stay like this forever. We need to find a different method. So you will see once in a while, by the way, I Googled this morning, because I give this talk regularly and I was like, okay, maybe something new came up, I don't know. So I found this. It's a, I will not show you the video, it's kind of lame, but it's a motive inside, which is a one electrode or a few electrodes EG system that controls the movement of this Tesla back and forth. Anyway, I think given what we know about this technology, I don't know anything about this implementation in particular, but there is a lot of, you have to be careful when you see this kind of things. Don't, by looking at these, most people would think, okay, so we're moving towards a situation where I don't need to use my hands, just think and the car will move. No, sorry, that's not gonna happen with this technology. At least I know things. Okay, so I will quickly go over some of the other commonly used techniques. Functional magnetic resonance imaging, this is a technique that is very popular now in cognitive neuroscience. It's a technique that allows you to measure blood oxygenation in the brain, in local parts of the brain. So what happens here is that whenever your brain is active, you're thinking of going to eat McDonald's or getting a cup of coffee somewhere. And then the part of the brain that is involved in that thought process uses oxygen, activity of the brain uses oxygen. So our brain is smart, blood flow increases to that area and blood flow is a bit slow. This is the time course of a blood change, oxygenation level change. So you can see for a small, very brief stimulus, even like a 10 millisecond stimulus, you will see a 10, 15 second change in blood flow. So you can already get a sense that the temporal resolution will not be too great with this. Let's see what's the state of the art in brain machine interface with fMRI. So he stretched the hand here, thinking a bit better than with EEG, but okay, he got it. So EEG gives you very good temporal resolution, really crappy spatial resolution. fMRI gives you good spatial resolution but very crappy temporal resolution. So both of these are not very good for this and probably will never be used for a sensible brain machine interface. Okay, as of, I wasn't going to talk about this, but this morning again I went online and I realized that all of Twitter, at least in my Twitter feed, it was going super excited about this new MEG system that was just developed. So MEG is similar in a sense to EEG except that instead of measuring electrical fields, it measured magnetic fields. And normally MEG is also a bulky machine, I'll show you here. So that's the MEG machine that you can find in a hospital today. So you can imagine that you kind of walk around with a brain machine interface with that. But researchers developed this portable MEG that apparently is super awesome, signal to noise ratio is excellent and you can localize a millisecond accuracy in a relatively good spatial resolution. So anyway, I know very little about this. It just came out last week. It's entirely possible that it will not go anywhere but it's also possible that it wouldn't be a revolutionary technique and in a few years we will see this everywhere. Anyway, this is a field that is moving quite fast and there are lots of different groups around the world working on new methods to measure brain activity and to manipulate it. All right, so since we already explored the non-invasive ones and none of them really work, let's go to the golden child, especially the one in my lab. We measure brain activity using electrodes that are implanted in the brain and we measure single unit activity, single neuron activity. This is normally done in monkeys, rats and humans. So in my lab I normally use monkeys. I'll show you a little bit about that. The first one that I will show you about is an experiment that was published last year and the goal of this was to provide people with spinal cord injury or with ALS or anyone that kind of move around, provide them with the capacity to move a wheelchair just by thinking about it. So initially what we did was train a monkey to control a joystick and move this platform around and chase a trainer that was holding a piece of apple and they were just following the apple basically. Then once they were trained, we implanted this arrays of electrodes in the primary motor cortex. This is a part of the brain that directly controls the movement of your arm. So we are listening to what the cells are saying if they wanted to move the arm. And this is the array, so it's a few hundred of these little needles are put into the brain. So this is assumed into one of the tips of one of those electrodes. So these are metal electrodes that are completely insulated except for the tip. So we're measuring electric fields just around the tip in a few microns around. And if you're lucky enough, so this is an electrode, this is one neuron. We put them blindly, so we just put them in and hope for the best. If it falls near a neuron, we will be able to measure single neuron activity and this is, in a second I'll show you how it looks like. So this is a setup, the monkey is sitting in the chair, he has a joystick here, the robotic platform, these are amplifiers. So at the moment there is no, the technology is not available to implant everything under the skin and just allow the patients to walk around in their lives. At the moment, and I will show you where we have everything is external. So we have external amplifiers, external wireless transmission and all this. Now in ASTAR, in NUS, and in many other parts in Europe and in the US, lots of groups are working to miniaturize everything, putting it all under the skin and then just releasing the people into the wild. So these are like, each one of these little boxes is the signal from one of these little wires inside the brain. It's hard to see, but a lot of these have these little deviations, those are individual action potentials from one neuron. So these are zooming into one of these here, each one of these yellow lines is one action potential. So as the monkey is moving around, you will see certain cells that will become active. So say when the monkey wants to move to the right, certain cells have selected it to the right so they will become active and so on. We know that it's there because you cannot do any movement in your life without activating your brain. Everything we do has to be, it starts in the brain. So we know that the information is there, we just need to extract it. Now we are collecting hundreds of neurons simultaneously and this means that we have a hundred dimensional matrix that we need to decode. So we need to use some dimensionality reduction techniques like PCA and then we can just use linear decoders to determine in real time what the monkey is trying to do. So I'll skip all the development, it took a while, but eventually the monkey learns how to use it. Now we remove the joystick. So you can see there is no joystick there, he's just sitting there with his hands. We're measuring the brain, the part of the brain that controls this arm but his arm is quite still and he can still control the robotic platform almost as well as if he was controlling it with a joystick. So this is already at a stage, at least in the monkeys where we can reach a point if we were able to implant everything, they could use it. I mean it would be just as good as if they had an arm. This is another example from a different group, this one in the US. This time it's the same technique, everything is identical, except that instead of moving the robotic platform they're moving a robotic arm to fit themselves. So now you can see this is an intact monkey, so he has the two arms, they're just in those tubes. And the monkey is, so in this sense this is approaching a little bit like Dr. Octopus in the second video, he has these two normal arms but he can also control a third arm. This one is a bit harder for him because it's 3D motion so it's a bit tougher. They have also tried this in humans, so I'll show you, unfortunately humans we, so in monkeys we can work with them every day and they get really good at understanding how to control the brain-machine interface. Humans come into the lab maybe once or twice a week, they work for an hour, then they get tired and they go home. So they, so far they are not as good as the monkeys to control it. So you can see here she's trying to, she also has the electrodes so they're implanted in the brain, this is the amplifier. And now she's trying to control this, you can see, eventually she will do it but it's a bit clumsier than with the monkeys. So at the moment today in Singapore we are in collaboration with engineers and National Neuroscience Institute where they have patients that are willing to go through these kind of trials, we're trying to export this technology to patients basically and whatever they want. If they want communication, movement, we're kind of like tailoring to their likes. Okay, I'll skip this but eventually she gets it. Okay, there she goes. Okay, so I mentioned also that our arms and muscles are kind of like a robotic device because they're physical systems that we're moving with our brains. So what happens if you're not an amputee but you just don't have the capacity to move your arms, say if you have nerve damage? It happens a lot in Southeast Asia, especially in countries where a lot of people go in their bikes, they fall, they hit their shoulders and all their nerves are cut and they're left with an arm they kind of feel with, they kind of move. So to try to fix this, so in this case, this again in monkeys, it was a reversible inactivation so you inject an anesthetic into the arm and the nerve stops working. So the monkey here is trying to play basketball, grab a ball here and put it in here, he cannot because his arm is too weak. Now with FPS on, they were recording brain activity from the same part of the brain and instead of controlling a robotic arm, they're stimulating directly into the arm, into the muscles. So you can invoke a specific movement that you want to achieve and now he can play his basketball. Now this has been tried last a couple of years ago in humans, so again, not as nice, but anyway, so this is without any help. So this is a patient with spinal cord injury, he can still move a little bit, but he's quite weak, so he's trying to turn, he got the bottle and turn it into the container and he cannot. Oh yeah, okay. Anyway, he will be able to do it later on, let me skip this. Okay, so I will leave it at that for methods to measure brain activity. Again, we're just scratching the surface. The second part that I will not talk about is how to manipulate. We can also inject activity into the brain by using the same electrodes in this case or non-invasive techniques. Each one also has its shortcomings. And I don't know if you have seen this before, last video, I promise. This is an application of stimulation, which is called deep brain stimulation. It's been, in this case, is a Parkinson patient that has tremor. So this is without the stimulation. This is stimulation in the basal ganglion, which is a part of this patient. So this is with stimulation. So you can see, for motor symptoms, it's quite striking. The same thing has been applied for depression, for chronic pain, for whatever. I mean, they're trying everything now, stimulating everything in the brain, seeing what they can fix. But the main idea is the same, you're just activating parts of the brain that would naturally not be activated with these electrodes. And we'll leave it at that. These are some of the institutes in Singapore that are working on this. And we'll take any questions if we have time. Okay, one question. Yes? Yeah. Are there sort of open source communities online hacking on this? Lots, but it's mostly on the non-invasive stuff. And it's primarily, there is a little bit on brain machine interface for control. But as I showed you, it's a tricky area to be in because the signals that you're dealing with are very noisy. There have been certain applications like rehab for motor rehab or maybe to help people, kids with ADHD to concentrate and things like that. But it's not like straightforward. And then on the other side, there is a huge community online, scary large community that is involved in stimulating the brain. And that's a bit tricky because we don't know what are the long-term effects of this kind of thing. You're actively manipulating something that we don't really know how it works. So we actively discourage this type of thing, although some of these techniques are also used in the lab. Even when they're used in the lab, they're used with extreme precaution. I've had any Google keywords that you can give us or communities that you can find us too. So for the stimulation, the main technique that is used, and I guess the keyword that you can use is direct transcranial stimulation. Direct current transcranial stimulation. There are a few companies that are working on EEG-based systems for controlling, say, video games or things like that, like emotive is one. But I'm sure there are plenty others who are not familiar. Yeah, it's a very simple technology in a sense. The hard part is cleaning the data. So it's a river of raw time. There's also open BCI community, which is going to be there. And they do measurement part of it. So it's going to be a three-degree test of open BCI, brain, brain, computer, etc. OK, thanks. Thanks a lot.