 Excellent. Today we're going to talk about how to control robots with your brain. Specifically, my brain right now, but we'll get to that. Full disclosure, I work for a company called Pinocchio. I usually work out of San Francisco, and I'm in England for a couple of months visiting. I've done a talk a couple of years ago here about cyborgs, so you might possibly remember that. This is sort of a more in-depth one specific instance of how to control, obviously, cyborg stuff with your brain. Alright, so first off, brainwave control, also known as EEG, Electroencephalograms, which is a device that reads the electrical currents produced in different frequencies by your brain when you're conscious. So you are constantly producing brainwaves across a spectrum of frequencies in different intensities depending on your mental state. So here's a chart showing the general correspondences. What we're going to be working with is alpha and beta waves, so those in the 8 hertz to 30 hertz spectrum. Above 30 hertz to about 40 is gamma waves, which no one really uses for anything that I'm aware of. And then if you wanted to do dreaming stuff, then you would probably be working with delta and theta. So choosing a brainwave reader is the first step you want to go through. So there are a few major companies that produce commercial ones. There's NeuroSky and Immotive are the main two, but there's also some like Muse or the OCC Nia, which are tougher to hack and may be more system specific. So the Muse is a consumer device that I believe just feeds to your smartphone and the data isn't really accessible. This is also something that I call sort of a jardware. It's somewhere between, it's closed, but there's lots of information available on how to hack it and it's fairly simple. So we'll start with the NeuroSky systems. They have three main ones that you will encounter. The MindWave has built in Bluetooth compatibility with a phone or a computer. And so you can use that if you want to simply write your own software to interact with it. People use that with processing too. The MindFlex is one that's marketed as a game. Also is the Jedi Mind trainer you might have heard of, which you control a little levitating ball. This is one that's really popular with hackers because it's easy to get into. There's really obvious screws and things. It's got its own power supply. And also there are various tutorials around. Also you can get it for much cheaper than a MindWave. So for example, the MindFlex comes... There's a kit called the dual set where you have two players facing off against each other. And that can actually be cheaper than buying a single headset. And it's also very hackable. And then finally, NecoMimi is produced by a Japanese company. Also based on the same chip as the NeroSky ones. It's basically a licensed version that includes two servos that control cat ears. And they have their own algorithm filtering on there to make it produce cute behaviors. So it's not as... It's not as sort of pure, I guess. But it does make it a lot easier if you want to build your own hardware that's controlled by brainwaves. And you don't necessarily want to dig into the internals of the device. That same company also produced a little flopping dog tail thing that you might also have seen. Then we have Emotive. And they're sort of the more sexy snazzy company. They have a lot more sensors. So the NeroSky comes with a forehead sensor and an ear clip. And the forehead sensor picks up your more direct EEG waves. And the ear sensor is basically a filtering mechanism. It provides a reference point and ground. So it helps filter out things like the way your blood is flowing and your muscles are interacting. So the older Emotive version is the epoch, which has, I believe, 16 sensors. I might be wrong on that. You can pick up directional movements as well as straight brainwave frequencies. So you can do things like think forward and back and left and right. And even disappear, which apparently no one's been able or no one's used yet for research because they're not sure how to tell subjects, imagine this thing disappearing. But they have managed to make people control physical things in real space. And then the inside is a newer one that they've built. Oh, the epoch is the one that looks sort of like a head squid, like a matrix sentinel wrapped around your head. And then the inside is this sort of sea creature thing with five tentacly prongs that wrap around. But it's not commercially available yet. Supposedly it is much improved. Oh, and they have SDKs for those. So they're encouraging you to hack them, but not physically, just software. And then there's a couple of open source systems. OpenEUG is an older one that you can find information on the internet. It's basically if you want to build your own system from the ground up, which can be awesome because it's open source, but also a lot more difficult because you have to gather the components and it can be a lot more expensive because you're buying everything from scratch. And then OpenVCI is one that recently did a crowdfunding campaign. I don't think they're available yet, but it's also supposed to be for the open source community. So based on what you want to do, you probably want to select a system based on that. I've mostly done NeroSky because it's easier to hack and it's cheaper than the emotive systems. So the first step is, of course, hacking the EEG device itself in order to get it to work with whatever robot you're talking to. I took off of this existing tutorial by the Frontier Nerds team. It shows you how to open up the device. Well, I'll show you this in a video in a minute, but there's connects the NeroSky to an Arduino, which you can then hook into your computer with a USB cable, pull the information off and use their processing graph to see the brainwaves in real time or control stuff. And then I've got a few tutorials. We're going to look at the two videos I have on YouTube. And then also there's a breakdown of the NecoMimi, the cat ear things, which I used to create some brainwave-controlled animatronic wings and horns for a fashion show. And so you can see how that worked online. I should have more pictures on here. Here we go. We're going to watch these with no sound, but I'll just tell you sort of what's going on. Let's see. So this is the Frontier Nerds tutorial that I was talking about. Let's go full screen. And this is the Pinocchio module that I usually use. So as I was mentioning, the Frontier Nerds tutorial teaches you how to hook it up to an Arduino. And this is basically another microcontroller that's Arduino compatible, but it also includes baked-in mesh networking so all of the boards can talk to each other. You usually have like three or four. And so with this hack you basically attach the Pinocchio to your brainwave reader and then have another one sitting in the robot or in the control system. And they talk to each other instead of having to implement your own, either through your computer like a wire connection or implementing your own Bluetooth. So here's the... Let's go back. Sorry. I'll show you the Mindflex controller. And what we've got here is the power switch on top. There's the forehead sensor there and the ear clip, which I've hacked into this set basically moving the chip into this pair of headphones and the sensor here and the ear clip as you can see. The Pinocchio boards and then some tools that you'll need to complete the hack. So you start by opening up the case and I'll just skip through this really quickly. But here you have the internals of the device. You've got the power systems coming through the headband and the signal down here with the blue and yellow arrows. And then so the blue arrow points to the ground pin on the controller itself and the green arrow is the transmit pin, which is what you're going to hack. So instead of allowing it to transmit via Bluetooth, which it does already, you can keep using this as a toy afterwards, but you're basically slurping that signal out of the board and piping it to your own controller. And that basically says exactly what we just did. And here's the connection of the transmit pin. And there's the ground pin. And you're closing it up. You're on the Pinocchio board. You're going to hook it up to... Oh, pardon. I'm not used to using YouTube videos. Here we go. So the ground pin from the board itself goes into the Pinocchio's ground pin. Same with Arduino. It goes into the ground pin there. And then your data cable goes into RX1. So you're receiving the data that's being trended from the reader. And this is me controlling a little helicopter with thoughts. You end up with this wonderful sort of meditating guru look on your face. Or you just look like an idiot either way. And then on the programming side, you can program it straight through the Arduino IDE. You basically load up some libraries. FrontierNerds has their own library that's called the Brandt Library. And you throw it on the Pinocchio. And then, yeah, pardon. And then you get a data feed that looks like this. All right. So what you're getting out of the reader itself with the Mindflex is you have... I'll just show you up here. It's easier. They have range. Excellent. So the first... You basically got 11 positions out of this. Eight types of figures. The first one is signal. It goes from 200 to 0. 200 is zero signal. And zero is best signal, weirdly. And then you have attention to meditation, which is two filtered EEG frequencies that they've run through normalizers so that attention is basically your beta values, normalized from zero to 100, with 100 being the best, because that makes sense. And then meditation is your alpha value. And perhaps some other ones that they've thrown in there, but they don't open their algorithms. And then the next eight are the various frequencies. So going from delta to up to even high gamma. And they don't correspond to any specific scale, but you can use them to compare to each other over time, or to themselves over time, or whatever you want. I usually don't use those though because it's easier to go from one to 100. Zero to 100. I think that's about all we've got on this video. Oh yeah. And if you want to check it out, I also include the commands that you put on the Pinocchio device itself. So there is pulling the attention value, that second value that you saw in the previous screen, and sending it out to the other scouts. And then running that once per second. So the Neurosky, here's the point. With either the mind wave or the mind flex, it's pulling the data once per second. So if you want something that's more responsive, you might want to go with either a lower level hack, which you can pull the raw data by performing some extra little tweaks on the hardware side, or go with a different controller. But this provides some sort of smoothing from data point to data point. And then here's the response that you put on the other board, which is basically setting a blue LED's brightness based on the strength of the signal. There's some extra information in there that I can just tell you about increasing connectivity. So with the brain wave readers, you always have a problem of getting a good signal out of it. And that's based partly on the strength of the brain wave reader itself, but also on how you connect to it. So when I first hacked apart the mind flex, I thought that I had destroyed it and completely failed in preserving its ability to read data, because I wasn't getting a good signal out. And then I added EEG paste, which basically improves connectivity. And then you learn to clean off your contact areas, including the sensors and your skin beforehand to get off any grime. And that helps a lot. And then it turned out that that worked better than the original sensor, which was a little disheartening, but also exhilarating. Presentation. Where am I? Apologies. So choosing a robot, you want to decide what sort of... Well, first off, you want to decide obviously what your motive is. If you want to make a flying robot, if you want to make an exploding robot, if you want to make a little crawler. And ideally all of them at once, because you can control all of them with a mesh network. And it's arbitrarily extensible. So as I mentioned before, you've got a hardware that you want to look for. Things like the chip isn't painted over, so you can actually see what it is. Or visible screws, translucent casing. Anything that makes it easier for you to see if the cables are exposed and easy to hack. And then you want to decide whether you want to control the robot directly or remotely. So they often have a built-in remote control. Like Hexbugs, for example, control with infrared LEDs. And so do lots of little gyros. That's awesome if you want to hack the infrared LEDs directly. You can pull down some Arduino libraries that will do that for you. Decode the signal coming out of the remote and then play it back. But it can be easier to just either modify the remote control to press the switches directly. Or to mount it on the robot itself. But when you do that, you want to consider how much weight the robot can carry. So with a little gyrocopter, you're not going to be able to mount a full microcontroller on there and still have it be able to take off. Same with the Hexbugs. Their motors aren't strong enough to control, to both move the robot and your microcontroller on top of it. The signal range. For example, if you want to control something through a wall to freak somebody out like a co-worker, then you're not going to go with infrared LEDs because you can't shoot them through a wall. But you can shoot radio through a wall. So you might want to do the microcontroller over there and pick a big enough robot to do that. Power control also. The microcontroller comes with its own battery. And you can run some kinds of hardware off of there. You want to consider how much power your microcontroller itself has. You can also use transistors if you're modding the robot itself to switch between... to basically use the microcontroller's digital and analog pins to switch the power system to the robot itself. We can go into that further. But we might not have time. And then finally, depending on how pretty you want it to look, if you want to mod the remote, it's a lot easier to keep it hidden, especially if it's for presentation. Or if you want to mod the hardware itself, then consider what you can fit in there and how big the robot itself is. Okay. So on the hardware side, there are a few different ways you can mod it. You can do this straight hardware mod without even touching the software. And that's what I did with the brainwave wings. The NecoMimi have servos up here to control the ears. And you can pull those down out and put them on whatever part of your body you want. Just make sure that you connect the cables in the same place. And then that just preserves their original algorithms and such. You can bypass the brain and hook your microcontroller up to the motors and servos and things and LEDs. And you might need specific Arduino libraries to do that, but it's all pretty open and easy. Or you can just completely lobotomize the robot and stick your own brain in there. Which is basically the same. It just doesn't preserve the original control system. So if you want to still be able to use it the first way, you probably want to bypass it. But if you want to think about weight, you probably want to lobotomize it. And then the remote side, you can use transistors to control switches or relays. And you can... LEDs. Oh, the infrared LED hack that I talked about. Okay, beyond that, I basically wanted to go into how you can optimize your interaction with the device. You can improve meditation by obviously, if you're better... If you're experienced at meditation, then you can go to your happy place or still your mind. Deep breaths help with that, picturing some bucolic scene where you're at ease. And then for attention, there's a few different interesting ones. So you're trying to maintain the highest level of focus possible. We can't yet do it off of emotions. It's just like focus or relaxation. So for focus, I've found that thinking of large numbers and adding them or multiplying them together helps. Or oddly, writing emails does it. Or if you're listening to an unfamiliar song and trying to type out the lyrics. These are all ways that if you're trying to do a stage presentation and your hardware hasn't conked out at the last second, then that's what you would do, speaking of which. So I want to do a hardware presentation for you. Unfortunately, that hasn't been the case. But if anyone has model rocket motors, I would love to talk to you. Especially if we can figure out something goat-shaped, I had a special surprise. But I also have a little mini helicopter that I'll be trying to get going over the weekend anyway. And feel free to hit me up with any questions or see how that's going. I'll give you a demo later on. All right. For now, there are any questions? All right, what's up? Ah, snazzy. To make it a bit concrete, to understand a bit better what controlling means here. You spoke about meditation and so on. Which is the bandwidth you can actually transmit? Oh, with the radio control? No, the actual signal, not the radio bandwidth, the control bandwidth. If I attach it to a simple system, a LED, binary LED, how many yes or no binary information are you able to transmit per second or per hour, whatever, doing your meditation or focusing or whatever? How much information can you actually transmit? So you get those 11 channels, the signal strength, attention, meditation, and the eight frequencies, and it only reads those once per second, or it only distributes them, I'm sure. Yeah, but at the beginning, which control you have on the data that you are sending, are you able to switch LED every second, every minute, every... Oh, like the microcontroller itself? Like your brain. Oh, my brain! Because controlling with the brain at some point, you should... I'm not sure what the bandwidth of the brain is. That's a good question. I want to look that up. And if you shoot me your contact info, then we can get back to it. My email is on my website, or it's alex.glowasci at gmail.com. I'll give you my... Glowasci is a bit tough to spell. Here we go. That. I don't know if I want to speak for this gentleman, but I think what you were asking was, if you were controlling a light with your brain, what kind of control would you have over that light? Could you control the brightness? And if so, what kind of speed of reaction would you get? How many lights would you be able to control with the systems that you're using? Oh, well, okay. So you could control the brightness. You could control on or off. So I've done it two different ways. So for the brain helicopter, right, I set a threshold. And when you're starting out, you can set it lower. You can say if it's above 70 or if it's above 80 for five seconds or longer, then turn on the light and you get this binary control. Or you can do a zero to 255 thing scaled based on the values that you're getting out of the sensor itself. And then you can control... We've done up to a hundred at least nodes on a single network that you could control from the same reader at once. If you wanted, you could also have multiple readers on the same mesh network controlling different things. And the data rate on those is adjustable. I need to check on the exact data there, though. When you say that you were able to control up to a hundred, are they like individually addressable or are they all respond to the same? Yeah, they're individually addressable. You can control them all as a single group and say everybody turn on your LED red for five seconds. Or you can send messages to specific scouts they're called in a troop or mesh network. You can send them to subgroups among them. You can send out a single command and have different scouts respond to it differently. If you wanted to use the same parameter, for example, send an argument with that, and then each one of them would take that and do something different. Does that answer your question? Awesome. Okay. Well, feel free to hit me up later. So you say you can control all these different things and adjustable, but I mean, if you just stood there without using your hands or anything, can you sort of set one of them to really bite, and then another one of them really dim without touching anything? You could if you programmed it that way. In that case, it might be easier to use an emotive set, because then you could provide different things like left and right and up and down. But with the NeuroSky itself, you can use the signal value, the attention value, and the meditation value independently. You might also, with some of the NeuroSky systems, you also get blink data. So if you blink once or twice, you could have them like different scouts to respond to different stimuli. But in terms of changing what happens on the fly, you'd have to program that in. Is that practical, or do you have to be some sort of trained ninja? You do have to do some brain training. So I gave you a few ideas on how to control the attention and meditation, but it does take practice. And in terms of the programming itself, I personally think it's pretty easy to get into. We've worked to make it very accessible, and there's lots of tutorials, but we also encourage you to hit us up. I wouldn't say necessarily it requires ninja status, but it could be awesome. You can do more complex stuff that way. Yes, so you mentioned brain training. Exactly how much of that do you have to do? Would it be possible to... I know that you mentioned that you can go from low to high, but if you tried hard enough, could you eventually get to the point where you could do a low-high medium? That's something that I've been asked before, and I've never tried to do myself because I'm not that good yet. Not good enough to do that. But in terms of starting out, I think the most important thing is realizing what you're controlling and what that feels like. So people aren't sure at the start if I make myself really excited and move around, will that do it? Just figuring out what triggers those numbers to rise for you, I think is important. One interesting thing is that focus and relaxation, they're sometimes opposed and sometimes in sync. It depends on what's going on. If you're in the zone and you're working on a programming project or whatever your art of choice is, then you can be really relaxed and really focused at the same time, and then someone calls your name, and suddenly your focus is still really high, but your relaxation just goes spikes down. Basically, whenever something grabs your attention, you get that sort of reverse spike, which is kind of awesome. Yeah, that's it. Like a traffic light to warn coworkers? Like green if I'm just screwing around on Reddit? You mean doing Coffee Creek? If it's red, it's like, oh, that question actually wasn't so important. I'll get back to you later. Maybe. You could have it like, you're losing my attention, you're losing my attention, say something interesting. That actually has happened. I've had this on in a couple of different meetings and I thought it wasn't working at first, but maybe it just wasn't very interesting. And then somebody says something about robots or whatever, or what else am I interested in? Music. And suddenly it's like the thing turns on and starts going. It's actually really distracting, so I don't recommend that. But pause, please. Oh, yes.