 So my presentation, robots love JavaScript too. So the two meaning everyone in the room of course loves it and robotics works well. So I wanted to make my talk really amazing so I'm gonna give away a couple of phones. I thought that would help. Is anyone interested in that at all? Okay, so I'm gonna make this sort of a drawing form. So Mozilla Utah provided us two Firefox phones and it's the Flame phone. It's an amazing phone. So I can suggest that you do some things but I can't force you in order to enter the drawing. Why not follow these fine folks, including myself, there's my plug. Why not clone the repo that I'm gonna be talking about? You can also, everyone I think in this room probably has Firefox running on your desktop. Why not open the web IDE and install a simulator? And I will say I'm going to force you to do one thing for me and that is you must join my robot army if you want to have one of these phones. All right, so we're gonna do that by tweeting Flamebot activate. So, and also I don't want this to be a promotion for Twitter. So if you don't want to use Twitter, you can write your name down on a piece of paper and give that to me. It's best if it's green and I'll just leave it at that. Okay, so one fun thing about robot coding is you expand the surface area of your app. So it has to interact with the whole world and you can think of, we're usually writing code that just has a tiny window to the world. It just interacts with even just one kind of person. It interacts in a very limited way and robotics makes that exciting because you have to deal with the whole world and you can literally crash and burn if you don't deal with it in just the right way. And maybe I shouldn't have segwayed that way but if you were with me like waiting on baited breath to see what would happen with Rosetta and Philly, you'll know what I'm talking about. Like this is a probe that we, I say we because I mean humanity, it's the European Space Agency. And so we sent this probe to a comet and we managed to crash a thing on it that's the size of a washing machine. It didn't crash exactly the way it was intended to but it was just exciting stuff and there's so much smarts that has to be on the other end and there's like a 50 minute delay to communicate with the thing. So it has to have lots of smarts. You can't have like opportunity running up to the edge of a cliff and saying, hey, I might be in trouble here. Just let me know if I need to change course. So how do we model behavior? What's a great way to do it? Well, I think one of the ways we tell kids to do robotics is we give them something like scratch and I like scratch but I am a little worried about the scratch being the first way we experience programming because it's a very linear thing. It's a flow chart, a flow diagram and it misses some things that you need when you're modeling behavior. The best way to model behavior is to think of how we have a hierarchy of needs, right? So a person will be very concerned about breathing if that's not possible for some reason for a few minutes and other things take a back seat if we're in trouble and the same kind of thing needs to happen with robotics. And this falls into maintaining robotics code as well because you really need to not accidentally lose something like if you're building a submarine you can't accidentally lose the code that surfaces if your battery is running out. So here's my architecture. I built this little framework and it starts out with announcers on the left and everything is event driven so we're very comfortable with that here in the JavaScript group. So we send events into a bot and the bot takes each event and it delivers it into behaviors hierarchy and this is a hierarchy that has one active state but it's not, it's important to know that anytime you have an active state you have a whole stack. You have all the parent states and they all matter. When you deliver an event to this stack you can get a new state back. If you get a new state then you have to deliver a new message which is entering and you deliver that at the new state. And the whole time we're doing things like processing events and processing the entering messages we do stuff in the world. We drive things, we do things and we move stuff. And event delivery is important because if you have a current state which is find sunken treasure, that's the X and you have a higher state which is surface when the battery is low. If you get an event that says my depth is 100 meters it's not gonna be intercepted by the one that cares about the battery. So that one will, it'll get delivered first to the lowest state then the upper, then the middle. None of these states necessarily would bail and change the state machine. But if you got a message that said the battery is now at 10% it would first be delivered to the parents that grandparent state and that one would act on it and say our new mission is to go to the surface. Okay, so let's look at our announcers. I mean I wanna look at the whole cast of objects that come together when we're doing this. So first announcers. Now I've written my code in coffee script. So everyone take a deep breath. It's coffee script. So listen, Chuck, I know that, well anyway I'm running a contest later so if you make me feel bad I might have to take that into account. So we, a couple of our announcers now these are things that you'll recognize. So one of our announcers says we have to know that time has passed. So every second I'm going to inject an event into my bot that says a second has passed. And window.setinterval is one that we deal with all the time. Another one that we deal with sometimes if you do mobile HTML5 is device orientation. And this one's a little more complex when you construct this compass announcer you give it some optional arguments to show that you want some calibration from the thing. And then as it gets these orientation events it'll adjust those per your calibration and then inject those into the bot. And then the behaviors are the things that we stack up in this complex hierarchy. And they're actually the most interesting part I think. So one, and I've given all of these little character block pieces. My kids love this that I was using their blocks to do my talk. So this one it's a building block that lets you put other things on top of it that are time limited. So if you have that sunken treasure thing and it's on a time limited 10 minute it will short circuit all that and jump out the execution when you reach 10 minutes. And how you enter these states is important. If you enter it I would say from the bottom if you enter it from a parent you're always going to need to do things like reset the timer. If you enter it from a child like if a child says I'm done I wanna go back to the parent then you don't care about the timer you don't reset it. Robofinding is one that you give GPS target to and we'll compose this with a steering strategy. So Robofinding actually just knows about GPS it knows if it's going in the right direction but it doesn't know how to do anything about it. Robosteering is able to do something with events that are course correction events and it has a driver instance so it can steer and drive a car or a robot. Robosequencing is one that I use a lot a lot of this has been discovery so like I discover what I need. So Robosequencing you compose other child's behaviors on top of it and it will sequence through these things. This is another one that's important if you enter from the parent you reset the index of the sequence that you're on. If you enter from the top you increment the sequence so it cares. And then Roboflagging I was able to use this little Lego piece. If you hit this state then we're just going to try to drop a virtual GPS flag and I have this in my sample code and it'll execute a closure that you provide to do something with that. All right so the fun part or the part we get to see is our drivers. So this is the car that's in front of you here. This is called the, it was Brookstone sold this. It's called the Wi-Fi Racer. When you turn it on it shows up as a Wi-Fi access point and they sold it, they didn't sell an app. I mean they built an app for this. It ran on Android and iPhone and you would run the app and the app would connect itself to the Wi-Fi access point and it would let you drive the thing around. Now that's a great start but what we really need is for the phone to go with the robot so that it can be autonomous. So whenever I see these things I always think well what if you want the phone to go with it? So what I had to do was reverse engineer this protocol which wasn't too bad because it's going across Wi-Fi. So I was able to sniff the Wi-Fi connection and find out it sends, it was actually UDP packets of a certain format when you wanna go forward, it's a certain thing when you go back and whatever. And I also discovered that there was some junk that you were getting back on each of these requests and it was slowly changing. This junk was the battery level. So it's pretty cool, it sends you back a battery level every time you send a command. And after some hacking I found that it also listens to a TCP port, not just UDP, which is really useful because the platform I'm working on is Firefox OS and there is no JavaScript API for doing UDP. There's only a TCP API. So I'm using the Firefox, I don't know, you can actually read that on that huge, huge screen, can you? So I've never driven a screen this big. So the interesting part here is opening the socket and you have to deal with the callback and be kind there and then the end of that is where I slice up the response and turn it into a battery level and inject that as an event back into the bot. And then there's also the problem that whenever you send a message to it, you say drive forward, it only works for a second that it stops driving. And it does fun things like if you tell it to go right, it will go forward a little bit, then it'll turn right and go a little bit, then it'll stop and roll forward a little bit. So you have to fiddle with it to make it go where you want. And that's why I do things like I have code to start turning before you go and then start turning after you stop. There's another car that I also like to play around with and this one's called the iRacer. This one was sold by SparkFun and it uses Bluetooth, which makes it a little easier to work with actually. So the phone is able to just connect to the Bluetooth on the chip and the cool thing about that is your cell radio and your Wi-Fi radio are still operating. So you can use those for other things. When I'm connected to the big Wi-Fi Racer, you connect to the access point and it's very unfortunate. It tells the phone, oh, and I have a default route to the internet. So the phone's like, great, I will stop using this cell radio and use whatever you give me. And it doesn't have a default route to the internet. So your cell radio, Wi-Fi, everything is dead while you're connected to that. This one was a little harder to do because there's not a native RFCOM socket communication API in Firefox OS. So I, this is the only place where I really cheated. I shell into the device, you have to use ADB, which you'll recognize if you're an Android developer. You shell into the device and I found I could construct a tiny little web server using NetCat. So I should get my Giver points for this because I thought it was very clever. So after that, then I can just use some Ajax commands to send stuff out onto that socket. This is what my test app looks like. So it's in Firefox OS. And why did I choose Firefox OS? I didn't really explain that. I love the idea that everything is in JavaScript and not just because it's a different way to do things. I think JavaScript has a level of dynamic nature that makes it interesting and I'm gonna get into that a little later. So in order to make this work on this phone though, I know how to use jQuery mobile. So I put jQuery mobile on it and what I did is I used the jQuery mobile accordion widget to display the current state. So that's the black box there. And when you change states, it'll expand the accordion so that you can see where the current state is and it has a little asterisk next to it. So in that top example, the current state is driving and it got there through an event called button drive. So the white section is the buttons that inject events into the plot. Okay, so here's me using the thing. I'll just let, oh, okay, I'm gonna start by resetting this. I didn't plug in the audio. The machine telling it to store a flag here where we are. Should be able to see it in the list. It's point one. Now we'll go to middle point. Okay, so you can't really tell what's going on there but what I did is I hit a button that sends a reset event into the state machine and it's a special case because the response to that event is to reconstruct the whole state machine. And then, so the result of doing that is you don't have any GPS flags in the new hierarchy. And then I hit the button that says go to a GPS flag drop. So it drops a flag in a certain place in the state machine for the location that I'm standing on. Did I say that's my other electric car? So this has happened occasionally where it went off course. I only have one super massive dent in my phone from when it crashed into the curb and threw the phone out onto the curb. But here it is running. After I've put down three of these virtual GPS spots, let's, I'm gonna turn this down, it's kinda annoying. So the steering strategy is getting this course correction, this constant course correction. But what I found is it looks a lot like a bloodhound. It's kinda going like this, like a fish or a bloodhound. And I think it's just because the compass readings are really noisy, like you can get a compass reading that is so far off, you just know it's wrong. I'm not tossing those out right now, but I probably could clean this up quite a lot. Anyone who's getting motion sickness is probably wishing I had already done that. So it's gonna reach a point that I had put down on this grate. And it's gonna seem like it's stopping, but really it's just getting stuck. And then it's gonna go off to point number two here. Declare's victory and runs off to point number two. You could see me for a moment there. I thought I needed to put crosshairs like that's where it turns on the person. Okay, that's another one. Okay, so this is all running in Firefox OS, which is a fun way to do stuff. It's JavaScript native. It's easier than PhoneGap if you've ever worked in PhoneGap, you don't deal with plugins and upgrading the framework and all that. I used to do that all the time. You just put a manifest in place and you open this IDE that's a built-in thing in Firefox, the web browser, and you can go. The great thing is you do everything in JavaScript and the terrible thing is you do everything in JavaScript. So when there's not an API that you need, you have to get creative. So here's an example of what I have coded on it right now. This is just a section of the test of the example app. So this finding state is a place where I put flags. So whenever you push the button that says drop a GPS flag, it creates a new state in the machine above that finding. And then when you hit the go button, it jumps to the finding state and then the finding state tries to run all those states in sequence. So the only thing that makes it, that will stop this is the interrupting state below it, it gets to know when you want to enter a state. When you want to enter a state, you send that message through the whole stack. So the whole stack knows that you're changing state and interrupting notices that you're entering and in this case, it has a little closure that has to evaluate to determine, am I going to interrupt this activity? And the thing that it evaluates is do I have a calibrated compass announcer in place? So the first time you hit that go, it jumps over interrupting. Interrupting in the meantime figures out, oh, you know what, we can't do this. We don't have a compass calibration yet. And it changes the current state to calibrating. And then calibrating is another one of these sequencers. And so it goes through the two second timer that drives forward for a while, the eight second timer that turns in a circle, and then the final state where it registers calibrated compass. And as it's driving and turning, it's collecting data that the registering state can use. So I wanted to do something else that would be fun and kind of push the idea. So I started to construct a game. And I have one phone, and it's running the very same software. I have one phone that can be the controller for the game. And you walk around and you put these flags down in the same way that you do when you're going to drive the car. But then instead of saying go, you push a button that says run the game. In order to do this, we needed a message queue. I wait for all the bots to talk to each other. So I put that on a BeagleBone, and I'll show that in a second. And then on the bots that are on the cars, you put it in a state where it's waiting for the game to start. And then as soon as that command comes out to start the game, then it can start going. And what it does is it sends that list of goals as just an array in the JSON object. And then the players in the game need to reach each of those goals. And actually the way I wrote it, I thought it'd be interesting if someone just needed to reach each goal. It didn't care who. It just said it would declare the last person the winner, the one who got to the last place. So a quick side trip to BeagleBone. And a question about this came up when Josh was talking. So there is a really big difference between some of the embedded boards out there. So a little bit unfortunately, if you go from Arduino to Raspberry Pi, you go from a platform where you write the program and you push the button that says flash it and when you turn the board on, it's running that program. And when you come to Raspberry Pi, you've got a computer. And it's not so simple. You can't just push something to it and have it automatically start and all the things that go along with this. Well, the BeagleBone people did some really interesting stuff with their software. Their preferred scripting language is JavaScript and they have Node.js deployed on the thing and ready to go. You can add these things to Raspberry Pi as well, but the upshot of building it into the platform like this is when someone builds a cape, that's what they call the expansion boards. When someone builds a cape, it's on them to write the JavaScript bindings. And that wouldn't happen necessarily with Raspberry Pi. There are lots of extensions for Raspberry Pi that don't necessarily have bindings in lots of languages we like to use. So in that way, the BeagleBone is Arduino-like because of this, I meant to mention, this auto startup folder. So if you put an app in that auto startup, then when you turn it on, you run that program. And so it makes it more like an Arduino. It's kind of a nice turnabout. So this is my cue that I put on the BeagleBone. And the irony isn't lost on me that I'm sending my phone out in the field and I'm keeping the embedded board in with me and it has all these wires for extension. I'm not using any of them. I'm just using it as a server. So it, but I needed a couple of things from this cue server. I needed to just use polling. So I don't, I'm not trying to do anything fancy with WebSockets. I just do a get, when you do a get to the cue, it looks up your session, see who you are to figure out which message would be next for you. And when you post, it does the same thing. It looks up your session to figure out who you are so it can attach that ID to the message. And that ID is basically the order in which you arrived, you first arrived after I turned on the cue. So it's a little bit interesting. You can't totally trust this. What you could do is, well, if it's the first bot, then maybe that's really the command bot. And if you're like me, you think about this stuff all the time. So I loved it that the police started singing about messaging in bots when I was in the car one day. So this is what the game messages look like. When you start a game, you're gonna have something that says you're the sender and then the game is running and you send out the flags. These are the GPS locations you want the cars to go through. And then you can also send a game over message and say, oh, so no one won. And it's because we ran out the clock or you can send out a message that says someone reached a goal and now the next, go to the next flag. While you're playing, the players need to say where they are. And if you can imagine this, this could introduce a trust issue because you have to say where your bot is. And if you're in a contest, there might be the temptation to have that be not true. So I love to think about all the ways. This idea isn't secure. The cue's secret isn't a secret. You could fabricate session keys. That's probably the hardest attack. I'll talk about the easier ones. All the messages are in clear text. If you reset the, if you jangle the power on my black beagle bone, beagle bone black, it'll empty out the cue. All the messages are in memory. The sample code that I wrote doesn't examine the sender of the message. So you could come along and even though your sender six and the game was started by sender one, you could say, oh, the game is over. You lost. Another interesting thing is there's a cross-site scripting potential. If you can get the phone to open a certain image, you can drive it. And I like to think that that image is, it looks like the car driving off the desk. That's what that image looks like. You can also crash the cue if you sent really big messages. Cloud nine doesn't have any actual password by default. So you could just get into the cue server and change it. You could also zoom to the last flag. You know the last flag is the player who wins or you could just lie and say you're there. Okay, so I kind of think of this as setting base cam. I mean, none of this is revolutionary and it didn't have to be in JavaScript. It wasn't necessary. It could have been written in Java. It could have been written in Objective-C and it would have accomplished the same things. But we are in a platform where people can ask all these questions, is it an object or a class or a function or a map or a dictionary or a hash? And the answer is yes. So I like that this camper or hiker happens to be an extreme hiker and also she knows all about JavaScript. So I wanna make this thing self-modifying. I want it to, well, and we have a start at this, right? It's self-modifying now because if you hit, if you send a reset message in, it modifies the whole stack. You get a brand new instantiation. If you hit the drop of your GPS flag, you're modifying it there as well. But I wanna do more. I would like to make a leap to class definition. This is kind of the hard part. This is making an expression for the peak. So I think what we could do is put in some more advanced UI and let the user design the behavior on the device. And I think there would be some steps to this. Like first off, you would let them choose a class that they want to specialize. And you'd subclass everything. I think this will make it simpler. And then inside that class that they've just subclassed, you would have a member that stores what the child activities are for that thing. And then you would name the class and put it back in the namespace. So it would share the same namespace with all the classes that came when you turned on the thing. The hard thing about this, I think, is the event handlers and the entry handlers, they're where the hard logic is. And it seems like you might actually be getting back to the sort of scratch interface where you're doing a flow control diagram. And even though I think that's not the best way to program a robot, within the confines of one behavior, of one node in one behavior, it's probably a useful way to do it. And this is what I think some of the class definition would look like in code, at least in CoffeeScript and it'd be similar. So I would have to modify the base class so that it looked at this new member variable to figure out what it would construct it at its own construction time, what child classes it would have, what child objects it would have. And then somehow, this second part would need to be parameterized, but you would do something where you name a new class, you put it in the namespace and then you extend something and you also specify that it has some child behaviors. Okay, so that's the hard sort of excursion to the hills. It reminds me of the third man on the mountain movie because I think, well, we know that people have made it to the peak, right? We have systems like fourth and we have small talk and all those that are self-modifying and programmable. And that's kind of what I want this to go. Where I want this to go. Okay. So this is everyone else and robots. I think, I love that talk by Josh. I think that's really fun. John Brown's talk also got me really excited because these are all things that we can build. I like the idea that I could build these without any soldering. I'm not sure if you saw that pattern. I didn't have to wire anything up. These things were made to be controlled by a phone already. And I like the idea that we start with the large surface area mindset and let people think about that to start with and then zoom in instead of going the other way. And I also, I love the make movement. I also do stuff with the maker space and we love to talk about this project. So that's it. Thanks, everyone. All right, any questions? I'm, pardon? Oh yeah, okay. Think of questions while I do this. So the first thing is to turn it on. I don't actually have enough turning radius. So I'm gonna run into this stage. What I can do is tell it to, and the GPS can't be read in here. So what I figured is I would have to tell it to calibrate the compass and just see what it does. It'll try to drive forward for two seconds and then go in circles for eight seconds. So I'll tell you, the first thing I'm doing is I'm telling it to find that access point that just came online. All right, it connected. This is where you put the phone, precious phone on there and hope it hangs on. That was the wrong one. Okay, I mean, I have to turn it this way so it doesn't jump off the stage. But anyway, this is, oh actually the turning radius might be okay. This one tells it to just circle for like five minutes, just keep circling. And then it'll stop and quit after a while. I should have it try to, one thing I could do is have it after it ends, try to come back to where it started. Any questions? Now if you had figured out how to hack my message queue, you might be able to stop it for me. I'll catch it on the next go round. Okay, I have one of those and I'd love to do that. Okay, so the question is, have I done anything with Mindstorm, the Lego Mindstorm? And a lot of those models have a Bluetooth interface and it's one that people know about, people have documented. So yes, that's one of the next things I wanna do is make it so it can drive a Mindstorm because that also qualifies in my, I don't wanna solder anything mindset. So that's perfect. I wish that we had a real Bluetooth socket IO and maybe this will be the time that I need to port it to another platform. Any other questions? Okay, yeah, I looked at the data that I was getting and I think the GPS is a pretty clean signal. That's probably already smoothed out but the compass was all over the place. And I think what people do is, what developers do is they will average out the compass over seconds rather than just trusting all these instantaneous measurements. The only thing about averaging compass is you have to make sure you don't ever average zero and 359. That was a joke. Any other thoughts? Yes, I probably could. All right, thank you everyone.