 Anyway, we'll get going. So my name is Joe Grand, I'm Zosbrooks and we are back for yet another fun entertaining session about prototype this. So oh, I guess we should ask, who has seen prototype this, which was a TV show on Discovery Channel? Okay, so that's more than last time, I think. Yeah, and it's totally awesome that people who haven't seen it still come to this talk. Yeah, why did you come to this talk? You'll be excited. Cool. So yeah, anyway, prototype this was a short-lived TV show on Discovery Channel. 13 episodes filmed over like 18 months. Yeah, it was from start of pilot to end of shooting, it was almost exactly two years. And it's, yeah, so we filmed it in 2008 and it aired for a little while in the U.S. and then started airing all over the place and then kind of disappeared for a while. Now it's revived itself on Netflix and Pirate Bay, which is awesome. So definitely check it out if you haven't seen it. And it basically was four guys building crazy prototypes of stuff and we're going to talk about some of those crazy prototypes. Yeah, and with not very much money or time, there was sort of like this artificial time limit that the network made us do to sort of make it seem more exciting, even though often we took more than that time. And there was, they forgot, they kind of forgot to budget for engineering supplies. And that's serious. And so they managed to cobble together from the budget, $12,000 per episode for all engineering supplies, consulting, everything. Yeah, which is not a lot when you realize that the show is about engineering. And yeah, so if you want to know a lot of the down and dirty secrets of the show, see our DEF CON 17 talk. And Sam production company is Myth Buster. Somebody the other day was like, hey, you guys should talk to those guys that make Myth Busters because they made an awesome show so they could do another one. And I'm like, actually we did. And our show was awesome, it was just one season. So, oh yeah, anyway, okay. So the original fan site is still up on the Discovery Channel website. And then all of the engineering documentation that I could gather for the technical portions of the build, the electronics portions that I had handled and then the software stuff that Zaza's done is up on my website. So if you want to build upon it or use it or see how we did it or just grab the code or whatever you can. So here's the talent. That's what they called us, talent. Which is kind of cool. Yeah, we didn't get a dressing room though. No, we did not. We asked for one here though, but they didn't give us one. Anyway, so I was the electronics guy, as I said, Zaza's. Yeah, I mostly took care of anything that needed to be coded for the show and also late night machining. Yeah. And Mike North was our man of many talents. He didn't really have a defined role. He sort of just did all the stuff that nobody else wanted to do. And then Terry was our machinist and special effects guy from L.A. So it's kind of a good mix of, you know, skills across the board. And the show originally started, the production crew was like, let's just put some cameras in a room and okay, go build something. And they didn't realize there was a whole process behind the engineering and design and testing and failing and fixing it and all of that. So it was a challenge. I should do not. On the pilot, they literally put us all in the room with cameras to design this robot and they said, all right, guys, start designing the robot. And then they got bored after like 30 seconds as we started actually doing real design. All right, just like throw some things in a pile and tell us how you're going to do it and then we'll leave you alone to do the real design meeting in a few hours. Yeah. And then what made us feel great is we had a separate room. So the machine shop was in one place. We filmed the pilot in a place called Tech Shop in San Mateo. The original Tech Shop. The original Tech Shop. So there's machines and lots of stuff to work on. So the machine shop was in one area. We had a room that we had called the lair, which was our electronics area and all the machines set up and stuff for soldering and coding. And we were in there building some stuff for the pilot and the camera guy, who I guess was like a famous camera guy and filmed National Geographic, like amazing whales and oceans and stuff. And he was filming us soldering and he's like, okay, this is boring, I'm going. And then he went into the machine shop and just hung out and smoked cigarettes or something or did Coke. Yeah. Yeah, the first few people we had on our show were, it was really bad. There was some production problems early on. Yeah. But these people that are up on here, we're not doing Coke as far as I know. And it was very helpful. So you know, the four of us were on screen, but we had so many people behind the scenes helping us. And that's always this myth with television is like, oh, you guys have your own show. You can do whatever you want. You can, you know, whatever. But there's so many people that were helping us along the way. So you know, PAs and production staff and various producers and stuff. And we definitely wouldn't have been able to do the show or the builds without them. Especially Joe Andreas, middle center who's a special effects guy from down in LA. You know, and he's just a total maniac and build anything and he was spending a lot of late nights on some of the later builds once it became clear that the show was so difficult to make. He got brought in for a lot of episodes and you'll see some of the stuff that he did. You know, when we get to the builds, which we should get to. Yes, we probably should. Yeah. Okay. So who is here for our DEF CON 17 talk? Okay. A few. Cool. And who's here at DEF CON for the first time? Oh, nice. I hope you're having a blast. It's like half the people here. Okay. So last time we talked about, I can't see the slides sort of, but traffic busting trucks. That was the upper right. It was a big giant truck that elevated itself and drove over track traffic and had some autonomous modes and crazy things like that. Fire fighting pyro pack. So this was a high tech pyro pack for firefighters that had a heads up display and thermal imaging camera and some sensors for firefighter identification and an awesome superhero dry chem thing that ZAWS had made. Like you pull this thing on your arm and chemicals come out. Right? Yeah. Watch the show. Let's get to the next slide. Let's talk about what we're actually talking about. See adventure waterslide, flying lifeguard. Here's what we're talking about. Go for it. Yeah. So today we thought we'd cover our other five favorite projects from the 13. Boxing robots, what Discovery called Mind Control car, six-legged ATV, the get up and go, sleep pod and autonomous pizza delivery. Yes, sir. All right. So Boxing robots, this was our first official build. We had filmed the pilot and none of us knew each other. And a few weeks later we started building these giant 10-foot boxing robots that would move based on the movement of players outside the ring and we made this whole spectacle around it. Yeah, basically this one was Terry's idea. We'd gone through the pilot and originally they wanted us to build two completely different things per episode. So it was even harder to begin with. And so once we got through the pilot and everyone was kind of fried already, Terry was just like, let's do giant boxing robots because we all know how to do it. Terry's done all this kind of special effects stuff. He basically had the whole thing in mind mechanically already and we knew we could figure out the electronics and the code pretty easily for it. And this one I think was actually a two-week build. Yeah. This was one of the few that we got done on time. And let's see. So, yeah, this... Oh, you can talk about the mechanical stuff. I don't really know what it is. I don't understand it. Basically, everything was super simple mechanically. It was just so universal joints welded together with pneumatic cylinders to move everything around. And then the outside, you know, this is one of the things that Mike contributed. He's like, you know, we really need to make these robots look awesome. But we don't have time to, you know, even really think about the exterior design that much while we're building the inside. So I know this guy. He was, he's really tied into the San Francisco art scene, Burning Man, everything. He's like, I know this guy who builds robot sculptures out of found stuff from junk art. So let's just get him to select some stuff to put on the outside. And Nemo came in and just made these like totally awesome looking, you know, badass robots out of bits of cars. And, you know, the heads are made out of gas trucks. Yeah, they're the outboard motor tanks for a boat. And so it ended up looking really cool. Yeah, those look cool. And it was, the robots were pneumatically controlled. So that was just this Pro XR serial RS-232 interface. So Terry kind of handled that build of the robot side and then Zaz and I handled the detection and control of them. And Mike actually built the second robot, but they didn't film it because they wanted to make like, make everyone like sort of pigeon hold. And so he just got totally ripped off and no one knew. But he built the second robot. Poor Mike. Okay, so the, let's see. From the hardware side, we needed a way to detect the punches that a player was throwing. So we decided to have two, the jab and the uppercut. And we used a free scale Z star module, which was this three axis accelerometer reference design. They don't make that version anymore, but they have some other one. If you want to mess around with them, there's a cool software interface that we could see the actual results and it was graphed. And that's how we figured out, we would throw our jabs and uppercuts and figure out how we were going to detect those. And so the, I can't see what we've got here. Oh yeah, I had to do it in Windows because of the cameras we were using. I wanted to get some nice high-res fire wire cameras, but the budget didn't support it. So I ended up getting these nice IDS micro-i's. They're USB 2, sort of like a webcam package, but it's an industrial sensing camera. And they were pretty nice for detecting. And then I just used an AR toolkit build to do the motion tracking. So that was like super simple. You know, basically out of the box, you get the six degree of freedom information from the glyph. And then for the punch detection, a pretty simple shape filter on the three different axes. It turns out that if you throw a jab versus this uppercut thing, like you get a pretty nice repeatable signal from that every time you do it. So it was just like a little filter on that. What else there? Yeah. And then what those classifiers did was send out like, you know, punch packets basically to the pneumatic controller. So I would assemble a queue, feed those out by UDP, and that becomes important. Oh wait, I'll say it now. So people asked afterwards, during the final battle, they were like, how come your hands were on the keyboard the whole time? You were just cheating for Joe, right? And the reason they were is because, you know, like anything when she just coded pretty much the day before, there's still some issues and we hadn't really done a lot of testing. So I was able to fill up the motion queue of the robot so they had a few punches left to go and then I could quickly kill the program and recompile and bring it back to the status that it was on. So I had like a few hidden keys to bring the score meters and everything back to where they were when I had to kill the program during the fight. It's the magic of television. And I think this was actually the first, like, television usage of AR toolkit, right, of augmented reality? It seemed to be really new. I'd seen it at a conference a little while before that and everyone, you know, a lot of people that I knew were using it. But when I told the TV guys about it, it completely blew their mind and they couldn't understand the way that this, you know, this glyph translated to something moving on a screen. And I had to eventually build this little ball maze, which I control with a glyph. And the TV guys were like, oh, right, okay, finally we get it. Cool. And then they ended up breaking it by accident, right? So besides the control of the robots, which mostly was software based, we also had what we called the helthometer, which was a scoring system because we actually had a whole set up boxing match. Oh yeah, I should mention too. So the boxing match was supposed to be like nerd versus awesome boxer. And since this was the first episode, they're still trying to pigeonhole us. And we trained, we went to a gym, an actual real boxing gym in San Francisco and got to fight a Golden Gloves national champion or something that, you know, she was about this tall, a female, very scrappy. And, well, you'll watch the episode, you'll see. But basically, you know, it was all like, let's see the nerds get beaten up by a girl and that's it. Everyone's going to think that's funny. It was awesome. Which it was kind of funny. Yeah, well. But so we needed a scoring system. So the helthometer was sort of an independent thing that we had magnets and hall effect sensors in the robot's neck. So as the robot got punched and the neck would, the head would go back. We would detect that as a hit and then reduce the score. That was a basic stamp too. And all of the lights were just 120 volt, you know, colored flood lights. So we had this, we had a bunch of SSRs with an EFX tech RC4 controller, which can control four solid state relays over serial. So we had a bunch of those. You can just send commands from the basic stamp when you detect a hit, then we could change the score. And I should mention too that people are probably wondering why basic stamp, how come not Arduino, this was in 2008 when Arduino was not even, you know, sort of coming up. So, you know, you could do these same sort of things with Arduino if that's your platform of choice and you can check out the code and sort of port stuff over. Oh yeah, here's a video of it. Is it going? No. So they're punching and stuff. It's a, it was more exciting in person. At first, you know, because we'd made all these like nice, you know, painted gloves and the legs and everything like that and we didn't want to destroy them immediately. So at first we had to turn the pressure down on all the pneumatic cylinders. So it just kind of worked. And then in the final battle, we turned it up to maximum and bits were flying off everywhere. Yeah. Oh, and the other thing too is don't let an animatronics guy script up custom things to do with two robots up on stage. I have videos of those, but I know there's some kids here that are under 18. So I'm not going to show it. But it looks strikingly real for two robots, you know. It's crazy. Okay. All right. So this was the one that the network decided to call Mind Control Car. And what we called it, this drove me freaking crazy. So this is a rant. We called it anger management. And the idea was that if you get road rage, then a therapy session would be to have to participate in a demolition derby where the angrier you got the worse your car would perform. So you'd have to learn to stay in control while you were trying to smash other people's cars. And then the network, you know, just either didn't get it or didn't, you know, they were trying to push everything as being this future product. And so they said, oh, you know, when you watch the episode, it says, you know, in the future, when you get angry, your car would pull off the road and stop. Because that won't make you more angry. Yeah. Everyone who watches the show is like, dude, that's so stupid. You guys are idiots. Like, who would want a car like that? We're not fucking stupid. It was never about that. And they showed it over and over again, that cartoon. So actually, this episode was an exercise for us in anger management too. I'm not going to get into the whole story. We'll just say that both of us almost quit every day. And you almost got kicked off the show, right? For your hair. I don't know if it was your hair, if it was something else with that other person they tried to add on the show. Yeah. This was like a miserable, miserable experience. It turned out okay, though. So this was four cars, remote controlled that would crash into each other and you had to stay calm. And it also took GSR and EKG as part of other things to monitor in your body. Yeah, we're doing this whole biometric thing with the measurements of heart rate and galactic skin response to see if people were stressed. But of course the big deal, which we'll get to in a sec, was the sort of mind control headsets. Just wanted to quickly say something about instrumenting or automating a car and making it remote controlled. We wanted to ask the guys from MythBusters about it because they've done it a million times and they would just tell us what to do and save us a lot of time. And the production company would be like, no, you cannot talk to those guys. We want to keep you completely separate because we don't want it to look like you're a MythBusters ripoff show even though we're basically trying to do, you know, have the same success we have with MythBusters. So we got Joe Andreasian and he figured out how to do it all and it basically is really simple. I talked to my brother as well because he had some guys who had done robotic cars and if you want to make a robot drive a real car there are some serious issues with it. Like you have to deal with like stickion in the pedals and things like that and there's like a lot of complex control stuff. But if it's just a human driving it, they just figure all that stuff out. So it's just super easy. You can just see it's like, you know, almost totally ghetto. There's like a motor with a cam on it that's pushing the pedals. The steering wheel is just connected to a belt drive. These are total beaters from the junk yard, right? So it's not like doing a Prius where there's already motors in the power steering and stuff. And then there's not a photo of it but in order to shift into gear, there's just a big fat linear actuator on the gear shift and it just... And so yeah, we had to do this for four cars and it was all originally, this was since Joe Andreas does a lot of special effects stuff, it was all just RC transmitters which we had to, Zaz had to actually get hooked up to his PC which I'm sure he's going to enjoy talking about. Okay, so from the electronic side we wanted to have like a pod. Each person would have a pod that they would sit in and they'd be hooked up to all the stuff and they'd have a screen and they'd be able to see what their car is seeing but then they'd also have this heads up or this on-screen display of like their current heart rate and their I don't know what else is up there. Oh yeah, what gear the car is in, the acceleration and all of that stuff. So you know they had to calm themselves down. We were actually worried that you know because it was the four of us driving and we knew the system that we wouldn't get angry enough and it would just be kind of a bust and the cars would just always just work. So we also designed in that Joe figured out how to do an electric shock machine that was like a shocker. We had to send shocks to each other while we were trying to drive but we didn't end up using it. Yeah, we used it just not on the show. We use it on Mike Moore. And we did find a we did find some way to cheat on this game didn't we? Didn't I win? Yeah, I think I don't remember. I think you did but I think it was because everyone else's car died. Oh no it was if you took off the emotive headset which we're going to talk about for the mind control part or something like that you could sort of cheat it to still move around like it wouldn't stop you. I don't exactly The mind control was only to get the car started and then you could also shut it down. But yeah if you if you move the electrodes I remember now so that they didn't get a good reading it would just kind of freeze out and then the car would keep working forever. So the way this worked is I had a PS2 PlayStation 2 driving controller that was just you know regular PS2 connector hooked that up to a basic stamp because the PlayStation 2 controllers are all serial so I could monitor that and and pass those commands off to ZAZ over network so I was using Lanteronics export which are serial to Ethernet adapters. They were basically the coolest individual component of the whole experience like being able to not deal with serial anymore and just do everything over the network was just so nice. Yeah it was nice and then the Bob 4H on screen display which was also a serial type of device you send commands to it and feed video in and get video out which is kind of cool. So the big deal though for everyone at the network was these new at the time you know mind reading EEG based headsets and so we talked to the two companies that were at the time working on the prototypes one was this company Emotive and it's a single electrode device that you've probably seen it now in a lot of products like the Star Wars force trainer and if you want if anyone wants to do any hacking of brain machine interface stuff this this thing is super super easy to do this everything all of the EEG based classification takes place on a custom ASIC and then it gets fed out over one of the lines on RS232 at like 9600 board so you can just tap in and you can get everything from that chip so just take apart a force trainer and you can use it to control all kinds of things you get I can't remember you get two eight bit values out of it the one is we have three one is like how well the headset is making a connection and you get two useful ones but you don't really know what they are it's like whatever I set up there which I forgot what it was attention and meditation and so nothing you know we weren't really sure what either of those things were and whether they really related to road rage and anyway at the time we think they only had a couple of headsets and they couldn't give us four of them so emotive had this one was also knew that was a video game controller that had you know 10 electrodes and did in addition to these kind of like calm this measures it would read the electrical signals from the facial musculature and give you facial expressions and then also it had this kind of classifier which they wouldn't tell you how it worked but you trained it was probably some kind of support vector machine so you would think about something while in training mode and then if you thought about that thing again it would recognize it and supposedly you could train it with like 10 different things but no one even that emotive said that anyone at that point had been able to train it with more than like two or three so we just decided we'd go with that one and we would use you know with the mind control thing because it was sexy in you and think about something to start the car and to show that off we did the good old spoon bending test yes and also this the emotive I think it's the emotive or there is another brain machine interface thing for toys I think it's a Mattel mind something game where you control a ball and yeah people are starting to hack on that now what's it called oh it's Neurosky okay so the Neurosky one okay so that's in a lot of different products now if you pull them pot you can see that little chip yeah all right so here's the spoon bending test actually I'm gonna have to hold my mic up to the speaker I think well we can no lame okay well never mind anyway just we just you know we a lot of our sort of attention came to how to demonstrate technical stuff so that the TV guys could understand it and so that then people watching the show also would think it was fun and cool if they didn't already know about it and so you know Joe and I ended up spending quite a lot of our non-design time thinking about ways that we could get technical stuff on TV you know it was a really a really a fight with the production company so once we're like well mind control you know what does everyone want to do there's that Yuri Gellagai you know bending spoons let's bend the spoon and then similarly this one I was really proud of like out of out of everything that we did for prototype this one of the sequences that I'm probably the most proud of is I managed to convince the network to let me talk about multi-threaded coding on national TV right so we filmed this little what Joe and I called them was like podcast style segments and you know once they basically come clean to us and said look we're not gonna talk about the nitty-gritty details of the stuff you guys do because we think it's really boring and we don't understand it so we would you know come up with these they said okay you can have one minute per episode to do a podcast style technical introduction to whatever it is that you're that you guys do and you know just to illustrate that this was so serious in the mind control car episode you know Joe had designed this whole circuit board to you know control the pods and everything like that and they didn't show it in the episode at all you know the most important thing that Joe had done and the reason was the network execs didn't know what a circuit board was and afterwards when we were really upset about it you know Joe was like what the hell man you just cut all my stuff out every single episode as a circuit board and I don't know what it is yeah and then someone I think John Tessier explained to the network guy that a circuit board was the green thing inside of VCR that made it work and then they're like oh oh right okay yeah we gotta show that show that yeah so so we actually have one on one episode we go to watch circuit boards you know getting fabricated for one of our builds which was awesome but yeah so getting making technical things look fun and sexy is really hard you know if you're in this room you probably would just you know we don't need to make it fun and sexy because you already think it's fun and sexy but like for my family had to make it fun and sexy so okay I think we have audio now so let's try this one this is the the the multi-threaded yeah when I when I scripted this podcast style thing I was like we're gonna do a four-way split screen and Joe is gonna be each of the threads and you know the TV the producer was like that's total chaos man like it's gonna take us all day to film this you know come out with something else I like no no this is gonna be really cool trust me and so I'm really proud of it so I wanted to show it yeah all right here we go and inside look computer coding right now sauce is coding might look like he's just playing around the keyboard he's actually writing specialized programs that are gonna tie all of these different parts of the build together it's pretty complicated stuff so for a little better explanation some more detail about how everything really works I present to you the coder there's a lot of custom code that has to be written to get this anger management system to work that's why you see me typing all the time code is a written set of instructions that tells a computer what to do and I'm the one that has to write it my code is made up of threads thread is like a little program that runs inside a larger one there can be a bunch of threads like Joe here but they all run independently to perform a specific task in this system each car is controlled by four threads a game controller thread which takes data from the steering wheel and pedals a bio pack thread which reads a driver's heart rate and perspiration level to regulate the car's performance an emotive or mind control thread which controls the cars transmission and a car controller thread so let's say the driver puts the pedal to the metal a signal gets sent to the game controller thread telling to hit the gas at the same time the bio pack thread is also getting data from the driver's body measuring heart rate and perspiration or galvanic skin response at the same time as this the emotive thread processes advance from the emotive system and decides whether or not the car should be put into forward gear finally the car controller thread looks at everything the other threads are produced and decides what to do with the car if the driver's brainwaves are calm and focused and their heart rate is good and their perspiration level shows they're cool and collected the car controller thread puts the car in gear and hits the gas but if the bio pack and emotive threads show agitation then the car controller thread will slow things down because the driver's getting too stressed this is all just for one car for four cars all working together that's going to be a lot of threads and if I can't get all those threads to work together we're gonna be dead in the water so I had to re-record that voice over about 12 times because you know they wanted to tweak things like we don't know what the emotive thread is you have to see emotive or mind control thread and also they of course made me say that thing at the end like yeah if I can't get it to work you know everything's gonna be screwed up I designed it of course I'm gonna get it to work but you know they have to they have to add the drama and so I don't have a picture of the actual demolition derby because you can go watch it on your own but this is like a behind the scenes of us testing it and I think yeah our producers driving it John Tessier is our producer and there's nothing more scary than seeing a car drive without a person in it like you know it's going to be cool but when you're sitting in front of the car and it's driving on its own it's terrifying so here's a here's a demo of that you just don't think about it when it's not something you pack together but you just like we just know at any moment something could fritz out and the accelerator goes all the way down and this thing just makes its way into the bay or us yeah so that one was actually pretty cool the six-legged all-terrain vehicle the bane of our existence yeah this so this one for well I don't know if I should spoil it but anyway it was a it was a hard project that may or may not have happened basically the idea was you know we wanted to build some kind of crazy contraption that a person would write in and we thought about you know a legged vehicle so you know building like vehicles is tough right some of the smartest guys that have ever worked on robots are working for companies like Boston Dynamics to make like vehicles and so we're not going to get it done in two weeks unless we start with some kind of small robot platform and just scale it up so that's what we decided to do right off the track we looked at a few that we thought would be fun and cool and the one that we chose was this one called Rex R-H-E-X and it has the distinction I think for being in terms of body lengths per unit time the fastest legged robot out there so we thought that would be cool if we can scale it up we'll have the fastest legged vehicle out there yes so we should make sure we have a crash cage so that the driver might survive and so we got this guy the Rex project was actually at several universities and the licensing for the Rex like a lot of the Rex patents was licensed by a guy called Halden Komsuoglu and he started a company Sandbox Innovations and he came on the show to help us out and right all the tune all the controllers the interesting thing about the platform it's got six legs and they alternate in tripods so three of the legs come forward and take the weight of the other three flip around one degree of freedom per leg so these legs are just flipping around and that's actually it's a biomimetic thing so it's based on what was like a cockroach or some bug we had to go visit it's inspired the alternating tripod gate if you look at hexapod creatures they all do it roaches actually do some interesting stuff with their back legs when they move really fast but mostly they walk by three on the ground to around the materials properties of the legs are super important if the if the legs are too stiff then the robot just stays in place it depends on that springiness of the legs compressing to take the weight and then bouncing the robot forward to give it its speed so that's why designing the legs off the bat we thought that was going to be the toughest problem and we really concentrated a lot of energy in the form of Mike North on those legs yeah and also scaling something up and the legs were part of that problem what you want to talk about this mechanical thing legs well anyway I'll just say one thing we went out at this point in the show we were able to get outside consultants to help us out and we went to spec design to do the design of the chassis and the guy from spec they lead mechanical engineer super nice guy he was really really concerned about the safety of this thing I mean you see that video and you know that basically no good can come of this so very resistant to using aluminum tube for safety reasons he just wanted to make this thing just built like a tank so we ended up going with chromoly steel which caused big problems later on because it was so so heavy I think you could do this safely out of aluminum if you just like you did a bunch of FEA on it and you just figured out you know it's crash scenarios and just making sure that structure wasn't going to break but there wasn't time for that but props to John from spec for caring about chassis at all we had some other guys build it outside who specialized in making chassis and then the legs Mike North helped design them but they were made by a place that specializes in composing making things out of carbon fiber materials so this build we definitely would have been able to do by ourselves no way they gave us a whole fridge full of like carbon fiber composite and so on because you had the cement you have to store in a fridge and they just delivered it so those legs hundreds of thousands I don't know it was like a lot of carbon fiber well that's the benefit of being on TV you just call someone up and say hey we're doing a TV show can you give us some free stuff and they'll be like sure until they find out you know what it is that they're doing and but yeah they committed a lot of money to a lot of resources it was awesome so Mike got these legs back and he wanted to demonstrate how springy they were because the springiness was the important part of this project so he built this whole contraption to so that's our omnidirectional forklift I'm driving the forklift and I'm just jacking it up and down to put some oscillation on their leg and let's take a look at what happens yeah check this out the leg is very strong and and pretty much unbreakable as you'll see I'm not sure that made it to air yeah that that that gave a lot of the TV people the brown trousers because they didn't have a lot of liability insurance for this kind of stuff so that that footage if Joe got back then that would never have seen the light of day I think that stuff actually disappeared from the from the reels you know from from their tapes so so I was I had our ways of getting some of this important footage to preserve history oh the drivetrain okay so just a little bit about this when you scale something up especially when you only have two weeks to do it in the big decision really is like how far out of your comfort zone do you want to get and we would decide this uses brushed DC motors with planetary gear heads you know just like these Max on you know nice ones but things that we're all really familiar with the Boston Dynamics like big dog and stuff like that has a hydraulic drive system and then we could you know could have gone somewhere in between like some of the people got amusement park rides they have these giant like three phase electric motors and stuff on them so you know we're just trying to toss up you know should we go hydraulic we've never done anything really with things that we don't know a lot about or should we just try and scale it up you know and we just said all right let's just you know we think we can do it we've done all the like torque requirement calculation stuff we think we can do it just with brushed DC motors and planetary gear boxes in retrospect that was a mistake we should yes if we were building this again today we would go hydraulic because that would just solve a lot of problems you could just you know throw a ton of pressure at the problem but what we decided to do was get these custom wound mag motors that really really high performance just a huge fat industrial planetary gearbox on it with a 40 to 1 reduction and you know we think we think it'll work out the motor control things started to not work out pretty soon the motor controllers we had even though they said they were rated for the currents that we were putting through this it started to explode pretty quickly and they had all these safety features in them so they would just turn them up so the machine would collapse so we disabled the safety features exploded the motor controllers we went out there to find the the beefiest DC motor controllers we could find and what we got with these Sevcon millipacks they're for electric vehicles so we put a whole bunch of those one per leg then we started to discover you know we know that we can do this but we still this machine still won't stand up like you know what's the problem here and we figured out when we talked to this guy Peef that when you run a thousand amps at 48 volts right 48 kilowatts through a wire around the outside of a chassis even though it's a single loop inductive effects that's coming to play it was scary shit yeah and you can't get those rise times that you need if you want to just turn it on so you know we had to strip down like rewire everything just get the slightest loop out of it to try to get rid of those inductive yeah there's a lot of noise and we made it we made an optical communication system so it was taking in the motor control signals and then sending it over fiber optics to the receiver so we could get rid of all the noise that was that was being caused around the system yeah these giant 48 kilowatt amp pulses were fritzing everything out so if you when you build EVs in the future if they're going to if you're going to do that go high voltage and keep those currents down because they're a problem yeah let's skip the podcast and the original version that Halden had worked on was a I think he was controlling it just from his Linux box but we wanted a real joysticks you could sit inside the chassis and you know control it we had a bunch of buttons that would stand up the thing and well we tried to stand up the thing have it sit and change the different mode so it's sort of an arcade style thing where you would sit inside the chassis so I had a PC no this was an arcade style joystick just a big huge arcade style a modified board that I had worked on so I had a project called the Stell adapter which was a Atari 2600 to USB interface that you could you know use paddle controllers and joysticks and stuff so I modified that to be the big joystick to USB interface and then plug that into Halden's control box which was some Linux machine and then we had a trigger that just served as a dead man switch so it was actually kind of cool and it worked out pretty well very exciting another chance to use something that at the time was new so these were these giant lithium-ion phosphate batteries from China so that was a pretty much brand new chemistry at the time that everyone was excited about the thing was they were so new that they didn't come with a battery management system so you know we're halfway through this project and suddenly hey Joe you got another thing to design a battery management system for these brand new batteries that you know but yeah I think we got the batteries first and then they're like well how are we going to charge them and they're like Joe hello so we had to design this thing we had one one day we had one day to design this so this was this was a time where we reached out to some of our contacts and and begged to have our boards fabbed right away and we actually got we got the boards fabbed and got them back the next day which was ridiculous so I so for each individual battery cell which was this lithium-ion phosphate self-maxim lithium-ion charger and just jacked that up to the max that I could so it was a 4 amp charging rate which is slow like that's good for like a cell phone or some portable device but not gigantic batteries that had I think it was like 90 amp hour batteries so it took 15 hours for full charge which means that production actually had to plan ahead for our builds otherwise we'd run out of battery power but it was cool so it had a charging circuit and also a discharge because you can't charge batteries below 3 point or below 2.7 volts or something like that 3.0 volts so I built a little discharge circuit and the LED would be green when you were above the right threshold it would turn red when it wasn't so we could easily just look at the system at NC so it was sort of a totally hacked together budget battery management system and normally they cost what like thousands of dollars for electric vehicles or something and yeah we we cranked it out and all kinds of you know just a terrible just terrible indignities were wreaked upon me in the show but the idea was you know you see those old cartoons like the Warner Brothers cartoons you know where someone you know is in the future and someone just jumps out of bed and they go on the conveyor belt and they clothes get stripped off and washed and fed and you know we wanted to see well could we really build something like that you know it's gonna strip your clothes off shower you we looked into automated feeding and then it's gonna give you clothes get dressed and you know gonna get out there one thing that we actually did have a lot of discussion about the get up and go behind the scenes and we're trying to figure out how to do it is the first thing you do when you get up in the morning you take a piss right everyone everyone knows that so how are you gonna do that in the pod and we're talking about like a you know on TV people don't go to the bathroom right and also you can tell that nerds built this because only one person sleeps in it at a time yeah so so this was actually we had a real hard deadline for this one this was scary because we were debuting this in Union Square in San Francisco which is a pretty serious you know kind of touristy spot so we had this whole thing kind of pushing it like a like a tele commercial whatever like you know like those pitchman things yeah Terry put on his white coat and was trying to sell it to the crowd in Union Square it was pretty bad so the system the hardest part of this was fitting everything inside there was a HVAC system and let's see what else do I have on here we have a machine that Zaz had that would control all of the pneumatics again we had a basic stamp to control we had like some optical encoders and we had some lights and LCD and a user interface and just a lot of crap in there that we had to fit in that was never actually shown right so it just looked really cool but there was all the stuff inside yeah it turns out especially the HVAC system you know we've got this super cool little compact one I don't know if it was supposed to be for like an RV or something like that but we had to put that in there plumb all the air around the water supply was off board so we managed to sort of cheat a little bit for TV for that let's see other oh yeah so the basic stamp we had an optical encoder to control the to know where the shower tray was another export to communicate with Zaz's stuff so at this point we sort of had a system where all the control stuff would just be sent over to Zaz either serial or network and then he could do all of the heavy lifting for the stuff so it sort of worked out that way here's the user interface yeah so I had got lucky I had a friend who was working for Kallikonetics and I said can you send me some Kallikonetics lights sort of under the table and so he sent me a bunch of these eye color coves and at the time I don't know if the situation has changed but at the time Kallikonetics was just being really bitchy about releasing the protocol for talking to any of these things and people were having to like spend a lot of time reverse engineering it so I said look man can you just tell me what the protocol is I promise not to tell anyone and so he hooked me up and gave me the Ethernet protocol for the Kallikonetics lights which is really nice yeah it was like Ethernet to DMX 512 I think which was standard that was one of them but yeah there was the whole thing was actually controlled you know just by a Mac Mini by the GUI on the side there so it wasn't all automated there was like just a nice little control thing that someone could use so when we actually demoed it Joe was off to the side VNC'ing into that Mac and then controlling the GUI using that so it was a total wizard of ours man behind the camera alright so here's what happens here's a demo so that's our producer giving it a test ride as the thing closed so we don't have a full demo of Zaz getting showered and stripped everything but I know you guys want to see that if you're really into that you can download the show yeah alright so the final build that we're going to talk about is the automated pizza delivery because everybody loves pizza and every you know no one wants to actually go out and get it it's a pain in the ass so you know why not bring it to you so the backstory behind this one is that the network exec in charge of prototype this had gone to like a Brookstone or something or no no he'd seen the Skymore catalog and he'd seen one of those little silver helium remote control blimps in the Skymore catalog and he said you've got to have the guys use that to deliver pizza you know that's going to be totally awesome put a pizza on that blimp and deliver it and we said that's a stupid idea that little blimp cannot carry a pizza and he's like no no no you've got to put the pizza on you've got to put the pizza on it can do it I'm sure of it but no it can't he's like how do you know have you weighed the pizza so finally we're like alright we're going to do this we're going to figure out a way to do pizza delivery automated blimp so you know he forced us to get in one of those big blimps that they have at the stadiums you know giving throwing out t-shirts and stuff like that and we had that carry a pizza and then we went to the Goodyear blimp and we threw pizza out of the windows of the Goodyear blimp and you know just basically to be like leave us alone we're going to do this but not with a goddamn blimp and because of this whole weighing the pizza thing every every day Tessier would get an email have you weighed the pizza yet so just just to kind of like you know stick the knife in a little bit we film the sequence of weighing the pizza yeah and and um working on this show is like working at any other corporation you have a bunch of dumbasses up here with stupid ideas trying to tell you what to do and being hackers that really sucked ok so here's weighing the pizza yeah this is weighing the pizza and uh yeah whenever you do a prototype with a blimp sooner or later you gotta weigh the pizza I got my scale and I got a pizza fresh out of the oven commence weighing the pizza 3.51 pounds gentlemen I've weighed the pizza three and a half pounds so that's a total prototype this in-joke and now you get it too and the guy approved it on the network I don't even think he knew we were making fun of him so there are two different versions we had a robotic version for city streets and then a vehicle for longer distances and uh you know this would have been cool to use like a DIY drone sort of thing quadcopter but there wasn't you know accessibility to that yet so the automated uh we had a pizza pie pack so for both of these systems we needed a way to autonomously give you the pizza once it got there right so we had a credit card based system this crazy wired up just by hand basic stamp we had an LCD mag stripe keypad to enter in all your infos um and motor controllers and optical detectors for the for the pizza tray uh and it was it was cool here's sort of like you know we have to build a prototype before we build the real thing so there's Zaz demonstrating the pizza pie pack and then that's the uh that's one of the final version of the robot version yeah so the robot on the on the right um is uh my again you know we hit up our contact network and I didn't have to go very far for this one because my brother uh started a company uh called Marathon Robotics they're doing really really great and they make uh live fire sniper target robots that are completely autonomous um they're they're I think the uh this is what he's told me anyway uh because the U.S. Marines bought some and so they're officially the first um autonomous ground vehicles to be deployed with the U.S. armed forces um so here it is you look closely this is my brother making fun of me because if you look closely he's wearing my signature t-shirt on that robot that just got shot in the head so I don't know what he's trying to say there um and uh the robot you know went after they get shot the other ones all scatter and hide because they know that the first one got shot so they're super super cool targets so we called him up and said you know how do you feel about getting a like a vacation that won't be relaxing or fun out to the United States and uh bring a robot and demonstrate it on the TV show and he said yeah okay we can figure that out so the first thing he had to work out was how do you bring like a 300 pound heavily armored Mildroid to the United States at short notice and the answer is you take all the armor plate off and check it as luggage and then you carry all the sensors on board and when anyone from the TSA asks you don't say that it's for a heavily armored kill droid or Mildroid so yeah the whole they brought the thing on with them on the plane um and what was I going to say about this guy um the LIDAR yeah so um it's based on a Segway RMP platform so uh this is a thing that Segway put out specifically for um researchers and you know robot people to build robots out of it has GPS but they weren't really using it because they couldn't get into the GPS um like they there aren't there or at the time weren't models out there that were generating the noise models so the really the only sensor that's being used is this ZIC LIDAR that gives you a range a single line of range and it comes out of the front of the robot it's also the only vulnerable spot on the robot um it's like a little tiny slit and the marine snipers have tried to shoot it and last time I went to visit him there was a bullet wound right on the edge of that lip and the the marine sniper was like alright you know if that's if I can get that close to me okay but you know the guys are gonna have a shot at trying to shoot that sensor out um but what it meant is that uh when it sees a hill if the hill is too steep it interprets that as a uh an obstacle because it's only got that line of laser range finding which led to some pretty significant challenges delivering pizza in the streets of San Francisco. Yeah but it worked out great it actually worked in the city and we were happy to get pizza delivered to us um so then uh uh real quickly we had uh and this was um a project that was run by a friend of ours Anthony Levandowski who it turns out was working at Google on another project and um uh but he was you know just doing this on the side so he came over and used his car and you know drove drove over the Bay Bridge it was actually the first um what rolling roadblock the first bridge closing of an autonomous vehicle or first bridge crossing it was totally cool and it turns out that the guys from Google um uh had seen me and said Anthony we want you to do this for Google and paid him a lot of money and now he's doing the self driving car for Google and he's probably the one that benefited the most from doing this show. Really quick story about the rolling road block you can see there you know the police they can't close the Bay Bridge you know for just about anything right because it's like such an important artery in San Francisco so um they said we'll do this rolling road block we'll basically just hold traffic back while you do this autonomous car crossing and um the cop uh goes up to the producer John Tessier uh said you know there's there's traffic lights in between where you're starting at the marina and the bridge what are you guys going to do about the traffic lights and Tessier just said we're just going to blast right through them and you know cause he didn't know like you know dealing with the cops you never really know and the cop was just like I thought as much alright let's do it so big props to the SFPD for that. Really cool experience so um yeah that's basically it that was our experience on prototype this one more in joke that was too to make it to air you can see here we've got a you know playing with the pizzas and every time we would take a pizza out of the pizza pipe or any of our testing of like delivering the pizza and so on you know Joe is an athlete he gets hungry a lot he would always take a slice of pizza out and eat it and um so we had this running joke that how pizza transmission is a lossy process and there's like parody errors you know and every time you get it at the other end there's a slice missing but it was too nerdy and they didn't make it to air but now you guys know it. And we ate a lot of pizza man I don't know how many pizzas we ate but I think after this episode I didn't touch a pizza for like a few months. Who thought you could get sick of pizza? Anyway yeah that's it we are going to um QA room number one if you want to ask us to say that if you could leave through the side doors and not the back entry door that would be much appreciated. The ones that say exit. Yeah awesome alright thank you guys.