 So, I would like to introduce Dan Petro, his tag is alt4. He is here to talk about creating a Super Smash Bros. Melee AI that abuses frame perfect inputs which makes things really, really difficult for humans and he's going to talk to you about how he created it and it's going to be pretty awesome, so enjoy. Cool. Thanks a lot. What's up, DefCon? We're going to talk about Melee today. We're going to have some fun. So, I am Dan. I am a penetration tester at a company called Bishop Fox. I do things there like hacking web applications. We do security evaluations for like the Fortune 1000 high tech startups, that sort of thing. I also have talked at DefCon a couple of times. Last year we gave this great talk about hacking smart saves. Before that I was known for something of a Rick Roller. It came up with a little device that hijacks the Google Chromecast and can play arbitrary video to that which has to this day not been fixed. Not because the Google was silly, just a low level design problem. But that's not really why I'm here. If you're like me and if you're in this room, I suspect you are. It's because you're into video games, right? Before we got into the information security field, before I got into hacking, if you talked to middle school me, I was super into video games, right? That was the thing that got me into technology. And so that's always been a side thing that I've been interested in. Particularly this game, Super Smash Brothers Melee. Smash Brothers Melee is not just a video game in that sense, it is also an e-sport. By that it means that there are competitive players, in fact there are professional players. You can see in the bottom left-hand screen here, those are some competitive e-sports teams that have professional players that do nothing but play this game, Smash Brothers Melee, for a living, right? This is even more popular games, and it's also known as one of the most technically demanding games. So it's very, very fast. You can see in the bottom right-hand corner, even though that's not Melee, that's Streetfighter. It gives you a good example of what they call APM, the actions permitted. Just how fast and technical the game can be, right? So in addition to the high level strategy of what it is that you're going to do to your opponent, you also have to worry about the low level intricacies of the game in terms of how to actually button mash fast enough. So it's not just that you're pressing buttons very quickly, but also with very precise timings. So it's known as a very demanding game. And as a lot of respect, it's also a very, very old game. Melee has been out for just short of 15 years now. I think it came out in November, so it would be 15 years and just a couple of months here. And I am a player. So this is me. I asked my wife to get me some of the most embarrassing and socially compromising photos that she possibly could of me playing Smash way back in the day. And I think I turned out alright. So yeah, I've been playing the game basically since it came out, and competitively more or less since that has been a thing. And those have been sort of my two loves, right? Information security. This is like what I wound up doing for a career and is also a big passion of mine as well as playing video games. So hey, why not combine them, right? So the story is, I was playing some Melee, something like last year, with a friend of mine back in the Arizona Smash scene since I'm from Phoenix. And I talked to him afterward and said, hey, so what do you think a computer could be like if you could play the game frame by frame, like perfectly? How good do you think a computer could be? And he responded that the game requires too much high level strategy, too much like mind games, that there be no way that a computer could be really good. So of course, I thought fucking challenge accepted. So I then begun on a months long journey of binary reverse engineering and AI research and programming until I eventually created what is now Smashbot. So before we get into some live demo stuff, I just want to give you a really brief high level architecture description of what Smashbot is and how it works. So there are four major components here that we're going to discuss. First is the Dolphin Emulator, right now at least. It works on the Dolphin Emulator, which runs GameCube and Wii games. The game then will export all the relevant game state information, so all the information about the universe, where characters currently are and things like that out to a separate process. So we're not modifying the in-game AI in any way. Smashbot is its own AI written from the ground up running a separate machine, which then does some AI magic for now that we'll get into later, decides what buttons to press, and then presses them on a virtual controller. So importantly, Smashbot doesn't cheat. It doesn't just make itself invincible, and it doesn't do anything that you, in principle, couldn't do. So it's just pressing buttons on a virtual controller and looking at the screen in much the way that you would look at it. All right, so before we get too far, we're going to do a live demo. So this is the time for you to come right up on stage right here and give Smashbot a try. So go ahead and line up. We're going to do this where you can take like a... I'm going to get some... There we go. In sec. Just restart the emulator. Just set up the game. Hopefully we'll get audio. Oh, is it? That's okay if we don't have audio for the moment. Let me just kill the emulator. I should have done this ahead of time. Smashbot, run. Yep. We are going to turn off pause. I'm not really getting audio, but I can twiddle with that in just a moment. And we will begin here. So just take one stock. Oh dear, just look right there. Yes. So yeah, I just set it up so that you can take... So Smashbot is the fox, as you can probably tell here. This is going to run... Okay, let me try to switch the audio then. That's all right. This is going to get in your way. Sorry, it's on headphones. Should I put it to HDMI? Should I put it to HDMI? Yeah, yeah. Is it testing? Front, left. It should be playing. Yeah, we'll do it afterward. There we go. Yeah. When I take focus away from the window, Smashbot stops playing. So that's what you saw. Just play. So there's a couple things that Smashbot is doing right here. I'll talk to it as the game is going. So it's going to be trying to take advantage primarily of the human player in two main ways. So it does reactions and predictions, right? So reaction is the easiest way to describe how this works because the game will often require that you commit yourself to some sort of an action before you are able to get an attack out, right? So you're going to start in a forward Smash attack and the very moment, the very exact moment that you start this attack and knows how long it's going to take and at what point the hitbox is going to come out. And so from there, it can predict exactly where to go in order to avoid it or to shield the attack, right? And so strictly right from the ability to react, like frame perfectly, to attacks, there we go. It's able to get quite an advantage on a human player, right? It turns out that quite a bit of the game depends on reaction. And so Smashbot is able to get itself pretty far entirely on the basis of that. However, that's not good enough. The emulator sometimes has a trouble lagging but it's basically good enough. The, what is it thinking? Oh, okay. I'm amazed this is working at all. By the way, I want to give a huge shout out to Duango AC who gave the amazing Taskbot talk, which I hope you get to talk to earlier. This whole thing almost didn't happen. Yeah, so it gets you in a tech chase combo. Oh, the emulator's having a hard time, okay. So this tech chase combo you see is Fox will grab Marth and throw him to the ground. And then from there, there's only a handful of options that the human player has. At that point, you can like fall to the left, fall to the right, and no matter what you do, Smashbot can in reaction figure that out, right? He can cover all the options that the human player has. So the other thing that Smashbot does is prediction. So he's able to look forward into the future in the game state, right? And like know the physics of the game, know all the attack animations of the game. And then once you've committed to some action, take advantage of that into the future. So depending on whether it actually comes to it, one neat example of this, one of them is the rolling, right? The very moment, the very instant that the opponent starts a roll, Smashbot knows precisely where he's going to end up rolling and exactly when. So he can just throw out a grab at that exact moment. So there's no way, even in principle, to get out of it. It's a little bit more evident during the edge guarding situations when the opponent is off the stage where the flow charting of the like options the opponent has is a bit more apparent there. So we're just gonna let this run until the time out, basically. And I can talk a little bit more about the, I think. So it should be noted that right now, as of this moment, the one matchup that Smashbot knows really well is this one, which is Marth against Fox on FT. And it chose that for a very specific reason, right? I wanted to tackle one problem at a time. I believe in the engineering principle of just solving one easy problem at a time, right? And then eventually try to add support for other characters in other stages. So one is that Marth, the green player that's currently getting beat up, is a high tier character. So he's a very top tier character amongst human players. So you will see lots of competitive, even professional players play that character. So it's not simply the case that Smashbot is like beating up on some low tier character, right? And in fact, this exact matchup, the Marth on FT matchup, is considered almost unwinnable for the Fox player amongst high level players. So this is something that like, if two human players were to be playing, Marth has a massive advantage. So you can take that as an example of like what is going on here. Try to get some audio after this is finished, I guess. So some other parts to talk about here, the tech chase combo is basically unavoidable. It is possible to kind of like slide off the stage or something like that, but it's very strong. So almost, oh, we actually missed enough Smash. So I'm not gonna pretend to have created the world's first bug-free program. There's certainly some instances where Smashbot will just sort of derp off the stage or something like that. It's been an iterative process. No, it knows about that. So if ever you, yeah, yeah. If ever you see it actually get hit, it's almost surely due to what's called a shield stab. So you can see when Fox puts up this shield, there's this big blue bubble, right? As it turns out, the way that Smash works, it only shields exactly where that blue bubble is. And so if you're able to hit his foot that's kind of sticking out of the shield, that will actually land. And it's hard to predict ahead of time when that's gonna happen. Also, Fox will only shield after you've done your attack. So first you have to commit to the attack and then Smashbot will put up the shield, which means it's not really reliable for you to try to hit it. It's basically like a random thing that'll happen something like one out of 100 times, where Smashbot will try to shield and then it just won't work. Okay. Now, you can tell when somebody actually knows how to play melee, like, oh yeah, you can kind of do a recovery there. Actually it was funny, for the first maybe six months or something of Smashbot's existence, I discovered that the novices did actually better than competitive players and partly due to the fact that they were just doing weird random things that I hadn't considered. Whereas all the competitive players were doing things that I had anticipated. Or maybe just, I would never consider that I would, oh, okay, emulators having a hard time for a moment there. The particular standing there and slashing, for the first six months of Smashbot's existence, I don't know how I'm gonna deal with that. I'm just gonna come back to it and hope that nobody does it. So to give you an idea about just how competitive SmashBots become, it has, I've been bringing it out in secret to the local Smash tournaments in the Arizona scene. And I don't wanna name some names of people that it has beaten because that wouldn't entirely be fair, but something like the last, like 50% of its matches at JB5 stocks, the players there, which is to say that it doesn't take a single hit throughout the entire match. And sometimes it'll randomly take a hit. Basically people realize pretty quickly that you can't just fight it, you can't just play it like a normal person as if it were a human being. You basically have to try to pen test it, right? To try to find some bug, some corner case that it's not considering. And if you're able to do that and then execute that four times in a row in order to take four lives, then you can beat it. But that's just been an iterative process over the last while to try to find all those little bugs and then fix them before bringing it out to the next thing. So this is really the first time that I've shown off SmashBot in any major way aside from in secret showing some of my friends. So it's definitely good enough to show off at this point. Though it's worth saying that there's still a lot of work to be done. So yeah, in exactly 20 seconds here we'll get back into like how this old thing actually works or how it began, the AI parts of it, the reverse engineering parts of it. See if we can get one hit in in seven seconds. I don't know, it's still got you. It does. It's frame perfect start pressing. Really wants to start the next match. I'll go ahead and kill this. I'll do another round at the end during questions. Okay, so now a little bit about the AI. How does SmashBot think? How does it decide what buttons to press? It's not simply just a series of heuristics. So the top level, it has a four tiered hierarchy of goals. So at the very top level are what I just call goals. It's the highest level of what is SmashBot trying to accomplish? What is the thing it's trying to do? So these are things like kill opponent. But it's not always kill opponent. Sometimes they just navigate the menu system because it wants to select its own character, things like that, right? And so the way that this works is that the little bubbles on the right hand side that you see here are actual source code files, right? These are the C++ files. Then the whole point of the file is to determine the next lowest strategy. So the next level is strategies, things like bait or sandbag. If our opponent just got back from the invincibility then we might not want to attack them. We just sandbag or bait. We try to bait our opponent into a wrong move. And so kill opponent might choose bait, for example, as a strategy. And this basically keeps on going down and down. So then we're trying to bait our opponent into a bad move. We're gonna weave in and out of their range, hope that they make an attack and then punish them when they do. And so bait might then choose punish. We say, aha, we know that this person has exactly 17 frames of lag, say, and so we know that we can run up and then give them an up smash in that time. And then the very last level is chains. Chains are like button combinations that smashers would recognize, things like wave dashing or dash dancing, up smash, things like that, right? So these are the lowest possible level of abstraction in terms of the actual button press sequences. And so then the punish would say, okay, I'm exactly in range and I'm in a place where I can up smash so let's go ahead and do the up smash, right? And then this is going to change frequently every single frame. All right, so let's talk a little bit about reverse engineering, right? This is something that was a lot of fun because, you know, being a penetration tester, this is sort of up my alley. And so there is an awesome melee scene of hackers. People like Dan Salvato actually has been a huge help as well as some other guys there. There's an entire Google spreadsheet that we eventually made about like getting this sort of information. But in terms of reverse engineering, what Smashbot needs to know is it needs to be able to figure out a picture of the universe, right? It wants to be able to see the screen in the same way that you do. There's no hidden information in Smash, not in a way like poker is, right? Like if I made a poker bot and said that it just plays by like reading your hand, LOL, like that wouldn't be interesting at all, right? So there's no hidden information to Smash. It's all just available on the screen. That said, how do we actually know where all the pieces are? So we have to make a couple of assumptions here. One is that the game does have the game state information represented in some way. It has to, right? It's got to know where the current player positions are. It's got to know what your damage is. And so rather than like parking a camera in front of the screen and trying to like visualize it that way, I knew right off the bat that would never work, right? So we want to be able to get information out of the game. The only trouble is to the game, or to the emulator, the game is a black box. So it doesn't actually have any idea of what's going on inside of it. It's just a virtual machine basically, right? In the same way that VMware or VirtualBox has no idea what's going on inside your like Windows VM, right? It's just running op codes and presenting virtualized interfaces for hardware. So unblack boxing, this black box is the reverse engineering that's behind Smash Bot. In particular, we don't actually care a whole lot about code and more than that, we care about data. So inside of the game, there's going to be a couple of pieces of key data, right? Things like your exact character position, XY care, like what character is my opponent? What stage are we on? What damage do we have? I want to be able to take that information, figure out where it's stored in memory and then ship that off to an external process. So it should be noted that when I started this almost a year ago, none of this was like worked on at all. There's a lot of trailblazing involved. So there's also not a great way of doing this reverse engineering. There are some tools like cheat engine, but cheat engine wasn't exactly going to do what I needed it to do. And a lot of built in debugging functionality to Dolphin, and there is quite a bit, also wasn't quite going to do what I needed it to do. So most of the debugging functionality is about trying to like disassemble code. And again, that's not exactly what I'm looking for. I don't necessarily care about the code, and that's a good thing because the Gamecube runs on PowerPC. I really didn't want to have to learn PowerPC. So basically what I did was take memory snapshots. The Gamecube is super old and only has about 24 megs of usable RAM. There are other RAM, there's like specific video memory, there's like registers, but the main system RAM behind the console is only 24 megs, which means I can just write it to a file and then inspect it manually basically. All the stuff that Cheat Engine does, I'm just doing more or less by hand. So I had to make a fork of Dolphin that every time we would take a snapshot, would write the entire contents of RAM out to a file, and then I would just be doing vbin diffs on the like memory instances right. And so I would put the game to a known state, say like, I'm gonna put my damage to 47, right? So I'll have 47 damage on a particular character, and then put the damage up to, take a snapshot, put the damage up to 98, and then just do a search to see what regions of memory have changed from 47 to 98, right? And that works really, really well when the memory regions are stable. That tends to happen if it's like stack allocated, right? However, not all the information is stack allocated. Some cases it's dynamic. So in those cases, it gets a little bit more complicated. So that tends to be when there's like a struct. So all the player information, the stuff about like actually what damage you are or things like what character position you are, x, y character positions, are stored in a big struct that's allocated on the heap. And so first you have to try to find where the struct is, right? That tends to be pretty easy. We could just look for like damages or something like that. And then you have to search for the memory. So suppose that was found at a particular address. Then you just scan the entire RAM again to find out, is there any regions in memory that contain that address? And if so, that's probably a pointer to our struct. And now we have a stable pointer to our dynamic memory region that otherwise we'd be moving around. So this sounds sort of kind of easy and then concept it is and practice this one's being a little bitch. The some data structures make no sense. It should be going without saying that these data structures were never meant to be read in the way that we're reading them. Because of course this is like 15 year old console game. So why would they have made these data structures to make any sense? So sometimes there's floats where there should be integers because it's clearly monotonically increasing like value but fuck it, they gave it a float. And that took forever to figure out or sometimes there's no consistency to whether things are indexed at zero or one as sort of like figured out. So before we go any further, I wanna talk a little bit about game programming because if you've done some programming before this is probably very different than what you might have experienced. So there's the concept of a frame and a frame loop which is very important. So on the left there you can see in real time Marth doing his forward smash attack. So he's just taking a sword and throwing it down super hard and it looks super fast and in fact looks really, really smooth. When in reality that's not how it actually works. When you slow the game way down which you can see on the right hand side there, you can see that it's basically just an animated GIF. And not only is it like kind of choppy but the animations are predictable. So at exactly frame 10 on the 10th frame of the forward smash every single time Marth will be exactly in the same position every single time, right? So the game is basically just a finite state machine running very, very quickly. So the game runs at 60 frames per second which means a single frame lasts approximately 16.66 milliseconds and so the processing looks basically like this. You start at zero, it pulls your controller input to see what has the player pressed. It runs the game engine and produces an image on screen and then keeps looping and that's more or less how the game works also basically every 3D game works. And so what's important here is that it's not just that the game is displaying at 60 frames per second, it's that the game engine fundamentally runs at 60 frames per second. So you can use this to cause all kinds of really cool bugs. So if you are running very, very slowly, right? And suppose you're somebody who's totally not Mario and you're trying to get to some treasure through that's past a locked door, right? If you walk slowly, you might just kind of run into the door and not be able to get through it. But if you're moving super, super fast on one frame you might be here on the very next frame you might be here and then on the very next frame you'll be there right past the door. And the game will have no idea that you ever collided with the door, right? Because you never touch the door. On one frame you were before it and the next frame you were behind it. So this leads to some really cool bugs like this. So this is actually a task from Super Mario 64 where you can see exactly that. This is the very beginning of the game and Mario is going to switch, go right through what is supposed to be a locked door just by going super, super fast. So hopefully some audio is here if not, it's not critical. Yeah, so basically what you saw is that you just did some little tricky bug thing to go super fast and then just zips right through some doors, right? So that's important not because we're going to be doing some like zipping through doors but just to give you an idea of that the game is running with this internal frame loop. So the game looks a little bit more like this where inside the game there's this looping thing that the emulator actually has no idea about. So the emulator is just the hardware, right? It has no idea that there's this internal frame loop. That's the game's business. And whenever it receives frames it will go ahead and output them. So in order to get the game state information out now we've figured out where the like bits are inside of the game, right? We have to have some mechanism of exporting it out to a separate process. And so my first foray at this was really hilarious. So first I set up a segment of shared memory between the dolphin emulator was a modified another fork that I made of the dolphin emulator to a smashpot. So this is a shared segment of memory. There's no like input and output. It's actually just the same memory that's shared between two processes. So what I had to do is write some code that took the game memory and copied out the relevant data into a struct that's in that game. So had to like move the data out into that struct. But of course I don't have any concept of when the frame is running because the emulator doesn't know when the frame is running. So the natural thing to do here is just make a spin loop. So we have one entire CPU doing nothing but spinning doing absolutely nothing but copying data into that shared memory region. So now smashpot has this like constantly updated real time view of all the relevant game state information. But it doesn't know when the frame has processed. It has to like trigger per frame and when a frame triggers is one of the pieces of game data. So of course I had to write a second spin loop inside of smashpot that would regularly check that struct. This is what computer scientists would refer to as sub-optimal. So eventually this was integrated into the official Dolphin build. As the Dolphin 5.0, there was a new feature called memory watcher which does this without the terrible spin loops. So I would like super big thanks to the Dolphin guys for that. So now we have three parts of the whole running system. We've got the Dolphin emulator. We have smashpot making decisions. We're able to pipe that data out over a named pipe basically. But it's still not playable at this point because we still can't actually press buttons. And so that was another kind of funny instance where my initial attempt to press buttons on a kind of virtual controller. Dolphin didn't have any mechanism for actually doing that, but what it did have is the ability to type on a keyboard so you can like map the A key to press the A button or something like that, right? So I thought, okay, great. What I'm gonna do is I'm going to write a helper that uses the XORG libraries to like press the button on the keyboard. And it actually sort of works. It's terrible and I would not recommend it whatsoever. Partly because if you like move your focus away, it just starts pressing buttons like into the random window whatever you like gain focus to and just goes haywire and it's like hard to cancel. But also because basically all of these mechanisms are gonna be buffered input. And so there's gonna be some indeterminate amount of latency from when it presses the button to when it actually happens. And normally you don't care about this. If you're just a human being, like pressing buttons on a keyboard, it doesn't matter to you if when you press the A button it doesn't happen for the next 30 milliseconds or maybe the last couple of buttons get buffered together. Like you just don't care. You're just incapable of physically noticing that. But Smashbot cares. It needs to be able to have exactly frame perfect accuracy on all the button presses. It needs to get there super fast. So eventually we wound up getting that integrated in with Dolphin as well. So now we have a mechanism for pressing buttons. So about programming. If you're anything like me, programming looks a little bit like this where you're more or less in a constant state of confusion because if you understand the problem that you're trying to program then you just solve it pretty quickly and move on to the next problem. And so to be a programmer is to be in a constant state of confusion interrupted only shortly by tiny bursts of like epiphany and coding things up. So if you were to walk up to me at any point while I'm programming Smashbot, usually to Saturday morning eating some like breakfast cereal and drinking some tea and you say, hey Dan, how's it going? I have no idea what the fuck's going on. Nothing's working. Nothing's working and I have no idea why. So I wanted to give you one cool example of what this looked like. So for the longest time in Smashbot's history up until maybe a couple months ago there was just this nagging bug that I had no idea how it worked. It was just like the only logical explanation for it was that there was a gremlin inside of my computer pulling on wires. And so it looked something like this where like Smashbot would be totally cool and then just derp right off the stage. I'm like, what is going on here? There's no reason for it to do this. I couldn't pinpoint in code why this was happening and it manifested itself in all kinds of ways. It wasn't just derping off the stage. So I implemented this entire debugging mechanism where you give it a dash dash debug flag and it will take the entire game state per frame and write it out to a big CSV file. That winds up being like megabytes large. It's actually the best thing I ever did in terms of debugging because this lets you retroactively walk through what happened throughout the entire game and see like, oh yeah, I could press this button when it should have pressed this button or whatever, right? So I could see in here that sometimes, not all the times, just randomly seemingly, I would press a button and it just wouldn't happen for a frame late. That was the source of the bug. I finally figured, okay, so like there's for some reason that's pressing a button frame late. I don't know why. It was only ever one frame late and not all the time. So it's super weird. I was kind of chalking it up to a dolphin bug maybe. There was some bug in the emulator. And so I eventually tried out this. This is what you're gonna see here is Fox doing frame perfect multi-shines. This is a Smash Bot. It's not just doing these blindly. It's actually reacting. So on the exactly the third frame of the jump animation, Fox is going to hit down B to start the shine, the little flashy animation and then jump out of it and then loop through it again. So what's important here is that he's even a single frame late on any of the inputs. He will jump accidentally. So it looks like this. So he's going along happily doing frame perfect multi-shines and then start jumping, he's screwing it up. And then go right back to multi-shining again. And then you start to notice that this is actually cyclical. This is like not random. This is happening on an exact like predictable basis. So he'll do it again in just a moment here. I was like, that's weird. I do like it when bugs are reproducible. So eventually me and Dan Salvato, another awesome melee hacker, figured out that this picture of how the game input thing works, was not entirely accurate. And so what happens is that the game input and the game engine processing are on separate threads and they're not perfectly synced up. And so what happens is on one frame, it'll look like this. On the very next frame, the controller input will drift by a tiny bit. And then the next frame, the controller input will drift a tiny bit until eventually they swap. And controller input is pulled afterward. So in the very beginning, you're going to press a button right at zero, right? The game will process without having read your input. Then it'll read your input and not process it until the frame afterward until eventually it would drift backwards. So then we put together a patch. I should say Dan put together a patch for actually like fixing this. He moved the controller input routine onto the same thread as the game engine basically. So in that way, we patched, we relived in memory a 15 year old bug in the game that up to this point no one has ever noticed. So that was pretty cool. So some of the bits about the future as it were. I wish I could have gotten this working for DEF CON. It's like 75% working is running on live unmodified console. So as it turns out, this is actually completely possible. And I was talking with Dwango who did the task bot talk right before this some of the parts about physically, like how do you send buttons presses over to the console. But one of the more interesting stories is actually how to get information out of the console. So remember, I wanna do this on an unmodified console. So I don't wanna just like put some like leads and open up a hardware, I'll open up the GameCube or something like that, right? And so in order to do that, we have to use a really fun exploit through the memory card port. So it turns out that in Mela, you're able to give yourself a little name tag like what my name is that's four characters. And of course, since it's four characters, people name it lots of colorful things. But if you go into the actual save file that's on the memory card port and change your name manually to be longer than four characters, overflows the thing. And you can get code execution on the game. And so there's already people that have been exploiting this and using them to make modifications to the game. If you've ever seen the 20XX hack pack or 20XX tournament edition is one that uses this name tag overflow. So it's a great way of getting code execution on the game which we can then use to grab game state information and ship it off over the memory card port, which is then attached over USB to a laptop. That way we can get live frame data out of the running machine. So we then put Smashbot inside of a controller that would then be pressing buttons. And you would just be sort of looking like you're playing the game like normal. And you would never notice that it was computer playing unless you looked closely, notice that Smashbot, the controller, was plugged into the memory card port instead of the controller port, or probably both actually. So before we start getting back to the end part here, I want to impart on you a little bit of Smash philosophy. So being a part of any competitive scene for sure imbues you with a certain amount of the philosophy from that game. And so I want to share this with the hacker world. John, what's a John? It was like 2 a.m. and he was tired. John's like John's. He'll get me on a days where I'm just not playing too well. John's, just John's. A lot of people don't know where the term came from. It just started, but I believe it was a guy in Texas. His name was John. And no matter what, every time you lose, he'd have an excuse. He'd have a reason for losing. My controller wasn't working the stage. There's a little bit of lag on the TV. I didn't sleep last night or I don't know why I'm not comfortable with my hands. It's too cold. I need a warm up. Yeah, I don't have a warm up. We have like a Swedish term, inga, you know, it's pretty much no John's. He used to like using much John's back in the time. My favorite one, I think, was I was playing somebody and they were like, someone's touching my shoulder. And I was like, no John's. So yeah, no John's. And thanks a lot for hiring. You've got something along the lines of eight minutes. So I'll go ahead and start the game up again. If you want to line up here, well, I guess we'll have to do two lines. One for playing the game and one for questions. And we'll do that right now. If you want to take questions, there's a microphone right there. You're gonna have to, otherwise I will not be able to hear anything you're saying. Here, let me set up the game first, actually. Hello? Can we see SmashBot versus SmashBot? Yeah, so that's a question that people actually ask a lot. Is like, what would happen if SmashBot played itself? Or like, the, right now, there's just a small logistical problem with it, which is just that it only knows, it plays on player control report two and it assumes that its opponent is on player control report one. And so there's just that. But I can get that solved. But the more interesting question is like, what would happen if it played itself? Or like, what is like truly perfect play look like? Let's just give it 70 minutes. Here. So it turns out that optimal play, I gave quite a bit of thought to this, is a really, really complicated. And this is actually a good question. So in case you're sitting in the audience thinking like, hey, I bet I could make a better SmashBot that would like beat this one, right? Well, let me take you on a tour of what actual optimal play looks like, right? So first off, all projectiles can be reflected. There's a two frame window at which point you can reflect projectiles. And so all of those are suboptimal. And so the only way to like attack is to just basically walk forward. And so the fastest move in the game is Shine, which is Fox's downbeat attack. And both Fox's, both bots would basically walk at each other until they're exactly within range and both use their perfect one frame move at the exact same moment. They would climb off of each other, not hit. And then it's a deadlock from there. At each point, both the optimal play for both characters is to jump and then to frame perfect multi-shines until the time limit runs out. When the time limit runs out, the game goes into sudden death. At sudden deaths, like the game goes for a little while and then bomb bombs start falling from the stage, sort of like randomly, right? And so it would be possible to put your opponent in a position where they have to either run at your attack or into the bomb. So it would be kind of sort of random. But they're not actually random, right? They just use the in-games random number generator, which is entirely predictable. So back up, the optimal strategy is not just simply run at your opponent and Shine, it's to put yourself in a position where once you deadlock your opponent into that Shining, you know that in exactly eight minutes from now, the random number generator will be seated such that the bombs will fall in a way such that you can put them in a disadvantageous position. So before you go around thinking, I'm gonna make the perfect bot, know what you're getting yourself into. Yes? No and no. I'm sorry, the question's at the microphone. Yeah. You taught him to do taunts at the most insulting times. Yeah, that was like, it didn't do taunting for the longest time, it just sort of sat around. But now it does frame perfect multi-Shines in between stocks as the like, how to taunt basically. I figured that would be a pretty cool way to do it. Yeah. So you mentioned that the beginner players will like confuse this. So how do you go around that you use machine learning or just keep on programming? Yeah, there is actually a separate machine learning fork of Dolphin called Philip. I wish I had more time to talk about it here. That uses the Google's TensorFlow neural network library. At first it had a really hard time doing more than just kind of moving around, but it's actually getting pretty cool now. And so one of the neat parts about SmashBots design is that those like lowest level like chains, like maybe there's no need to make an AI learn how to wave dash all on its own, right? Why don't we program that in as a primitive and then use AI to kind of choose which lowest level primitive will be best. And so that's actually like a goal of mine for the project is to do exactly that. This is about as far as I've taken it like right now, but it's absolutely, actually I should have mentioned this is an active open source project. It's available on my github, just github.com slash alt f4 or just Google for this basically and you'll find it. Yeah. You mentioned that with the game in the future you had plans to have this run on an unmodified console. Yeah. Do you anticipate that you'll be able to overcome, like was it strictly on the emulator side with the drift problem, with the controller that was causing the bug where? So it is actually a bug in the game. That isn't like, so the game is responsible for that frame loop and the controller polling. So that is actually a bug in the game. That said, we haven't been able to empirically verify that, right? So in theory that bug should be present on console, but without Smash, there's really very difficult, there's basically no way to know without like verifying that via like maybe some task wave doing it, but Smashbot would actually be the best way of verifying that because it is reacting in real time to the frames rather than just like having a scripted set of button presses. Do you anticipate that there will be some way to maybe overcome that so that you can have that? Oh, exactly. So we can code execution on the game, right? So we can just modify the running game to fix the bug, just patch it live. Okay. Yeah. Thank you. Hey, first question. Are you coming to Super SmashCon? I actually only discovered that SmashCon existed after the CFP closed. That's too bad. So I really would have liked to have done this over at Super SmashCon. I'm based out of the Phoenix area, so if everyone wanna like play Smashbot near me, if you wanna run it yourself, just download the source code, run it yourself. Otherwise, I'll be around in the kind of Phoenix area. I'm hoping to take this out to a larger tournament sometime in the near future, but I have a busy travel schedule of work and stuff like that, so no promises. The other thing was, I just wanted to say thank you so much for figuring out that bug, the 3.5 to 5 frame thing, because I actually had an idea for a project a long time ago where I was like, all right, I'm gonna take a high FPS camera, solder an LED to a controller and figure out the amount of input lag difference. Yeah. And there's so many problems that happen in the analog world that like there's, yeah, it's really difficult. This way it doesn't matter what's going on the screen, Smashbot's reading the live bits out of memory. So the very frame that something happens, it knows about it, with taking the entire analog universe of display refresh rates out of the equation. Yeah, but it's so awesome that you guys figured out that bug and I was just wondering what went into it because like I would have been like really freaking confused and I have measured. Oh, I was really freaking confused. I have measured the FPS lag and you can see it, it's like every like quarter frame, something like that, it just takes longer and it makes no sense. That is absolutely correct. So thank you. You bet. I had a couple of questions. So can it be anyone else other than Fox or does it have to be Fox? Smashbot plays Fox and probably will for the indefinite future. It's clearly at this level, like at the task level, the best character in the game. It's just faster than every other character. One could make an argument for Falco, but I'm not so sure. I mean, it is kind of an open question about what is optimal play, like at the highest levels. Who knows, maybe like if you could play at this fast Donkey Kong is like super broken. I don't know, right? I doubt it. There's good reason to believe that Fox is the best character and so this is my best stab at making that happen. And what about having Smashbot play like three other characters at once? Simultaneously, yeah. So right now it only acknowledges the existence of the player one because I just wanted to make that work first. I suspect that's just a losing battle. Like once you actually have 3v1 at the theoretical level you just lose because even though I can frame perfect shield stuff there's lag after the shielding and so like you could just hit me after that happens. And one last question. Is there any possible plans for other fighting games that you would use this for? Smash is really the only game that I personally played competitively like at that level. So not for me, but there actually are of similar projects and other AIs for other games. There are Starcraft and Starcraft II AI tournaments that actually happen. So this very similar sort of endeavors in that world. Thanks. So you said that this bot is supposed to be able to mimic human behavior? It's supposed to be more, if you look at it, it's supposed to, my question is, when I notice whenever you die it goes left and right really, really fast. Yeah. Was that on purpose? Was that just? Yeah, so it does, which is called a dash dance, right? It just moves, moving back and forth. For the first, it depends on each character. I think it's seven frames or maybe 10 or 11 or something like that for a fox. The first, when you start up running, you're in a dashing animation, which point you can dash backwards very quickly. It's a good way of keeping mobility based. It's something that even high level players do, but never with that exact amount of precision and that amount of speed. I guess to the earlier point, Smash Bot is not intended to make you feel better. It's not meant to play like a human. It's meant to play like a computer. In the same way that like an aim bot, for like a shooter, does not play like a human would. And so we're trying to break fundamentally how the game is played at that level. So if you're playing like a shooter game with like team-based strategy, there's a lot of high level thoughts in terms of like by getting your opponent to use, like are you using cover, getting your opponent to like move into the center stage. But if you're a computer, optimal strategy is to stand in the center of the stage, spin 360 as fast as you can and then blame people in the forehead the very moment they come out, right? And so Smash Bot's kind of taking advantage of that it's not trying to play like you do. It's trying to play like a computer does. Thank you, and thank you, let me try it out. Absolutely, I don't know, is that time? All right, thanks a lot for coming out.