 So today I'm going to talk about writing a solver for a game called Ricochet Robots. This is a board game. I've got a couple copies if you want to come and play after, find me in the hallway, we'll set up a table somewhere and play. Super fun game. So why would I want to talk about writing a solver for this game? Really, I wanted a solver for the game because we play to work all the time and we never know if we've got the best solutions. And of course the best way to do a side project is to propose a conference talk about it and then get accepted and then madly write the solver and then talk about it all at the same time. That's the best way to get a side project done, right? But really what I'm going to be talking about is some computer science stuff, graph search algorithms and things like that. I learned this stuff back in college, but not everybody knows this stuff. We had a local Ruby users group meetup where we were working on maze solvers which use very similar algorithms and very few people knew these algorithms. So I thought it was worth talking about and the game is a fun way to do that. So why do we care about graphs and graph search algorithms? Graphs are very common data structure in when you're modeling business organizations. Actually in a Rails app I'm working on, we just had a problem where we needed to use graphs and a couple of these algorithms to solve a problem we had. Relationship maps like your LinkedIn connections because I'm sure you all have LinkedIn connections or Facebook, any kind of social network. That's a graph structure. Travel planning, trip planning, network routing. All of these things use these kind of data structures and these kind of algorithms. Obviously much more complex than the ones I'm going to talk about but still they're important so it's good to know about this stuff. So what is ricochet robots? As I said at the board game you have a 16 by 16 grid of cells. There's these colored target cells on there and you have a set of five robots distributed around the board and the idea is you pick a, there's actually 17 little discs with the goal symbols on them and you go through them in order, in a random order, sorry, and the idea is you have to get the proper colored robot into the goal cell. So in this case we're looking for the green square cell and we need to somehow move the green robot into that cell. Now the robots can only move in straight lines and they can't stop unless they hit something. So you have to run into a wall or another robot before you can stop and change directions. So all, what's that? Yeah somebody called 7. And so you can move any number of robots and so everybody starts at the board and figures out how many moves that would take them to solve this, calls it out, whoever gets the best answer within a certain amount of time gets to go. So here's an example solution. We're going to move the blue robot first and then we're going to chase the green robot around after it, bounce off the blue robot and down into the goal cell. That's how the game works and if you're like me you can immediately say, hey I want to write a solver for this because it seems like you should be able to do this, right? But before you embark on writing a solver like this you need to kind of characterize the problem and decide, you know, just what am I dealing with here? So first of all what's the size of the problem space? There's 256 cells, four of them are already always taken up by that center island. So there's 252 cells and five robots. That means there's 976 and a half billion possible board states, different positions of all the robots. That's a pretty big search space. The other complicating factor is the branching factor. From each board state there are anywhere from 9 to 20 possible moves, you know, five robots, four possible directions, that's 20. Sometimes they're in corners, sometimes one is just moved in a certain direction, so it could be as few as nine. That's a pretty wide branching factor. And because there's so many states you have to think about how am I going to represent this board in memory? You don't want to take up too much memory if you're going to have a lot of states in play at any one time. So I studied the board a little bit and thought about it. The grid itself is fixed. It's always this grid with the center island. That's static. You really only need to represent that one time. At the next level are the walls and targets. You can actually have a bunch of different board configurations, but for the duration of one game this is fixed as well, so you really only need one copy while you're solving the game. A little more variable are with the goal, what target are we shooting for right now. That changes once per turn, but it's still relatively static. And then there's the robot positions and those change all the time. So in the solver I wrote I ended up representing a board state as a combination of the goal cell and the robot positions and that's how I represented it. Okay you also have to represent robot movement and in this case the board states are the nodes in the tree and each edge of the tree is the movement of one robot in a certain direction. So the red one moving left or right or whatever. So you hopefully get the idea. So what is a tree? A tree is a fairly common data structure in computer science. You have nodes which I'm representing as boards here and edges, the lines between them. And to make it a tree every node has at most one parent. Okay so the root node at the top doesn't have any parents. Every other node in the tree has exactly one parent and that's what makes it a tree. You have any number of children but one parent. When you have trees you generally need to search through them and that's where search algorithms comes in. If you want a really accessible introduction to search algorithms, see James's book that he just wrote. I read it when he was writing on his blog and it's awesome. Very good accessible. He's talking about maze solvers but the algorithms are very similar like I said earlier. So I highly recommend that and James is speaking tomorrow so I'm guessing he's going to be talking about some of the same stuff. So highly recommend it. So the first search algorithm I'm going to talk about is kind of one of the most basic ones. It's called the depth first search and in depth first search you search all the way down one branch of the tree and then you back up and try a different branch. So that ends up looking like this where we go all the way down to the bottom of the tree back up and over up and over and that's a depth first search. Okay very simple algorithm the program it's a recursive algorithm. So I've got the solve method where I'm taking an initial state solving recursively and as I'm solving I'm building up the list of candidate solutions and then I find the shortest solution and that's my answer. The recursive solution you just look at the path is it a solution if it is put it on the list of candidates return otherwise find all the successor paths. All right sorry about that. Okay so you generate all the successor paths and put them or then solve each of those recursively and that's a depth first search. Now in this solver I started out solving for a single robot just because that was an easier problem to work from and the first problem you run into is the idea of a cycle. So you can get a robot up in the corner here and it starts going around in circles. So you have to guard against that you want to make sure that you don't end up in an infinite loop because you'll never find your solution. So you can think of maybe well let's keep track of all the board states that we've seen before and not process those again but the problem with that is that you can sometimes get to the same cell via two different paths. So here on the left I've got a board where it took four moves to get to that cell and on the right a path where it only took three moves to get to the cell and if my goal is that green circle I've got one more move to go if I if I start with the board on the left and then I find I got to the same cell I've been in before I throw that away and I just missed my shortest path so I haven't actually found an optimal solution. So what you find is that this nice little tree I showed you is not really a tree at all. It's actually a graph where nodes have multiple parents so in this graph I've got a length two paths of the same node and the length one path so you have to watch for that. The other complication is that these algorithms I'm talking about are designed to find the optimal shortest solution but in the rules of ricochet robots there are some short solutions that are not legal. For example if the robot starts on the goal cell already that's not a legal solution you have to leave and go back or if you start one move away that's not legal either you have to actually ricochet once before you get to the goal cell. So here the best solution is to go left down right up now I'm back where I just started that looks like a cycle right except that because that first move was not a legal move I have to not count this as a cycle so it's a little bit tricky then I move into the goal cell that's actually the shortest solution for that board. So there's ways to solve all of these but really this is a bad algorithm because if you if you have to search all the way to the bottom of the tree you have to search the whole tree the whole state space in order to find the shortest solution and with 976.5 billion states now we're not doing that we don't have time so we need a better algorithm so the next algorithm I'll talk about is one called a breadth first search and in a breadth first search instead of working down the branches of the tree you work across the levels of the tree and that's so that looks like this so we look at the zero move solutions then one move then two moves and then three moves the nice thing about this algorithm is it's guaranteed to find the shortest solution first because you're always looking one move then two then three then four and so on so that's nice so the code ends up looking like this and use a queue to help you implement this so I put the initial path on the queue and then I pull the first one off the queue if it's a solution I return it I'm done I don't need to search any of the rest of the tree otherwise you put the successor states all on the end of the list and you move on so with this algorithm we can actually use a global visited list that didn't work with the depth first search because if we get to a state we've already seen before we're guaranteed that we got there faster the first time or at least as fast anyway so that's a nice optimization we don't have to process states multiple times like we did with depth first search but even with this this was not fast enough this can't solve all the boards fast enough so it's time to optimize the global visited list is one form of optimization but really there's three kinds of optimizations we can do one is to do less things we want to somehow try to reduce this the size of the search space another is to do things faster and this is what most people think of with performance optimization where you're just trying to speed things up speed up your code make it run faster run a run a profiler just less work per state with search algorithms there's a third thing you can do and that's introduce heuristics which are basically rules of thumb the thing about heuristics is that unlike the previous two optimizations which are very objective and work all the time heuristics don't always work and they kind of some some states it may work some it may not and so they're they're much more variable often you can find good ones that work most of the time but they don't always work and so you have to be careful with them so when I was working on on my solver the first optimization I tried was actually heuristic and I was thinking about the game and the last move you make is always to move the active robot into the goal cell so why don't we try heuristic where we always check the active robot first because when we get to the level where we're going to find a solution we'll find the solution fastest by moving the active robot first so I tried that with this graph represents is all 17 turns in a game there's several places where the two lines coincide and that's because they both move the same robot first most of the time the red line with the triangles which is with the heuristic in place is a little bit below the blue line so it most of the time helps but there's that one outlier out to the right edge where it was way worse and I don't know why it was like that but just kind of illustrates the point that heuristics don't always work I kept this heuristic in place because overall it seems to be a win I need to analyze it a little more to make sure it really is enough of a win but it did kind of skew my results a little bit because of that one outlier so then I started moving on to more objective optimizations and I realized that I was processing way too many states you remember the algorithm I showed you earlier we check a path force whether it's a solution when we pull it off the front of the queue but what if instead we checked it before we even put it on the queue in the first place so originally to find a solution at node 16 here I had to search all 16 nodes but if I change my code to look like this where I generate the successors and then I check whether any of those is a solution before I put them on the end of the list what happens is I can find the solution at node 16 when I'm processing node 6 so I've knocked 10 nodes off of the search here and you can imagine with a branching factor of 9 to 20 at each level and you get down eight or nine moves in the tree this is a pretty big savings and it turns out it was this is the number of states that I had to consider to find a solution for my example game and it was almost a factor of three improvement a reduction in number of states considered so doing a lot less things and that sped things up quite a bit then I started running a profiler and the next thing I found is that most of my time was being spent trying to figure out where the robot was going to stop because I had a really dumb algorithm I was just going can I move can I move can I move can I move oh I got to stop there and that was where all the time was going and so what I did is I pre-precomputed the stopping cells for the robots so for each cell on the board and for each of the four directions figure out if I move that direction where am I going to stop and and I didn't consider other robots in this pre-computation and this is a classic trade-off that you make in optimization a lot where I needed to use extra storage space to store these pre-computed cells but it made the algorithm much faster so I was trading space for time and that is a classic optimization technique even in your RailsOps database indexes you have to take up extra storage space to store the index but it speeds up your queries drastically so you're trading space for time very common optimization technique and as you can see this sped things up this is the number of states I'm processing in a second and it was a pretty good jump without optimization then I did a little bit of research and I found another talk by Michael Fogelman where he was writing a ricochet robot solver and he made the observation that there are some board states that are equivalent even though they don't look like it so here's two boards the robots are in exactly the same positions but it's different robots in those positions if I'm solving for the green robot it's in the same place I don't really care what color the other robots are it just doesn't matter because I can bounce off the red robot or the blue robot to get where I'm going I don't care and so I was able to implement what I call a board state equivalence class that that treated these two board states as equivalent and that was a slight improvement a few less states to consider but also in order to implement that I came up with a more concise representation of a board state equivalence that was faster to compare so I actually ended up doing things faster at the same time and I was able to process more states per second so sometimes the optimizations you do are going to have both reduce the amount of work you have to do and speed things up the board state equivalence class I implemented I implemented as a set of position representations and I realized that the second person was taking long a long time and so I switched over to a sorted array instead which is actually faster to compare another slight speed up the next thing I found was I was creating a lot of unnecessary objects every time I tried to move a robot whether it moved or not I created a new board state that I would then have to compare and so I stopped creating new states when the robot wasn't actually going to move if I'm up against a wall and I try to move I'm not going anywhere why create a new state for that that was a huge win so definitely creating less objects is another good optimization technique and finally I found some places where I could compare objects by identity rather than a DP quality and that was another speed up now the solver is actually pretty good but it's not good enough yet there's still some places where it's slow but at some point I had to stop working on the solver and actually put together some slides for this talk so that I actually had something to talk about so this is still a work in progress but it's pretty good so here's where I ended up with the optimizations I did this is the total solving time for the example game I was running and you can see it sped up quite a bit those first couple of algorithms were pretty naive and pretty slow and with these optimizations I just talked about it sped up quite a bit so let me show you a quick demo of the solver in action it's very text-based and I'm running an example that I know how fast it runs I stole a trick from Minitest thank you Ryan where I print out the random number seed that was used so I can feed it back in and run the exact same game over and over again that was great for testing this so it's a great trick so let me just run the solver this is going to be playing an entire 17 move game by itself I can guarantee you I cannot play the game this fast so blue square is a little bit of a slower one I think it's actually a nine move solution there we go about 30 seconds to play an entire game of ricochet robots so like I said there's still some board states where it's not quite that fast but it's pretty good so how else might I improve this thing well obviously I could try some better algorithms there's a very famous algorithm called a star it's kind of like the breadth first search I just show you but it's actually more of a what's called a best first search and what it does is when you have a list of states to consider instead of just choosing the one that has the shortest distance traveled so far it adds in a factor which is an estimate of how much further there is to go and as long as that estimate is never an overestimate as long as I always estimate low then a star actually comes up with an optimal solution so the way that would might work we'd have to come up with some some way of coming up with an estimate of how far there is to go and I got this idea from Michael Fogelman's talk as well you can kind of you can compute if the robots could stop anywhere where they wanted to how many moves would it take him to get to the goal so obviously if you already have the goal that's a zero anything in a straight line is a one anything in a straight line from there is a two and then threes and fours and in this board five so from those cells mark five even if the robots could stop anywhere they want it would take five moves to get into the goal cell from there just because of the structure of the board and where the walls are so that's actually a pretty good estimation function for how many moves there are to go and so the way the algorithm would work is let's say I've got one board state where I've already made four moves but I estimate one more to get into the goal and I've got another state where I've only made two moves but I estimate five more to get to the goal well four plus one is less than what I say two plus five so I would actually consider that state where I've already made four moves first I would try that one first and this actually should speed things up a little bit I haven't tried this one yet it's probably my next one to try but that's a star in a nutshell maybe collision detection algorithms for figuring out when robots are going to run into each other or into walls maybe that would be an optimization I haven't tried that yet that's an idea another thought is to take a different approach and maybe work backwards so you look at the goal cell and then figure out where's all the possible places I could stop before I get into the goal and then work back from there maybe that would be a better algorithm I don't know I kind of use this one when I play a fair bit I kind of work backwards a little bit whether that works for a computer or not I'm not sure I could also try some better heuristics for example most solutions only involve two maybe three robots so what if we always move the most recently moves robot first and try that maybe things will be a little bit faster or maybe some combination of moving the active robot and the most recently used robot maybe we could be smarter about which directions we explore first like maybe from this state moving left is more is is likely to be better than moving up or down and so we we choose more intelligently that way what my profiler is telling me right now is that the slowest part of my algorithm is figuring out whether I'm going to run into another robot so I've got the pre-computed stopping points with the walls but that doesn't factor in where the robots are at that time because they move a lot more frequently so the pre-computation didn't seem like it would pay off as well but maybe I could pre-compute per robot stopping positions so that's an option could always try to use less objects and more primitive types you know sometimes creating a lot of objects is really slow and if you could use primitives things would go faster then there's always the ever-popular throw more hardware at it parallelize things process states in parallel and see if we can find an algorithm faster so those are just some of the ideas of where I'm going to go with this next maybe you got some ideas for how this could be optimized love to hear about them I want to thank a few people um Trevor Yarish who's one of the founders at zeo and also one of our designers did all the design work on these slides and I think he did a beautiful job the animations where he did those um so I'm very thankful for to Trevor for the time he put in the other guys at zeo paired with me on the solver a little bit gave me ideas and feedback so thank you to them I mentioned Michael Fogelman a couple times his talk was really interesting so I got some neat ideas from there and Trevor Lailesh Manaz the guy who introduced me to this game at RubyDcamp a couple years ago and so I thank him very much because I love playing the game it's a lot of fun code for my solvers up on github I'm going to keep working on it but if you want to play with it feel free to fork it and play with it and I've got a few minutes for some questions if anybody has questions all right thank you very much