 Thanks all for coming today, I'm Cameron Dutro, my talk is cheating with Ruby. I work at a company called Lumos Labs, we make a product called Lumosity, yep. Four of my wonderful co-workers sitting down here in the front row, I've been wonderfully supportive, thanks guys. You can find me online, GitHub, Cameron Tron and Twitter at Cameron Tron. So a couple of personal things, those are things I am, these are things I am not. Not a mathematician, I'm also not a computer vision expert and I am most definitely not an LA Rams fan, Seahawks all the way. I got married this year, that was a lot of fun, thank you, feels good, that's my wife Kelly, this is my cat Chester. I figured that if Gorby Puff Thunder Horse gets to be in slides, Chester should get to be in slides too. So let's talk about cheating, chances are if you have any moral compass, you understand that cheating is wrong, right? So why are you here? Well because hopefully you think I'm super handsome and you understand that occasionally cheating can be a good thing for educational purposes, right? So cheating in this case was my opportunity to explore some elements of Ruby that I hadn't really gotten to explore before, like image manipulation and computer vision, things like that and it also helped unblock me from this game called Pet Detective which we're going to talk about in some serious depth. So the game, this is a Lumosity game, if you haven't used our app before we have about 35 or so mobile games, they're also on desktop and this is one of my favorite ones. These are brain games by the way so this one's designed to help you improve your problem solving skills. So Pet Detective is a pretty simple game, your goal is to return all pets to their houses and you can see on the screen here a number of pets and all their houses, of course the image of the pet appears in the front of the house and then you have a car that drives around and picks those pets up and drops them off. You block that you travel on that black road there, costs one gas, you have 25 gas in this screenshot, that's the number that appears over the car, yeah so you have N gas units for the round and if you run out of gas before you've dropped off all the pets to their corresponding house, you lose the round. Your car by the way can only hold four pets at a time, that's what that was a little bar at the very bottom of the screen next to the replay button and that's where your pets will appear when you pick them up. The car also automatically travels along the best path so if you were to, you don't tell the car where to go in terms of up down left right, you simply tap on a pet or a house and the car chooses which path to take and it's always the most, it's always the shortest path I should say and that's a pretty key part of the whole thing. So let's run through an example of how a round might work or how you might pick up a pet and drop it off, so the first thing I'm going to do is pick up the turtle, so we have 25 gas for the round, there are no pets in my car currently, you can see that at the bottom, then it costs four gas, I go to the right, then up, and then two to the left that uses four gas, I pick up the turtle, the turtle is now removed from the game board and added to my car. The next thing I'm going to do is go up one square, pick up the Siamese, so right so I set the cost of four gas, now the turtle's in the car, I'm going to go up one to pick up the Siamese, so now that cost me one more gas, so a total of five gas were used, and now the turtle and the Siamese are both in the car, so the next thing I'm going to do is drop off the turtle, his house is two away, so now I have 18 gas, the turtle is no longer in the car, then I'm going to drop off the Siamese, he is six gas away, and now he's also no longer in the car, so now I have zero pets in the car, I have 12 gas left to complete the rest of the board, so fairly simple game mechanics, hopefully that illustrates how it works, and this is sort of how I progress through the game, so for levels one and two, drop dead simple, you can't really lose, because it's supposed to teach you how the game works, so those are almost sort of boring and interesting, you can progress through those pretty quickly, levels three through five, I was like yeah, I totally got this, like I can see the whole thing in my mind, I know where to go first, I can count the number of gas I'm going to use ahead of time, levels six through eight, we're like okay, this is getting a little more challenging, I'm not getting everyone right away, and then levels eight to ten, I was starting to sweat a little bit, like this is really hard, by the time I got to levels 11 and 12, my brain was just exploded, because it was just really hard to figure out how to solve these puzzles, in fact it got so bad that I couldn't get past level 12, and that's what sort of spurred this whole talk and this whole thing that I worked on called Pet Detector, so I was also thinking the game has 20 total levels, and I was thinking, if I can't get past 12, how am I going to get past 13, 14, and so on, and probably turn into a zombie or something to try and do that, so I thought to myself, what's the problem, what do I want out of this game, how can I help myself progress, and I didn't want anybody to give me the right answer to the level, because that is like legit cheating, right, I didn't want to be able to progress through it without actually working, but I also didn't know how to move forward, I didn't know how to get past level 12, and I wanted to learn how to play the game smarter, there must be something I'm missing, I thought to myself, so what techniques am I not thinking of, what inherent concepts can I use, how can I change my concept of how the game works to understand how to play it better, so as an example, is it always better to pick up the nearest pet first, I have no idea, maybe, maybe not, and I want to know the correct order really after the game is over, so I want to know what I did wrong, essentially, is the real question I'm asking, so I got from, I don't want anybody to give me the right answer and sort of noodle through it and found out that what I really want to know is why I usually go wrong, so how can I get to that point without some person coming up to me and saying, this is how I solved it, well, I know Ruby, I know how to write Ruby code, I know that I have heard at least that I can play with image data using R magic, which is a really cool Ruby gem, by the way, I know the game is laid out in a grid, so I know that each individual quadrant of the game, I know that there are certain things happening in there, there's a pet, there's a house, there's a road behind the pet and house, there's a car, right, and I know that the game has parameters, right, so there's a number of gas units, there's a number of pets in your car, et cetera, and those sort of limit the total solution space that you can use to solve the game, so I wrote a Ruby gem, it's not really published anywhere, it's sort of just meant to be an executable in your system, it's called pet detector, and what pet detector does is analyzes a screenshot of the game, it extracts the locations of all the pets, houses, where the road is, it builds an in memory map of the entire game board, detects via, I should say a dykstra search algorithm, it finds the shortest distances between all pets and all houses, so we have a map of how far apart each of these things are, and then using sort of a brute force, a modified brute force algorithm, it tries to solve the board and give you the order in which you're supposed to pick up pets and drop them off. This is sort of the main way you would invoke this, the script, you pass the file, the level, and the gas into it, and then it spits out the correct answer. Okay, so I am about to do a live demo without telling you how this completely works yet, this may not work, so just, you know, we'll see what happens. So the first thing I'm going to do is fire up my terminal that's running a copy of our back end, here's the pet detector repository, and here's my iOS simulator. So I'll go to games down here, it's a problem solving game, here it is, let's play the game, and I'm going to do level 12 because that's the one that I struggled with. By the way, this is a fake simulator version, so I haven't made it all the way to 16. I'm still working on that. Okay, so the game starts up, we get our first board, and what I'm going to do is take a screenshot that saves it to my desktop by default, and then I'm going to say bundle exec, bin slash pet detector, minus F is a shortcut for the file, dash L, level 12, dash G for gas, 22, and if this doesn't work, we'll just restart and try a different one. Okay, so it just gave us an order in which we should pick up pets and drop them off, and by the way, just so you don't think I'm cheating in the presentation, the screenshot that it created is exactly the same one, it's right here, and this is just a PNG file that the script ingested and found the answer. So let's see if this works. So pet cockatiels, the first one, I'm going to go there, house cockatiel, pet husky, I'm not sure whether husky is yellow, pet turtle, house husky, pet doxent, house doxent, pet hedgehog, pet siamese, pet tabby, house tabby, house turtle, house siamese, house hedgehog, and it's done. Okay, so what's going on behind the scenes, let's do it one more, or actually we'll do, yeah, we'll just do one more demo in the end of the time, I'm going to delete that screenshot. There's a couple of options that I want to invoke here in the command line that will show you some of the, some debug output as well as a simulation of how the game can be solved. I'll just use this other round, there are three rounds per game. So we'll save a new screenshot this time of the new round, and then who saw Ryan Davis's graphics talk that he gave on Monday, I think it was Monday. Is Ryan Davis here right now by any chance? No, that's too bad. Okay, so I saw that talk, I thought it was awesome, I thought his graphics library looked super cool, so I thought why don't I try to use it in the next couple of days to illustrate how as opposed to just printing out some text telling you what order to drop and pick up and drop off, why don't I do a simulation showing the actual route that the car would take. So there's two debug options we're going to turn on right now, the first debug is true, and then simulate, it was true, oops, you know what, I need to use the new file, still level 12 and 24 gas. And this time we should see not only the answer spit out, but also some debug output showing you where all the houses and pets are, and then a simulation showing how the car would pick all those pets and drop them off. Almost done, nice. Okay, so that's pet detector. Now, does it work every single time? No, but I'm actually super glad that it just worked twice in a row for you guys, it's pretty great. And sort of in anticipation of it working I had to throw in an integratory success kid. All right, cool, so glad that I could show this slide. Okay, so how does this thing work? Well, there's a sort of a series of steps that the code goes through in order to analyze the game board and then eventually solve it. I don't wanna read all these off, you can read this yourself, but we're just gonna mark through the presentation, little green check marks will appear next to these as we talk about them. So the first one is to identify the boundaries of the game board so we know what sort of space we're working with in 2D space. And actually I should also say I'm gonna show that sort of the top level API design of the system. So this is what the bin pet detector executable is doing. So the first thing we do is look up level information. So I have hard coded in a YAML file, some level information such as how many rows and columns there are per level, and then the pets that are present in that level as well. And that helps to know if we've found them all yet, if we haven't found them, that kind of thing. We load the bitmap, and this is just a utility class that wraps our magic image, our magic image. We use the thing called the boundary detector to find the outer boundaries of the game board. Called getBounds on that thing to find a rectangle, so it's a top, bottom, left, right, width, and height. And make a grid, so we split up the image into a number of grid quadrants. And then we use the entity detector. This is sort of the hand wavy magic part of this, right? And the entity detector finds the animals and pets, and then returns them as a matrix that maps to the game board. And then we use the solver passing in the number, the entity matrix and the gas, call solve on it, and then that gives us a solution object, which is another matrix, or I should say it's an array of pets and houses that you have to visit. Okay, so let's go in depth here. So we're gonna talk about the boundary detector class, which identifies the outer edges of the game board. Well, the grid class, dividing that up into quadrants, there's the animal, and track detector classes, which do exactly what it sounds like they would do. A track, by the way, that's sort of the internal name I use for roads, and then the solver solves the board. Okay, so the first thing we wanna talk about is the boundary detector, and this thing finds the outermost edges of the game board, and it does that using something that I sort of came up with, I don't know if this is something I came up with, I don't know, probably not the only one, but it's called probing, and I'll talk about that in a second, and then we add padding around the edge so that every quadrant is the same width and height. So in this case, in the screenshot, that's the outermost edge, so it's the outermost where the track is versus where the background is. And then we add padding to that, as I said, to make sure that every quadrant is the same width and height. Okay, so how does probing work? Well, we start at one edge, and by edge, I mean the very edge, so x equals zero or y equals zero, and then we stop when a pixel of a certain color is encountered, and that's where the probe stops. It's sort of downward or left or rightward trajectory. So in this case, after the boundary detector uses seven probes from the top, bottom, left, and right, here's an example. So from the left, this first probe goes along looking for black, it doesn't find it, because it's just too high, that's okay, we'll do another one. This one finds the track at the correct location. The next one does the same thing. The fourth one unfortunately skips a little bit past it because there's a house in the way. But the roof of the house has a little bit of a black segment in it, and so it stops there. The next one goes all the way to column number one, not zero, number one, which is also wrong, of course, so you can see sort of the problems you run into here. And this one goes all the way to the other side before it finds black. Just so happens, that's how this one will shake, sort of shook out. The last one, of course, does the same as the top. So after performing seven of these probes, you look and find their max, or their minimum, I should say, their minimum x value, and that's the x value that we use for the outermost left-hand side of the board. Same thing happens at the top of the board. We do one from the top, two from the top finds the border, three from the top goes past the husky, so that's wrong, of course. That one hits the correct spot. This one stops all the way at row number one, and then this one, of course, hits it, and the last one goes all the way to the bottom, like the one on the left. So again, we take the minimum here, and if we were going from the bottom or from the right, we'd use the maximum value. So the code for this is pretty straightforward. There's a probe function, and that probe function asks where you wanna start for x and y, how much you want the x and y to change as the probe progresses left, right, down, or up, and then we loop until we find the stop color or until we run out of space. Like in the case of the first top line, it would run off the edge, and so we wanna make sure we don't blow up and try to access a pixel that is out of bounds. So this is the base probe function, and we have probe x and probe y. I'm only showing probe x here, and what this is doing is saying, if this is specifically probing the x dimension, so start at x, x start, change by x delta, there's a y start and a y stop to make sure that you can position y where you want it to go, and then just, it never changes, except for changing it slightly per probe. So as we iterate over probe count, we do a new probe, we increment y a little bit, we march x across, that probe is finished, we increment y just a little bit, do another probe across x, and then keep going. And then each individual direction, we have a probe function for that too, so this is probe left, and this is probes along the x dimension, and once it's finished collecting all the probes, it takes the minimum value from those. I'm not sure why there's a compact in there, I think that's just left over from testing. Okay, so we've identified the boundaries of the game board. So the next part of the algorithm is to divide the game into, or the board I should say, into equal size quadrants. So this is pretty simple, we know from the level data that there should be, in this example, four columns and five, six rows. So we know that if the width is a certain amount, we can divide that by six, and that's the quadrant height, and we know that we can divide the, or that's, sorry, the height by six, that's the quadrant height, and we can divide the width by four, and that's the quadrant width. So once we've gotten this all finished, and by the way, this is the code that does this, we just sort of do an in-memory overlay on top of the game board. So we know the outer bounds, or the outermost track bounds, and then we just sort of superimpose rectangles on top of that, each one corresponding to one of the quadrants. So I don't wanna dive into the code too much, because you guys can look at this later, and I don't wanna read it to you either, so we'll just move on. It's a little dense, we'll move on. Okay, so this is what you end up with. If you save each one of these quadrants out to a file, you end up getting these little squares, usually squares, and they each have sort of some features in them. So I'm only showing the first four rows and columns here, sort of blown up, but you can see track very clearly, you can see where track enters and exits, you can see each animal, and you can see the house, you can see the car. So this is sort of the basis of how we would then start detecting the entities inside each of these great items for these quadrants. Okay, so we have divided the game board into equal sized quadrants, and the next step is to identify pets, houses, road, that kind of thing. So this is where the animal detector comes into play. How many of you have seen the Ace Ventura videos? Okay, yeah, so I don't wanna, it's a very limited sort of joke space, because if I'm incredibly raunchy that movie is, but. It's a, you know, the comedy value stops here I think, so. Okay, so how do we do this? Well the first attempt, because you know, I'm saying I'm not a computer vision expert, so my first attempt was sort of naive, I computed what's called a histogram of all the colored pixels in the animal or house. I stripped away all the gray pixels, cleaned up the filter, or cleaned up the pixels a little bit, right, and then computed a histogram. A histogram just means, how, what's the frequency of each color in this image? And then I compared that to a histogram of the original game asset, because I work at Lumosity, I have access to all the game assets, right? So I can compute a histogram of that, compare those two together. And the closest match wins. So the accuracy of this approach is about 65%, that's a total sort of guess, you know, but it got things wrong all the time. This by the way, only for levels one through 12, for anything higher than that it really messed up. And that's because, oh sorry, I'll go into that in a second. The second attempt also used histograms for the color pixels in the animal or house, right? Also used the same game asset, and then uses the, I think it's chi-squares test, or chi-squared test to compare the two. And the chi-squared test, this is from Wikipedia, is just a way to compare the differences between two sets of frequencies. This was a little bit better, about 10% better, 75% accuracy, again, just for levels one through 12. And the problem that I keep running into though, was that when you get to the higher levels especially, you get animals that are the same freaking color. So the ferret and the hedgehog are both purple, and they have basically the same amount of dark purple and light purple. Turtle and chameleon, same thing. Husky and pug, the rabbit and the siamese, and the tabby and the cockatiel. So trying to distinguish between these was extremely error prone, and it would always sort of, was to the point where I couldn't make it work reliably enough to be happy with it. So what do we do to fix this? Well, this is the third and the final attempt. So I found out that you can compare images using something called a perceptual hash. And without going into too much detail, perceptual hash just gives you like a fingerprint of an image, and lets you compare that fingerprint to another fingerprint to see how similar two images are. So I compute the perceptual hash of the animal or house in game, again removing all gray pixels, and then do the same thing for the original game asset. Find the hamming distance, that's the number of bits that are different between these two things, and then choose the animal or house with the smallest distance from the game asset. So that gives an incredibly accurate look at the animals, and this works for all the levels. All right, not super important. The idea, this is Mike Purim. He was, I guess he has a library that does this, and it's wrapped by the fashion gem. The idea just is that, as I mentioned, the hamming distance and the calculations thing, not super math heavy, not something that I really understood, but I could just use this cool gem. It's called the fashion gem by Weston Platter. So before the comparison, remove all gray pixels. We trim the edges to make sure that the car specifically doesn't intrude. We have orange pixels from the car that really throws off the calculations. You have to trim the image a little bit. And then the image is just, the comparison is as simple as making two fashion images and then calling distance from them. All right, so here's an example. From the game board we have the pet, but this is actually the house tabby here, and we're comparing it to the house cockatiel. And of course, the hamming distance is huge because they're not even close to the same thing. Well, they are close because they're both pink, but they're not the same image. And so the hamming distance is 33. So if we compare, however, to the actual game asset of the tabby, we get a much better hamming distance of 10. Okay, so the animal detector, how does this thing work? Well, we map over all the quadrants. We trim off a little bit for that annoying pesky car gets in the way. We compute a histogram, we clean up all of the gray pixels in the image. And then unfortunately, because the fashion gem requires you to load a file, I had to save it out to a temp file and then load it back in. So when I say quad fashion here, this just means this is the fashion for the quadrant itself. And then we do the same thing. I have a cache of these things actually. For the game assets is a function that does this house fashion four, pet fashion four. That loads the one from the game assets. We compute the distance and assign that to an animal score object. And then at the bottom here, we reject all the ones that have a hamming distance that's too large. If we do that, if we don't do that, we end up thinking that just because hamming distance can be a little bit variable, right? We end up seeing no pet or no house in a quadrant and it will still say there's a pet or house there. So we have to filter out ones that don't contain any animals or pets or houses or pets. Okay, and then we assign a matrix to these things so we know where they live. So when it comes time to choosing the best animal or pet, we call best match on each element of the score group. And best match is just the minimum of house or pet distance. Okay, so we identified which houses and pets exist in each quadrant. The next is detecting the roads. So roads are also very similar to finding the boundaries because that's, again, that's the technique we use to find the boundaries. So we use the probes to find the places where the track meets the edge of the quadrant. So in this case, there's a bottom right, it's a bottom segment and a right segment, that's the first image there. There's a top bottom and right with one of the tally on it. It's a top bottom left, right and left for the doxment and there's a top bottom right for the house husky. So we search in the middle of each edge and I can show sort of the, this is the search space because sometimes the edge has, again, another house in it, could have another pet in it. You just don't know what's gonna be there. You search along this entire sort of tolerance area, trying to find the track and we have to consider, yeah, I said the interior pickles is a small distance away from the edges. So, and then the probes sort of go in from the left, right, top and bottom and in the case of the house, though, we have a problem, right? So in cases where the house hides the track from view, which happens all the time, we have to use adjacent quadrants to know if there's supposed to be a track there. So in the case of the husky, we just saw this. The house hides the track. So the Dockson's Quadrant, though, the one immediately above the house husky's track has a bottom segment and we know that because there is no house obscuring our view, right? We use that tolerance to make sure that we can find that and because of that, we know that the house husky needs to also must have a top segment. So because Dockson has a bottom segment top where the house husky must have a top segment. So we can sort of guess. That doesn't always work, but it's pretty accurate. Okay, so how do we do this? Well, detect left just runs a probe and this is the same code we saw before with probes and then we just make sure that X for left is within the tolerance that we want and make sure that if there is any black pixels, for example, on the complete other side of that quadrant, we don't count those. We do count ones that are within that tolerance percentage. And then when houses get in the way, we have resolve left and by the way, there's resolve top, right, bottom, left. Detect left, bottom, right, top, right. So this is just an example of how these work. So we say if there's a left neighbor, turn left on, if there's a right, you know, and so on and so forth. Okay, so we've detected the roads. The next thing is to, so now we have, basically we've identified where everything is in the image. We know exactly where everything is. We know all the track, all the houses. So now it's just a matter of figuring out what the correct order of pets and houses is. And to start that process, we have to build what's called a graph containing the positions of all the pets and houses so that we know how far apart they are. So if you're not familiar with a graph, I'll go into that a little bit too. A graph is just an interconnected set of what are called vertices. So a vertex, and we're seeing the image at the bottom here shows a bunch of vertices connected by a bunch of edges. And an edge is just a connection between two vertices. These are just fancy math terms for, you know, very simple concepts. So for each pet and house, this is the algorithm. For each pet and house, an empty quadrant. So that means any empty track as well. Find or create a vertex, one of these little balls, these little circles, for the pet house or quadrant. And then for each neighbor of that pet house or empty quadrant, find the top left, right, bottom neighbors and add an edge between the current pet and house and that neighbor. And what you end up with is a graph of all of these paths between these objects. Let's look at an example of this using the same screenshot that we have used so far. So we're gonna start at the top, we're gonna start with the husky. So we make a vertex for the husky and we add that to our graph. The husky has the pet cockatiel, the pet siamese and an empty space next to him. So we'll add all those edges. So we add, remember we add a vertex for each neighbor that we haven't seen yet. So we add the siamese, we add the cockatiel and we add an empty space. And then we add edges from the husky to all those things. So now we have a sort of a semi-built up graph. Next thing we'll do is look at the cockatiel. The cockatiel has only one neighbor we haven't seen yet, that's the doxon. So we connect them together. And also it's also connected, the doxon is connected to the siamese, which I should have been in this slide, my bad. So then we add the turtle's house which is also a neighbor of the doxon. And then we just keep going. We add the siamese's neighbors. We add the tabby's neighbors. We add the turtle's neighbors, the turtle house's neighbors. And I'm not showing every single empty space here by the way, that just gets a little too complicated. So some of these won't show where that's happening. But we keep going, we add the husky's house. This is an empty space that happened to have a neighbor of the hedgehogs. We added that. We have the neighbors for the car, the house cockatiel, the house hedgehog itself, house siamese, three empty spaces, the house tabby, and by this point we're actually done, but we keep going and check the doxon's house. And by this point we have a completed graph of all the connections between all the pets and houses. Okay, so now that we've got this, we can do some pretty cool things. And this by the way, this is the code for doing this. So we add vertexes, we loop over, but simple graph by the way is a little library that I built a long time ago that just is a very simple graph system that lets you add vertexes and edges. So make a new one of these. The each here is eaching over all of the members, all of the items in the matrix that we have of the whole game board. So we add left neighbors, top, right, bottom neighbors. Here's an example by adding left neighbors. We're just saying add a vertex and then add the edge. And by the way, if the vertex already exists, just use the one we already made. So this is the, I would say, sort of dense, but just a few lines that does this. So now that we've built a graph, we can find the distances between all combinations of pets and houses. And this is sort of maybe the most, I don't know, this is an algorithm for this that it's just the Dijkstra's algorithm which finds the shortest distance between two items in a graph. And so that's all this is really doing. There's our shortest path method that simple graph contains. We just call that a bunch of times. And then we store a mapping between each entity of the distance. So the matrix we're building up is from entity to entity and then how far apart those two entities were. Okay, and another thing I mentioned at the beginning that we have to revisit here is because the game AutoMap that uses the shortest path, valid solutions can be determined by just generating and checking permutations of pets and houses. So that's what we're gonna be doing next. So this is gonna be intelligently generating these and then finding the solution. Okay, so I'll be on the board. So the first question I had to ask is how many permutations do we need to check in the worst case? So if the very last permutation that we generate actually contains the correct answer, how many do we have to go through to get there? And the answer to this is in the worst case, n factorial, right? So where n is the number of pets and houses, total number of entities on the board. And factorial gets big really, really fast. So four factorial, so there are four pets and houses. This, by the way, is the minimum number of pets and houses in level one, right? That's 24 total permutations. Five is 120 and you just get much, much larger until finally, and 22, by the way, is the largest number of pets and houses you can have which equals just an insane number of permutations, right? This many permutations, one sextillion permutations. And even if we compare or checked 10,000 of those per second, it would take us 3.5 billion years to find the answer, potentially, right? So that's completely untenable, we can't do that. And this, by the way, is a representation of the traveling salesman problem. And I don't want to go into this too hard because traveling salesman is very complicated. It's in a category of problems called NP-complete. And NP-complete problems, by definition, at this point in time at least, don't have fast solutions. So NP-completeness is defined as non-deterministic polynomial time. It's a category of problems where the answers can be checked quickly but generating those answers takes a long time. And it's generally characterized by problems where the computation time increases really very rapidly and we saw that happening with n factorial, right? Really just means there's no current fast solution. So what is a guide to do? I was very happy to be able to make this joke though. I also figure out, including an Aaron joke in this talk is important because I included cats before, right? So he's asking, do we even need the traveling salesman? I was very happy to be able to make this joke in response. All right, so we need a smarter approach, obviously. So yeah, is that it? No, no, remember the game's constraints. So only four pets in the car at once. Gas is limited to a certain number of units and units. And what else do we know? Well, we know that the first move must be picking up a pet. You can't visit a house first because you have nothing to drop off there. And the last move must be visiting a house because ostensibly at that point all pets have been picked up or dropped off except for one. And also the pets in your car determine which house you can visit. So you're limited right off the bat in a bunch of different ways. And as the game progresses, each next step in the sequence is also limited in a number of ways. So because of that, the problem is actually solvable in a deterministic amount of time. So the first thing we do is construct a list of candidate pets and houses for the current step that we're at. If the car, for example, if the car is empty, only pets are viable candidates to pick up. So we would only list those as viable candidates. If the car is full, only houses are viable candidates, you can't pick up another pet, right? Otherwise, all the pets are eligible as are all the houses of any pets that are currently in your car. So using this, you can build up a list of potential next steps to make. So for each candidate pet and house that you've discovered, using the algorithm above, we have enough gas, we visit the pet or house by calling another level of recursion. Passing in that level, that next set of candidates. Otherwise, the current solution is wrong. We ran out of gas, so this must not be the correct solution. So we'll go back one level of recursion and try a different candidate from what we had before. It's a little abstract, but this is the code that gets the list of candidates. So we pass in the number, or the list of cars, pets in the car, and a list of remaining entities. These are things that still haven't been picked up and houses that haven't had a pet dropped off at them. So no pets in the car means you only visit pets, et cetera, et cetera. So this is the code version of what we just talked about. And then this is, I divided this into multiple slides, so bear with me here. This is the solve function. This is what does the recursion. So we check the gas to make sure, whereas I should say first, we make sure that if we don't have any pets in our car and there are no more remaining entities, that means we've solved the game and so we can return early and return current path. We know this is the correct solution. Otherwise, we get a list of viable candidates using that function we just talked about before. We check to make sure we can visit that candidate by computing the distance or looking up in that hash we made before, making sure the distance between those two is not more than the amount of gas we currently have left. If the candidate that we loop over all the candidates, if we've determined that we have enough gas, if that candidate is a house, we drop the pet off there and we try to then select the next candidate. So we say we've dropped a pet off so we can recurse, call the solve method again. This time without the pet we just dropped off and without that pet in the list of remaining entities. We've computed the gas, we passed the gas back in, this is our current number of gas units and then we add the candidate to the path that we're accumulating that tells us the answer. So that's if the house, candidate's house, if the candidate's a pet we pick that pet up, this time we're adding that pet to the car, subtracting from remaining entities and adding it to the path just like as above. So that's the whole thing. Eventually you'll get an answer from that. When I say, I made a mistake before, this is not deterministic but it is, it does work. Like we don't know if there is a solution mainly because there could have been errors along the way. We don't know if the game board we looked at is completely correct. I can't say for certain given a random game board that I'll be able to solve it in a certain number of seconds but I am pretty sure there will be a solution. So yeah, that's the whole thing. That's how I did it. So question is, did this help me progress past level 12? That's the whole, the whole goal of this was to do that. So yeah, I did, I finally got past level 12. And no, I didn't cheat, I didn't use the algorithm to solve the levels for me. I took screenshots before I played the round and then I ran pet detector afterwards and tried to learn what I did wrong. And here's some strategic takeaways if you guys ever play this game. So it's best to sort of go in a spiral motion. So pets are generally sort of picked up in like a sort of ever tightening spiral motion. That's not always true, but it turns out to be true a good percentage of the time. I also did find that it is often a good idea to pick up a pet that's very close to you. But again, not always, right? So your mileage may vary on that one. And then also the biggest thing I think I learned was that it's best to look for opportunities to pick up a bunch of pets all at once and then transport them to the other side of town. And that's because the game thinks it's funny to like give you pets and houses that are wildly disparate in disparate locations and then hope that you can figure out how to deliver them all. So over the end of the game, you're stuck at the very bottom of the game map with one pet left in his house all the way at the top of the board. So it can be, it's a good idea to think about that ahead of time so you know which ones to pick up and then head for one side or the other. So thank you very much for coming. I'm Cameron. Yeah, check it out. How long does it take to run for the higher levels? Okay, that's a good question. For level 12, it runs anywhere from one to four seconds. For level 13 and beyond, I think the last one I tried was level 15. There was one that took a good 20 seconds to run. Yeah. And I don't even know, because I haven't gotten to 20 yet. I don't know how long this would take. The question was, what do the people who designed these puzzles think of this? This was actually, I presented this for a hackathon at the company and there were smirks, there were giggles, there were like, oh my gosh, I can't believe you did this. And there were also people on the QA team who were like, oh my God, can we put this into production? All right, thanks everybody.