 So, we've already seen a few different heuristics. We talked about this idea of a simulated annealing sort of approach. And with simulated annealing, what we were using sort of as our inspiration was metallurgy. This entire idea, let me get that second L in there. The entire idea is again, as metal is heating up, the molecules are moving all about, and then as we start to reduce the temperature by plunging it into water, those molecules that are bouncing around, almost like configurations, are solidifying themselves. And well, that's allowing us to now have sort of a hopefully best case local maxima, but or hopefully the global. The other approach that we talked about was this idea of a genetic algorithm. And the entire idea here was this was inspired by evolution, you know, survival of the fittest. Figurations are based on sort of the implementations of our forefathers and mothers and survival of the fittest was kicking in because at some point, hopefully we reach a pinnacle. And so the last one that we're going to look at is something known as Aunt Colony Optimization. Again, what we're looking at is this idea of biologically inspired algorithms, algorithms, items, items, because, you know, we see that this is how something in nature or something in science operates. And we're just saying, well, can we take that same similar behavior and use it as a problem solving approach? Because again, something has used this to solve problems. So can we do it as well? And that's where, again, we look at this idea of ants, the entire purpose of an ant, I'm working off of a grid structure just because it makes my life easier, is that it's going to be traversing its environment in search for food, right? Oh, I found food. So what I need to do now is I need to go back to my colony with this food. Ta-da. But that's not just, obviously, they're not just working off of a straight line. There might be nooks and crannies in a maze. And secondly, it's not just I bring the food back. It's also I tell all of my ant buddies, and that's where we sort of look at this idea of it being a multi-agent search. The entire idea is that those ants are going all about the environment to hopefully find food sources. And as they do, they make sure to tell their sort of ant buddies again. So to at least start with that. Then when we're looking at it from sort of some kind of larger environment's sake, you know, when we're dealing with one possible route, perfectly fine. You know, again, the ant sees, oh, this is the only way, I go to it. But more importantly, what we're looking at is this idea of, which do I take when I have a fork in the road? Should I go left? Should I go right? Should I go left? Should I go right? The entire idea here is we actually start to introduce a little bit of the math equation and I'll walk you through it as well. What we're looking at here with this fancy symbol is this is something known as ADA. And for the purpose of ADA, what we are looking at is, again, this idea of attractiveness to a particular move. Now the XY is just sort of in our case referring to what position or what tile that we want to go to. If this was P1, and this was P2, then what I'd be looking for was the ADA of P1, and the ADA, ADA of P2. And so, okay, we've just established, here's a number, here's what we're looking at. What does attractiveness sort of boil down to? Well, that's where, again, we're sort of looking at it from the how much further am I working off of? So if I'm kind of drawing out my Manhattan distance lines, right, again, ADA is just saying how much further? How much further am I to my goal? Now we'll see sort of that's not the only thing that we work off of in this heuristic, but it at least helps build sort of our stepping stones. Because the second thing that we need to worry about is something known as the trail level. So we just talked about, again, ADA. So again, this idea, ADA, this idea of how far we have to go. And then we also have new kind of design, Tau. And the entire idea with Tau, let's see, the best way to describe Tau or how many ants have taken this path. Because again, if we're looking at ants and behaving like ants for a moment, we'll see sort of pheromones a little later, but they're pooping out pheromones or they're depositing pheromones onto the ground. And so an ant, looking at the ground, it's seeing that the pheromones are there and it's like, oh, well, I know that, you know, my colleagues or prior generation of ants have gone this route. I'm going to go this route. And so that's just sort of what these two things are coming into. So again, Tau is referring to number of previous ants and ADA is referring to how much further. Now to just kind of walk through this a little bit more and just to kind of get a little bit better understanding of ant colony optimization. Again, as we're starting to look at this, we'll see that the ant sort of picks one of those routes and is again still kind of making its way to the food source, the ant in this case found the pathway. So this is a valid path to our goal. But this is also technically a valid path. You know, again, we see that this route gets me to my food. So it's also valid. So very similar to what we saw with a star. There's multiple ways I could get from RGU to Seattle. And so one other little thing that we could work off of is the fact that you may have some limitation of how many steps your ant could do. And, you know, again, in this situation, maybe going this far and the ant still hasn't found food, we just go ahead and say, you know, that was an invalid path or that path doesn't work out that well. The ant died that way. So no other sort of pheromones are being dropped. And again, we just never take that path. Either way, what happens is as we, as our ant finds the food, it takes the food, it does a fancy little animation. Look at that. That's the quality that you get in this video. But my point being is as it was doing that, it was depositing pheromones along the way. I found a good source. So I'm going to let you know as I go back home, this was a good source. And for our sake, I'm just going to go ahead and say, you know, we're going to make our trail level something super easy. Just a simple one. I dropped one pheromone on the ground. That just makes, you know, life a little easier for us. But again, we're talking about more than one ant. I'm only working off of one ant. But what if all the ants are dropping one pheromone as well? Well, in that case, we may see that, you know, on this first tile here, right? This one, every ant had to traverse. So it's obviously going to get, in our case, all five ants pheromones. But again, let's say that three went north and two went south. Oh, well, you can see pheromones are being deposited in a different amount here. One thing to take note of with this is you could have these pheromone levels be anything you want them to be. There are certain sort of recommendations that you can work off of. But they just are whatever we want. Again, I could have 20 ants, and they're all depositing one fifth of a pheromone. And that's just giving me another number. That's the important thing. This is just a number. And so is just this. So again, for our sake, just going to work off of one because it makes everyone's lives easier. So again, what we're looking at is this fact that, yes, I do want to have, you know, the pheromone levels. But specifically, I want to be able to identify something about exploration, right? Maybe this food source dies off. For whatever reason, that food source has been expended because, again, it's a finite resource. But maybe there is another food source right here, food. And so again, we would want to, even though this took a longer route for, say, this path to get to this food, it still may be a valid one later on for whatever reason. So we still want to have the ability that even though this is our shortest path, right, maybe it's not over here, but maybe if you were to take, let's see, if you were to take this route right here, where my head is apparently in the way, right, that would be where the food source, the next food source is. So how do we take all of this? Again, how do I take my towel? How do I take my ADA, right, X of Y? How do I do all this? You know, how? And that's where I will warn you. It's a lot of numbers and it's a lot of formulas, but we're gonna work our way through that. Here is the formula. In essence, what we're looking at is we're going to select a probability of moving to a specific direction. So this path one is gonna have some probability. So probability of path one, the probability of taking path two. That's all this symbol right here is looking at, is given this ant, so this ant, this path, right? That's all that's kind of working off of and there's the better picture of that. But you see that we've already sort of built these points out, I've introduced them. So again, we're looking at tau. Tau is saying how many ants came before us and ADA. ADA is also just saying how close are we to the kind of end goal. So again, we're just sort of taking and working off of these two things. So again, the next little piece here you can sort of see going on is some sigma. The sigma is again, just a some summation. What we mean by that is what we're dealing with is given a particular route. So let's say P one again, this is all for P one. Then that's divided, so P one is divided by sort of the routes of P one and P two. So okay, we're just starting to flesh this out a little bit more and you can see I have sort of an explanation here. How do I bring these value? How do I make these numbers? Tell me how to make these numbers. Super simple and we're gonna work off of here is just the inverse of the distance. So the distance is in this case, oh well before we jump to that, just to a little see these because I didn't show them before, alpha and beta, these are mostly just letting us have some kind of flexibility. If they are both equal to one, we're just saying that the trail level and the pheromone or the tract of the trail level, the pheromones and the attractiveness or how much further they're equal. But I could make one go higher or lower than the other and so I could add a little more weight to them. That's all those are saying. For our sake, we're just gonna act like they don't exist. Why did you even bring them up if you weren't going to look at them, right? But that just sort of helps kind of convey what we're doing here. So again, if I'm looking at this right now, all I see is that I'm working off of a trail level of a one, right? I'm just arbitrarily saying that that's all that these are in our case. Okay, right? We've taken sort of that first step, but I still haven't worked off of sort of the eta, right? What's the distance? How much further? How do I translate that into a number? Again, we're just looking at this from the calculation of the inverse of the distance because at the end of the day, if you think about this, it's no different than just working off of a grid, right? A graph structure. I don't have to have all those tiles. This is now just some edge moving to another node, right? And so what we're gonna do is if this had a weight, if this had a weight of 10, we're gonna say that it has the inverse, so one-tenth. Now, same kind of concept's going on there. I'm going to work off of that second path and say that it had, that's my P2, that's my P1, P1, and what's the probability of taking P1? Again, what we're doing here is why we take the inverse is specifically we wanna promote the lowest cost. Again, thinking about this like it's a shortest path problem. I want sort of, if we look at sort of 10 and 14 by themselves, right? I want the bigger of the number, but the problem is the bigger of the number's the 14. By taking the inverse of these problems, suddenly my one-tenth is the bigger of the number and that's sort of how we are able to get shortest path, ta-da. Now, again, as I plug this in, luckily this is where sort of at the start, this is super easy, so one times one-tenth is just one-tenth, one times one-tenth, just one-tenth, one times one-fourteenth, just one-fourteenth. I take this, I plug this into my calculator or my program or I do this math by hand and I'm gonna get one-tenth, again, one-tenth over one-tenth plus one-fourteenth or in this case, 0.17143. I cut it off for the sake of visibility, but you can notice what that's going to give me is 0.58 or what we're saying here is that the probability, probability of going to path one is 58%, 58%. That's it, and we can do the exact same thing on the opposite end, so rather than it being the one-tenth on the top, if I'm looking at this now from what's the probability that this ant is gonna go to P2, well, again, one times anything is just gonna be whatever that thing is, so this just becomes one-fourteenth over one-tenth plus one-fourteenth, and that's what we're gonna see. Again, we take that one-fourteenth and that one-tenth plus one-fourteenth, and that's gonna spit out 0.41 or, roughly speaking, what we're saying is 42%. And you may be thinking, oh, of course, I could have just done the first one minus 58, oh, well, it's one minus 0.58, and we're done, but what happens if we were dealing with three possible pathways or even four possible pathways? This bottom calculation is still this path over the sum of all the paths there. This one is that path over the sum of all possible paths. This one is the sum of all possible paths. And so again, this is just allowing me to see what kind of frame or what's the proportion of this being the good path in relation to all the others. And so we look at this, and the same thing that we saw with our simulated annealing and our genetic algorithms, we roll some random number. We are controlling random. And if that random number, if we said something like 0.25, if that's less than our 0.58, congratulations, go the top way. If, however, it's above that, that's no longer the case, that's too big, well, that was a false statement, go to the next one or consider which pathway you end up taking. Okay, so there you go. But we have one other issue because as I take these, these numbers are just gonna get bigger, and these numbers are just gonna get bigger, and these numbers are just gonna get bigger. So the last thing that we have to deal with is also something about those pheromones. And specifically, we have something known as evaporation, pheromone evaporation. As sort of pheromones are just sort of chemicals laced on the ground, well, air's slowly getting rid of them. And as a result, we need to account for that. Because again, bad paths start getting taken less, and we want to start to erase those pathways from our place. So how we deal with this is again, we've got a bunch of stuff going on here, but specifically what we're looking at is when we're working off of sort of our first approach, how do we wanna initialize the pheromone level? So again, if you think about it, what's the old tau? Again, tau is just talking about trail level. Well, that means if no ant has touched this place, it's a zero, and zero times anything is just zero. And we'll get to this, don't worry, we'll get to what this is in a bit. And so we take that and we multiply it by some initial value. Or you just set it as one, either one works. But then we get into updating our search, and now I'll start to break this down because now tau may be, tau old may be one. And so I need to know what this is. Well, as you can sort of see, this value, if this is where, well, you're gonna hate Greek letters, this is row. It looks like a P, row. Good way to think about this is, again, it's just the pheromone evaporation coefficient. It's some value, what percent of, let me, what percent of pheromone evaporates? Again, we're just saying, all right, there's pheromones on the ground, I need to have them slowly go away. How much do we evaporate at each cycle in our search? So again, row, or I'm just gonna call it P because it looks like a P, right? All that is is just telling me in proportion. So if I said that the evaporation rate was 0.5 or 5%, then what's left over is 95% of whatever my old tau was, old. All right, so 95% of the old tau is left up and then we get this, another sigma. Oh, all right, well, again, sigma just means summation. So we just play this out a little bit more and specifically that sigma is looking at delta tau, but what it's actually really asking is how much pheromone was deposited in this round? So how much was just deposited? That's it. And we take all of that amount as you can sort of see here by M referring to number of ants and K referring to this ant or this given ant. So given all of our ants, how much pheromone was being dropped by them? So how do I calculate this? Well, guess what, it's more math. What we're looking at is this is where we officially say how much pheromone one ant drops. That's all Q is, Q is that constant value of assigning, again, how much we're working off of. And then we do kind of work off of a cost of path as well. So how much pheromone was it dropping along the pathway? But again, that's just sort of our kind of, that's how we do it, that's in a nutshell kind of what we're working off of. So again, just playing that out a little bit more. Again, Q, some constant working off of how much pheromone the ant does. Length of the LK is just what's the length of the path taken by this ant at this time. So again, I've gotten to the goal. What was the length it took? So if we're looking at this, I have two possible routes. I have a P one and I have a P two. What are the new pheromone values after my search? Now this is where I have to sort of fake some randomness and say, let's arbitrarily say one ant went up to the top and one ant went to the bottom. Just for our sake, you know, to simplify things. Rho or P, again, the evaporation we're gonna say is 5% and one ant drops one pheromone on the ground. Okay, well, again, we're working off of tau old. So again, this is just tau, that's tau of the top path or I'm calling it P one. Well, let me correct and sort of follow along with what I'm doing here. So this is again, just our tau of old for our path top. So there, that's all. Again, if you see, I just said that that's a one. That's all. So I plugged in that value. Then I took my row or P and said that I'm going to have 5% being evaporated or 95% is being kept from the last kind of iteration. So in this first little chunk here, right, I would end up with a 0.95. Then I'm adding in how much pheromone. So this is the sigma of the delta of the, it was of, what's the symbol again? Yeah, tau, tau, the tau of this path. So again, this is just saying, well, I had one ant, so q of one over the path. So again, you know, in this case, 0.95 plus 0.1 is going to give us a new trail level of 1.05. Okay, well, guess what? You do the exact same thing. So again, tau old, we get the same q, but again, the pathway is much further this time. Again, remember this is LK or distance. So again, oh, well, you know, one path. So 95% is still kind of being carried over. That's like a 0.07, I'm not going to draw it out. But you can see, oh, well, the pheromones on this pathway, it's no longer a 50-50 shot, it's a 1.05 versus a 1.02. Not a major difference, but again, we're only working off of an M of two ants. What happens if I expand it? Now my M is so much more, it's eight ants. And specifically, I'm saying that five ants are taking the top, three ants are taking the bottom. Okay, well, again, this portion right here does not change. It's again, just the evaporation. How much, how much pheromone stays? Again, it's just 95%. So we do that same song and dance again, that works. But notice again, we had more ants. So the sum of this Q, again, over the path, this is more, more ants went the top route versus the three ants less went here. And if you plug in this calculation, you can see it. I would get now a trail level of 1.45 versus a 1.164. Again, these trail levels are just telling me how many ants came here before me. And that's in our way, sort of telling us which path is gonna be sort of the more attractive of those paths. So again, we're not working too terribly much off of this idea of probability, but just so you can sort of see this and walk through it a little bit more, those numbers, those towels that we just calculated out are used in the next iteration. And so I do the same song and dance. Now, probability does have to kick in. So with p1, right, the probability, the probability of p1 becomes 1.45 over 1.10, 1.45 times 1.10 plus 1.161.14. And you spit that out and I'm not gonna do that. By all means, take a second to calculate that out for a moment. And that's, just to give you the warning, that's the type of question that you might see on a quiz or an exam type of thing. But again, there's a probability of going to the top. There's a probability of going to the bottom. You do this same song and dance and you generate sort of which ants go where. For my sake, because it just makes my life easier, I'll just say instead of five, now six ants because it had a higher probability. So, oh, well, six ants went this way and only two ants went the bottom way. Well, again, notice we're just updating the pheromone levels. Instead of this being a one, instead of the old tau being a one, it's whatever that path was. So again, this is my tau of top. This is my tau of bottom. So again, that 1.45, I wanna take 95% of that 1.45 and whatever that turns out to be, I move on. And I do the same thing with my bottom. So, 95% of 1.164 gets carried over plus, again, how many ants took this pathway this time? And as you can start to see, again, as more ants are taking that top route, that pheromone level, sorry, that tau level, that trail level is getting much bigger because, again, we're saying that more ants are taking this route, more pheromones are being deposited on this route. And guess what? We take these values, the 1.97 and the 1.24 and we plug those in. And so we get, again, so our tau of p1 or top and our tau of bottom, just to correct myself here. Top now. So again, same thing. What gets carried over? 95% of that. What gets carried over on the bottom one? 95% of that. How many ants were this way? Well, again, to make my life easier, seven ants went this time, only one ant went this time. And you can see that we continue to go over and over. As we make these new iterations or these new steps in the ant colony search, one pathway is starting to become heavy with pheromones and because of that, this bottom pathway you're gonna see has much less probability each time versus this top path. So just to plug it in, right? Just so we can all see this. So two, seven times one-tenth, 2.597 one-tenth plus 1.258 times one-fourteenth. I'm blatantly going to cheat and pull out my calculator because I'm not doing that math by hand. Let's see, 2.579 times one-tenth. I know that that one's an easy one, but you know, brain. So we're dealing with 2579. So there we are, 0.2579 over. Let me not eat that. Times 1.258 times one-over-fourteen. That's gonna generate a 3475, 3475. And if we calculated that out, so again, there we are, 0.258 times one-over-fourteen. 0.2579 divided by 0.3475, the probability of taking the top route is now 0.7421, 0.7421 or 74.2%. And guess what, you do the same song and dance on the bottom here, so this becomes, rather than it being, it would still be the bottom of 3475, but just to calculate out that bottom one again, let's see that. Let's see, that's 1.2858 times one-over-fourteen. So 0.8, 0.0898, 0.0899, again, it's rounding up, if you're looking at that number. We take that number, divided by that 0.3475, and what we should see is 2585, eight, so that is 0.2589. 86, or roughly speaking, 25.8%. What do you know, we add this up, we do the song and dance, and the probability's coming out to 100% again. And so again, we're just seeing that top path, this top path has higher probability as we work through it, and the bottom path is starting to slowly, slowly kind of become less probable. And as this happens, as you can see, we eventually are starting to settle that this is the better pathway. But again, I only worked off of five ants, so here's my cultural relevance meme of the year. What if I used eight million ants?