 So in the last video we identified that one of the big things that we're looking to do is find these terminal states that sort of end our game in an adversarial search and specifically what we can do is assign these states a Value this first one. We're looking at. Oh, well if my player my agent was playing as the X in tic-tac-toe You know, it could see that this is a terminal state and it's a bad one Again our node Instead of it being max, that's a terrible color for it right there Instead of thinking about it as max X loses X draws X Wins So again, we can assign some type of value to these individual states This is a bad one. You can see the next one. It's not as bad You know, we I used to call this a cat When I was a child It's an okay move, you know, it's not the best one But it's better than bad and then we can see oh well if I'm at a state where X is Winning that's a good move and specifically you can see we can assign some type of value to those given states Bad moves could be a negative value Draws could be some zero and then positive or wins and goods can be positive Now one thing to kind of just throw out there. They don't have to be That it's not set in stone. There may not be a draw or you know draws are just a little hard to get You know, this could still be a three. This could be a five and this could be a 10 Right those values can change and it depends on your game or just how you're sort of building out your evaluations of Or your utility functions for your minimax search. I I just gave you a little hint there Essentially what we're doing is something called a minimax search which in itself is really just a depth first search I'm thinking about sort of my move and Then what state happens from that and then I want to Plan out sort of there. We are I want to plan out their move because I'm looking at this in the sense again This is an adversary. They have my worst intentions in mind. They want me to lose So this is me planning out their potential moves from Different outcomes, right? their Move or men's move When I think about what they may be doing they probably want to pick a move that's much worse than One that's gonna make me win. And so the idea here is that when we get to sort of leaf nodes, you know Hey, they can be happening anywhere in the search. That's one of the reasons we're kind of making depth first Approaches, but what we need to do is we need to get those values at those terminal states back up To the root node because that's gonna allow me to decide whether or not I take this move or some other move That might also be a you know, that might be the better move So how do we do this? Well the way I like to think about it is to start building out that recursive function And it's gonna be hideous, but you can think about this first condition as sort of our base case Right. I'm at some terminal node I have reached sort of a state where let me just draw one of these out So I have reached a state where I've won max is one Well, again, what's the utility at this given state? And I use the example that oh in this state. It's a plus one Well, awesome because what I would be doing in a recursive sort of cases I would return That value But as you can see one of the things that we also have is sort of our recursive cases We need to alternate between act earth, you know thinking about what men would do what what we what our move Would be and what their move Would be back and forth and that's sort of the thing you might notice about this notion of highest and lowest value Specifically again what we're looking at to kind of throw a little bit more of the pseudocode into this is We're essentially doing some big old function mini max on some state S and you can see again We're just dealing with if that base case. We're at a terminal state awesome return the value but notice again, we're dealing with sort of these ideas of when this is at a Max node or a min node. We're not at a terminal state Notice again, I'm thinking about it in that sense that oh, I need to make a recursive call I need to look at what the actions are What all the possible actions are for a given state? What's the result going to be and based on those again this notion of I want to select the largest of them Well, sorry, I want to select the largest of them. That's again only for our Move when we're at men we're at a men node That's a their move and they want to think about the smallest possible outcomes And that's just what the next slides are gonna say is again You know when the state is controlled by max max wants to pick the maximum value when the state is controlled by men They want to pick the minimum value But more to my point is to take this and you know I hate doing these types of those, you know one-liners of code just because it's still a little too confusing Let's start to expand it out a little bit and I immediately show you a one line of code but work with me here again This is sort of just sort of the the big, you know mini max search. It starts the entire process of again starting with max and Then specifically you notice that we start jumping back and forth between men in max values because I'm thinking again as max of The outcomes of two possible states in this case that were occurring occurring from men's Decision and so again, I'm jumping down to the men value function Same thing if that was a terminal state, right? It's not really now men's turn. It's that the game is over So return out the value The utility at that given state but one thing you might might notice is that there is some positive infinity going on here because as I start to look at these potential moves, right a Positive infinity is I want to find something smaller than that I want to find a state that is smaller than that and if I found something that was a one. Well, guess what? V is now negative one because I found a number that was lower technically speaking Good moves or moves that end up winning max the game at least are part of that consideration again It's because as I iterate through all the possible outcomes, right? Negative or sorry positive infinity. Well, that's still quite large That's it max a min is thinking that max is gonna win so I'm trying to pick smaller values to ensure that that may not happen and so In this case, well one is smaller One is smaller than positive infinity So that's what happens then when we get into that situation of something like one versus negative one Well, we see that one is bigger and so that negative one would prevail but notice again, we're still doing that same iteration going through all the possible actions and Then we're doing a recursive call. We're jumping now To a max note. We're going to take this notion of well What are the actions at my move men's move and then well? I got to now make the plans for max's move Just to draw them out as haphazardly as I can and then the same song and dance is going on max does the same thing It's going with men's instead of a positive It works off of a negative infinity instead of it going for the smallest number It's going for the larger numbers and instead of calling for max value. It jumps to min value because that same Thing is going on as we keep on traversing deeper and deeper down our tree Assuming this is men's turn And so just to kind of visualize that a little prettier you can see again We're looking at it from max's perspective of a negative infinity and better moves Are going to be larger than negative infinity. So we want to take those and versus on the men's side We're working off of a positive infinity and we want to pick moves that are making that number that that Possible max will always win move Smaller along the way and we're just alternating between the two of these until we reach these terminal states So the way you can think about this is in essence what the mini max is doing is thinking about Let me just pick out the worst possible case scenarios for max So I like to sort of present this Initial example to the students and we're not at alpha beta pruning quite yet We're going to have to do an exhaustive search That's sort of a hint a little bit later But you can see one of the things that I have done is I sort of added these little triangles here to indicate whose turn It is at a max turn You know, I want the larger of the two so the numbers pointing upward But again, I think about that as some action occurred that max did that leads to men's turn and notice that the triangles are going pointing downward now to kind of Give that mnemonic. Oh, hey, I want to pick the smaller of the two. Well men has some action and then it's max's turn and Then max makes their turn now in our case we you know if we following the pseudo code Oh, we're at a terminal state. So that terminal state is going to get passed up to because two is better Than negative infinity But again, we're just traversing through all the moves and that's why I put it sort of at this little corner here Because I go down and I see another terminal state that terminal state return returns its utility and we see that that too Is much smaller than a nine And so that nine happens Then that same thing we're again doing recursion since I've traversed all possible moves that max could have made Now I'm returning the value that this would sort of result in and we're dealing with men Now again men is looking at it from this positive infinity perspective and guess what? Nine is better than positive infinity. So it's considering it But that's not all we have to then traverse Downward and you can see what we're gonna end up with Especially in this case again, we're at a max node. So we pick the seven I'm gonna draw it larger because you can already see that as we move through this that four is not better than seven We take that and we move it upward. Oh, well men sees that seven seven's much lower Than nine and so that nine is no longer a possible move men would pick the seven between the two of them And as you can imagine it gets passed up there as well I'll leave it there for now But one of the things that sort of I cheat with with our exhaustive mini max Traditional searches again since I know that these are gonna be max nodes max will always pick the the larger of its two So nine and then five men is gonna look at the two of these Pick the smaller of the two and then again, we're at a max node. So Here. Oh, well max between seven and five just to get rid of that Max would clearly pick the move that would lead to this seven value and so We can see that this is the if everyone played optimally This would be the pathway that we would end up taking and this is just some Confirmation of what we see going on here. So again max's turns are looking for the bigger numbers This is me just noting that you may end up in situations like this Specifically, this is if you're working off of some type of iterative deepening search Because there are too many possible moves if you're thinking beyond tic-tac-toe into things like chests or checkers or I'm trying to think of another one Connect for Right, you may want to stop at some point and do evaluations at non-terminal nodes. That's you know, your own little fruition but again as we work through this we can see again the nine and the seven are being picked by max and those Potentials same thing seven and five are being picked by men in their sort of respective spots and then max would pick the seven resulting in This pathway along that way So one of the things I'll say is go ahead pause the video for a second and you know Try and work through this first thing Now that you're back if you hopefully pause again if I'm working through this one of the things that I'm typically doing with my students is For this version we'll talk about limitations in a second and get into alpha beta pruning in the next video again, I like to tell my students start by Establishing whose turn it is along the way Again for at least my sake I always want to make the assumption that max is going first again The notion is this is an hour move versus their move We only want to make decisions on our turn. So our turn is always focusing on maximizing our effects Even if you're trying to get the smaller of the value, you know, that's where you got to do funky math and instead This becomes one-fifth versus one-sixth But I'm digressing so let me So again, this is max's turn. This is men's turn. This is max's turn. All right Well, again, not a terrible problem. We're just looking at the larger of the two of these So this would become six the larger the two of these would become seven and since we're dealing with I'll just keep on doing the max's for right now Largest of this one would be eight largest of this one would be seven now We move up to the men's level so the smaller of those two would result in a six the smaller of these two Would result in the seven and then when we get to max we're again looking at the larger of those two seven so This is where I'll leave you is obviously I've said it a few times and in the next video again We'll take an exploration on how we can resolve this But one of the problems is it is Exhaustive and because it is all is exhaustive it can get Exponentially harder as the game progresses when we're dealing with something like tic-tac-toe, right? There's only possibly nine moves that could be had here So it's actually relatively small, but when we're dealing with something like a full-on Chestboard and I'm not going to attempt to draw a full-on chessboard But just to give you sort of the the notion here, right? look at all the possible grids that can be going on here and Do I move this piece or this piece or this piece or this piece or this piece or this piece or this piece or this piece or this piece or this piece or this piece or this piece or this piece or Then you get my point again Here's where you know, it's super hard and sure it's still going to find you the solution, right? But the big issue is it's going to take potentially ten years or a hundred years or a millennia To find out your win actually not even your win whether or not you do literally That move first So again, this is a limitation to at least the initial mini max approach And so how do you sort of evaluate this or one approach is that you can do iterative deepening searches? So I only allow for a depth limit of Planning ahead, I don't know for moves for you know, that's not great or bad either It's just I'm only going to plan for moves ahead then instead of thinking about terminal states I have to make evaluations on non-terminal states. How good does this state? You know present me in how good is this and you know, you have to kind of do some funky evaluations for that but Have fun with that and now finally what we'll talk about in the next video. Oh, what if we use some magical? trick called pruning to eliminate parts of the tree because that would help us sort of reduce this Exponential exhaustive search so in the next video, we'll talk about alpha beta mini max pruning