 So, I will just do an introduction today to some I will tell you a little bit more about multi act games. So, multi act game is some is simply one that is not a single act game ok that means there is at least one player who acts at least at least twice on some path of some path of the tree ok. Now if the so once a game is a multi act game can you tell me qualitatively what is it that is changing from as compared to a single act game. What is this one we need to model or we at least pay attention to let us say some some new aspect of the game and what is that sorry strategy space yes of course, but that is that that is that anyway you know changes from when you go from static game to dynamic game. That now from single act to multi act what qualitatively something is changing yeah that is the case even in earlier. So once a player is acting more than once a new angle comes in to the theory of games which is the issue of memory ok because you are now with up till now we were only talking of what a player knows at each step whenever it is it is turn to act. Now you can also ask what does he know relative to what he knew earlier and that is the issue of memory right if he was acting just once there is no issue of memory at all right. So so in a multi act game you have to pay attention to memory now so the so games the general rule is that you can say multi act games get extremely complicated if you are if you allow for arbitrary sort of memory structures ok where players can forget things across time instance that they have played or and forget what you know with whether they are at this stage of the game or that stage of the game and so on all the if you allow for all kinds of crazy all of those are well defined games ok well defined multi act game, but they just become extremely hard to analyze you the only way to analyze them is to write out a huge normal form and then you know just go about mechanically finding the equilibrium if you want to hope for some kind of structured way of thinking about these games you need to have a some structure to the memory of the of each player ok. So the way so a typical so multi act game will as I said a broad category is that any you know any player there is at least one player who acts at least at least twice along some path ok. So now a but the kind of multi act games that you can actually study you know in a systematic way or what are called multi act games in feedback form ok. Feedback basically this word comes from control theory and the reason feedback essentially refers to something that you knew in the past which is being passed on to the next stage right that is essentially the idea of feedback. So if you so if the if the game has a feedback form then you can what you can kind of what you what kind of happens is that players do not lose lose what access to information that they had in the past ok. So they have effectively recall and those kind of games are the ones that we can we can study you know more systematically ok. So I will give you a I will give you a a first let us write out a general structure of a multi act game again as I said an arbitrary multi act game is very difficult to analyze we can start and getting before we get to feedback games we can at least start talking of things like stages to a game ok. So if there are no stages also it is where you know anything can happen then it is again extremely unstructured. So first step is to actually have a game where there is a time axis involved which keeps track of stages ok time 1, time 2, time 3 and so on ok. This starts becoming more like a state space type model where you have a way of keeping track of time and you have a way of defining what the state of the game is at that time right ok. So let me let me write out that sort of model definition here stage wise. So the tree is divided into stages and what do you mean by stages each stage is a you can say is a single act game. So each player plays at most once in each stage ok. So stages means that at most or you can even say exactly once you can add a trivial action where he does nothing. So you can says let us say plays once in a stage ok. So when you are in stage 1 all there and there are suppose n players you are in stage 1 n players will act each of them will act 1 and then stage 2 will begin ok that is so there is some you know kind of an independent clock or a referee or whatever keeping track of stages ok that is the sort of structure here. So tree is divided into stages and now players are aware of which stage the game is in ok. So that means what does this mean different stages exactly. So no information set contains nodes from more than one stage right that is essentially if there is an information set that passes goes from one stage contains nodes from more than one stage then it means that that player does not know whether he is in this stage or that stage ok. So no information set contains nodes from more than one stage ok. This is stage wise. So this is what it makes a game stage wise now in addition it will be so it is in feedback form. Now feedback form means that players so I will let me explain this. So feedback form so at each stage begins with so player let us say so what we have assumed so far is no information set contains nodes from more than one stage right. But at the start of a stage is there clarity that the stage has begun let us say a part of the tree that we can say is has started in that stage. So let me explain what I what I mean. Suppose you can you can have a situation like this where suppose these are this part of the tree is stage 1 and now stage 2 is beginning ok some there could be multiple nodes and all that in between here ok. Now the first acting player at stage 2 this part is stage 2. Now the first acting player at stage 2 could have an information set that goes like this ok. So he knows that stage 2 has begun but what has what information has he lost? No so when he is confused between these two nodes this node and this node right what is he confused about? Yeah there could be multiple players in this. So this could have you know I am just drawing this as a tree but there could be you know the there could be this could be a very complex tree here with but each player plays once in each stage. So but the point is the new stage has started here with these and there is at the first acting guy here he is confused between these two red nodes. What would that mean? He knows that stage 2 has begun but what he has the information he is lost or what he does not know let us say forget about lost what he does not know is what happened in the previous stage what was the conclusion in of the previous stage. He does not know if the previous stage concluded with the red node or with this blue node right. So when you divide the game into stages if you want for the game to be in feedback form essentially whatever happened in the first stage there should be clarity that that is from where the next stage is beginning. So essentially there is a well defined sort of end to the first stage from which a new game the next stage is beginning of course players have to strategize across whatever is the number of stages if there are K stages to the game they have to strategize over the K stages but when when playing in the next stage they know that this is what is the history of the previous stages is this clear. So what this means is you cannot allow an information set for the first acting player to have more than one node. The first acting player in each stage starts with a singleton information set. So let me start putting down these requirements. So in the feedback form you need in addition to this the first acting player in the of stage one so the first acting player in each stage now acting player in each stage has a singleton information set. So think of this as like you know like in tennis you know the first set is done you know what the score at the end of the first set is and then the next set is beginning right. The reason we are imposing this is because otherwise the problems are this is the only class of problems that we can systematically analyze. So we are putting some structure on the class of problems so that there can be you know some sort of systematic analysis sorry. Now a player is playing more than once this is a multi these are multi act games the game is divided into stages okay and in each stage a player plays at most once or exactly once okay. So in stage one each player plays once stage two each player plays once. Now the first acting player of stage two should start with a singleton information set okay this is one requirement there are more requirements also. Now this guy is what this means is if this is a singleton information set what this means is that so if these are singleton information sets okay what this means is that the first acting player let us say it is I will call this some guy suppose this is player three who starts in this stage two okay. Now player three now knows that this is where the game started from this blue node is where the games is where the stage two has started. Now the players that follow this should also have this information okay. So everyone must be clear that which sort of sub game in stage two they are in is this clear. So you cannot have a situation where for example something like this for example. So player three for example here player three knows that this is that this is where the game ended he knows that it is ended at blue as a blue node or ended at the red node sorry this is where the stage one ended he knows that it ended at blue node or the red node but the later acting player is confused about that say let us say later acting player is four player four player four does not know whether stage one ended with the blue node or red node you can imagine the kind of complication this would result in essentially what this would happen that what it would mean is that player three knows that the score at the end of stage one but players four does not right he does not know I mean what was the conclusion of stage one okay. So a lot of these things we take for granted when we have a game that happens through multiple rounds and in fact whenever we module things in state space form we actually take these things for granted that there is something that you know that there is a systematic sharing of passing of enough information alright. So the next requirement is that information sets of other players are such that none of them nodes okay nodes corresponding to branches emanating from two or more different information sets of the first acting player of the stage. So what this means is essentially if you now these each of these you look at any of these trees these are sub extensive forms in their own right because all information sets of players are contained within these if the nodes are here the information sets are also here itself no information set will grow from go from one sub branch like this to another sub branch out here right. So that what this therefore means is every time a stage begins it can it can be thought of as an entity in itself you do not have to think of there is no uncertainty about whether you are in the you know this sub tree or you are in this sub tree for any player alright okay but that does not mean that players have perfect information okay. So players could have loss of information for example it could very well be that so this as I said is not allowed so but any of these kind of things are allowed so for example it is quite okay for a player to not know with you know within the stage whether he is at this node or this node. So long as they do not stretch outside of you know so long as these information sets do not go across from this branch into this branch and they do not go across from one stage to another we are perfect that is perfectly okay yeah now we can ask for more okay now we can ask for more it does not have to be ladder nested but you can now so every stage effectively can be sort of is kind of like a single is now basically like a single act game provided you could slap on the payoffs at the leaf nodes right. So now you can ask okay what kind of single act game that is okay it can be nested ladder nested etc etc and then you what that is then you can hope what you can hope to do is basically use feedback essentially that you start doing sort solving for the game using a dynamic programming like argument start from the last stage solve for an equilibrium in the last stage for every subtree that comes in the last stage replace you know put the payoffs at the leaf node then go to the the last but one stage and so on and so forth so if as I said I keep emphasizing this if the information sets go across different legs of the tree you are in trouble because then you will not be able to decompose in this kind of a neat manner right okay so yeah so I will just add to what was just said so if these if the single act games at each stage ladder nested nested then the game is ladder nested or nested feedback form okay so we can just write out what we are looking for as far as the equilibrium is concerned now of course a Nash equilibrium is very would you can just write out Nash equilibrium in the usual sense but now we can look for a Nash equilibrium which will hold in this feedback in this feedback sort of manner right that what would that mean what so what that would mean is that suppose you are you have a game like this of K stages okay so suppose this is stage 1 this is stage K now if you have a game like this of K stages now what you can look for is that well in the case stage every such node each of these nodes is a singleton information set right so you start from that subtree that starts from that in that in that stage look at that subtree corresponding that that is emanating from that singleton information set find its equilibrium okay then go go up keep replace that subtree as I said by the payoff of that tree and go up and and so on now what you would be then doing effectively is you would be solving for you what you would be solving for then is a is an equilibrium in which you are in which every player at every stage is being recursively rational right so which means he is the in every in every stage the equilibrium is being found for every possible past actions that you every possible past strategies that you could have played that the players could have played right so which means that no matter what he was played earlier suppose you get to this point then as a function of this node I will find the equilibrium irrespective of what I had played earlier right it is what you know in in in control we call this a Markov policy right essentially you take an action you where the the action is a function of the state at that time so think so sub so what you find is an equilibrium correspond for this tree for this subtree or this sub extensive form which is now a single act game you and you find it for that one without taking into consideration what was done prior to this right and then assuming that you have solved for this you then propagated backwards what this gives you is a is a is a is a is a profile of strategies that are in equilibrium at every stage in addition to being in equilibrium across in the full game okay so what this ends up doing is basically it means that players are being rational at every stage and not just you know rational in the game as a whole okay we saw this even in single act games the same kind of logic will now hold but now it will be in a multi act setting because now in every stage that same logic will be applied is this clear okay so how would I express this so this is what this is then a more demanding notion so this is what is called a feedback Nash equilibrium okay so I have J I which is cost of player I so I will explain what my notation is just give me a minute so I have written this for stage K okay so this is this is written for stage K I will explain what my notation is here my so gamma remember play player I will now have a strategy in each state for each stage okay so gamma I is will be denoted like this gamma I 1 to gamma I K this is these are the strategies of this is the strategy of strategy of player I in stage 1 and this is the strategy of player I in stage K okay so every stage he has some information sets actions you have a function mapping information sets to actions so that is the strategy for that player in that stage okay now now gamma sub this thing so sub K here or sub K okay is this is the profile of strategies of all n players in stage K this is okay so this is the pro so this is gamma 1 K till gamma n K okay so this is the profile of strategies of all n players in stage K now what am I asking for in a feedback Nash equilibrium what we are asking for in a feedback Nash equilibrium is this so I have a first time asking writing this only for stage K okay so at stage K regardless of so you have this is the payoff of player I when he plays his start strategy in stage K and all others pay their start strategies in stage K and what is this this is the payoff of player I when he plays his start strategies in stage K others plays their start strategies in stage K and at previous stages players have played some some arbitrary strategies gamma okay arbitrary K minus 1 arbitrary profiles are here now this has to be better than this okay so what is there on the right hand side here player I has deviated in stage K to a strategy gamma I K is lift deviated from his start strategy in the in stage K others are still sticking to their start strategy which is minus minus I K and on the right hand side I have this the same K minus 1 profiles for the previous K minus 1 stages the one same ones that are there on the left and we want this to be less than equal to this for not only all players so you want this for all players for all gamma I K for all players I and for all gamma 1 to gamma K minus 1 why is there a for all gamma 1 to gamma K minus 1 because regardless of what was played in the first K minus 1 stages the K stage strategy should be chosen such that they are in equilibrium okay so so which means that your K stage what you are going to do in case in the K stage has to be basically is essentially tuned to the exact information exact subtree that has come up in that stage okay so here remember you have a for all gamma 1 to gamma K minus 1 okay so for every so you have to pick you have to pick a you have to pick a strategy in the K stage such that regardless of which history has brought you to the K stage you are it is your it is your in equilibrium all right yes no no no no so so gamma K gamma I K right in stage K will have one function here another function for this another function for this another function for this etc what it what this is preventing is from you to issue it is preventing you from coming up with a function that will work you know that will ignore this information for example it is preventing you from coming up with a common you know sort of doing a threat across these these sub trees because it has to be in equilibrium regardless of which tree you have got into that does not mean you are taking the same action in each tree of course there is a there is a function here there is a function here the function here all of these collectively defined gamma I star K okay but each portion of it has to be chosen in such a way that it is in equilibrium regardless of which history got you there okay so this is at stage K now let us write it for stage K minus 1 can you tell me what should I write for K minus 1 for K minus 1 I now have profiles up till K minus 2 okay stop till stage K minus 2 but K minus 1 is anticipating the equilibrium from stage K right so here now I would have gamma I star K minus 1 gamma minus I star K minus 1 and then I have gamma star from stage K this is less than equal to 1 to gamma K minus 2 gamma I I do not need a star here gamma I K minus 1 gamma minus I star K minus 1 gamma star K so what is the for all now for all is over gamma I K minus 1 which is his deviation in the Kth stage it has to be true for every player so for all I and it has to be true regardless of what what history got you there right regardless of all of this so so regardless for all gamma 1 till gamma K minus 2 and coming back now working back now to stage 1 so at the at stage 1 what do you expect you stage 1 in stage 1 your play each player is playing taking into account that he is play that in all the subsequent stages he is played in this kind of recursively rational way okay so stage 1 he now has gamma 1 so where do I put the stars on the left since I am writing for stage 1 all should be star right yeah so gamma 1 star gamma 1 star 1 gamma my sorry gamma I star 1 gamma minus I star 1 and then I have gamma 2 star gamma 3 star all the way till gamma K star less than equal to J I gamma I 1 gamma minus I star 1 gamma 2 star gamma K star okay so this is now for all gamma I 1 for all players I and for there is no other for all left right because I do not have anything anything to you are at stage 1 so there is no history up till now okay this is clear so so this is this is a feedback Nash Equilibrium now so what is what kind of let me ask you this what kind of threats is this preventing this is preventing the kind of threat where a player may say well he will couple his he will say that well regardless of whether he is in this subtree or this subtree he is going to play a certain you know certain type of strategy where he is basically ignoring the subtree information inter subtree information cannot be ignored when in an equilibrium like this because you are because you are in an equal you are in equilibrium in every subtree however intra subtree you can still issue threats right because for example suppose this subtree is some kind of say let us say some ladder nested of some form like that in which there could be again multiple equilibria and it could be threat equilibria for that for that subtree that are threats within that subtree not across that do not extend across different sub trees but within that there could be there could be imperfect information and a player could ignore information that is there in that subtree okay so now if you look for a ladder nested or whatever type of equilibrium in each subtree then you would get a ladder nested feedback equilibrium feedback Nash equilibrium okay now so what this means is that if you look at this here right this is just so ignore the first part then this is if this is essentially just a Nash equilibrium at stage k there could be threat equilibria at stage k also if for every node there could be every subtree there could be threat equilibria so if you make sure that you ignore there also players are not allowed to make threats or you look for delayed commitment type equilibria in every subtree then you would get a delayed commitment type feedback Nash equilibrium okay