 Welcome back to this NPTEL course on game theory. So till now we have been seeing non-cooperative game theory. Now we will switch our focus to cooperative aspects. First we start with one of the fundamental concept in non-cooperative game theory which has the cooperative aspect state. So this is known as correlated equilibrium. So first we will start this correlated equilibrium concept with some examples. So we consider the following example which is known as battle of sexes. So the payoff matrix is given in the following way. There are two players, one wants to go to movie and other wants to go to let us say dance program. The payoffs are given this way. If both of them go together then they have a positive utility and if they go to different places there is no utility. So this is a coordination game, in fact we have seen this earlier. So let us look at this game and as we noticed earlier this game has two pure Nash equilibrium. So they are given by mm, both choosing mm is a pure Nash equilibrium and both choosing dd also pure Nash. So these are the pure Nash equilibrium. Apart from this there is also a mixed Nash equilibrium, in the mixed Nash equilibrium so the game actually they play 2 by 3, 1 by 3 the player 1 plays m with 2 by 3 probability d with probability 1 by 3 and player 2 will play m with probability 1 by 3 and d with probability 2 by 3. So this is a mixed Nash equilibrium one can verify easily. So once we know this mixed Nash equilibrium now we would like to consider the other thing. So in mm the Nash payoffs the payoffs that they receive are 2 and 1 and in dd 1, 2 the player 1 receives 1 and player 2 receives 2. In the mixed Nash equilibrium we can see that they will receive 2 by 3 each, both of them each is this is the payoff vector, these are payoff vectors. Now we would like to consider another kind of a strategy where there is a third party, so we would like to call this as a trusted, so with both the people will trust. So the third party who advises the following thing, following. So in this what he does is that he said he will toss a coin this is observed by both players. Now so he will tell the people separately. So what this is if when if coin lands hats then they should choose mm, if tails come they should choose d. The third party advises them that based on this commonly observed random variable which is the tossing a coin if the outcome of that is hats both of them should play m, if outcome is tails they should play d. Of course they do not know what the other play is playing, fine. Now this is basically this strategy depends on a common randomness and in fact it is not like a mixed equilibria mixed strategies that we have seen earlier this is a joint strategy. So now under one if they are following this strategy so we can see that with the half probability they will get 2, with half probability 1 and in the other player will get the half probability with half probability 1 and the half probability 2. So therefore they can see that the players get 3 by 2, 3 by 2. So this is this payoffs are higher than the mixed Nash payoffs 2 by 3, 2 by 3 in the mixed Nash they are getting 2 by 3 each but whereas in under this joint strategy which is known as a correlated strategy they are getting 3 by 2 and 3 by 2 which is higher than this one. In fact if we really see it carefully this the total payoff that they are receiving here is 3 and which is same as what they receive in any of the deterministic Nash equilibrium. In fact one would like to point out here is that once this strategy if the players know that both the players are following his this third party advice they have no incentive to deviate because they know that the other player is going to stick to this one. So therefore this advice they will not deviate and in fact this is actually known as a correlated equilibrium. So we will see one more equilibrium and then one more example and then we will see it. So we consider the following game. So the game is the following thing. So the payoffs are given by 3, 3, 0, 5, 5, 0, minus 4, minus 4 there is a yield drive yield drive this is known as a game of chicken. So in this game what the game here is that there are two people who are driving on their cars and they come to an intersection point in a different of course different direction. Now they need to cross okay they need to take turns or whatever way. Now if both of them drive try to cross they will hit each other and then they will incur a loss. So therefore the minus 4, minus 4 is basically both of them drive then they will lose. Now if one drives and other yields the one who is driving get a utility because he has reduced the time of waiting. So he will get 5 the other person will get 0 because he has waited and so he missed it. But if both of them yield that means no one is losing anything and then they have some incident to here and both of them will get 3 and 3 okay. So now this is the story of this game of chicken and then in fact the game has again 3 Nash equilibrium and they are basically yd one yields and other drives and similarly the other person 1% the first person drives and other person yields this is this thing and then the other game other this thing is 2 by 3 1 by 3 that means with one third probability they are driving this is the mixed Nash equilibrium these are these 2 are pure Nash equilibrium. So here what we are saying is that with one third probability both of them will drive and with the remaining probability they yield. Now let us look at it what are the payoffs here. If in this pure Nash equilibrium yd 0 5 these are the Nash payoff vector and in this case 5 0 and here with two third they are yielding and then with one third they are driving it. So therefore if you calculate it so the payoffs actually are going to be 2 and 2. In fact this is a simple exercise to see that this is a mixed Nash equilibrium and this thing. Now let us how can a correlation be obtained here. So let us look at the following thing. Suppose we have a traffic light which instructs the yield or drive that is assume uniform half probability half probability. So uniformly it is suggesting whether they should yield or drive. Now in fact under this what of course when they say when I say instruct y and d what it means is that the player 1 is going to yield and the player 2 is going to drive and here the player 1 is going to drive and player 2 is going to yield. So basically among these two strategies the traffic light is randomizing it while one person yields the other person it is suggesting to drive. So this choosing this the person to drive it is choosing with half probability. Now if we calculate the payoffs it will be 2.5, 2.5 under this correlated strategy one will get it. In fact once this is done there is no incentive for people to deviate from this. So this becomes a correlated equilibrium. So let us look at another strategy. So now here the random choose between now instead of 2 I am taking 3 yd dy yield. So there are 3 strategies, 3 choices and the randomization is between these 3 and how you do it because when you are saying one of them is driving then these 2 should have a uniform probability. So therefore let us say this yielding we keep it as probability P and this will be 1 minus P by 2 this is 1 minus P by 2 let us look at this one and I would like to choose P the best here what is the best here let us look at this one. Now given that suppose given that a player is instructed to yield there are 2 possibilities one is with probability P and other is with probability 1 minus P by 2. So when one person is told about yielding what about the other player? So the other player the player knows that the other player has been told yield with conditional probability. So we are looking at the probability the conditional probability that the other player also told to yield. This is going to be let me call P y this is nothing but P by P plus 1 minus P by 2. So and he is told to drive with conditional probability let me call it as a PD that will be with 1 minus P by 2 by P plus 1 minus P by 2. Let me write it here 1 minus P by 2 by P plus 1 minus P by 2. So this is basically the probability conditional probability that is advised to drive. Now therefore the player's utility is going to be because with P by probability the other guy is also yielding and then PD is driving it. So therefore the player's utility is going to be 3 P by for yielding and then 5 P by minus 4 PD this is for the driving. So this is going to be this one therefore player will not deviate it the instruction as long as 3 P by is bigger than 5 P by minus 4 PD if this inequality holds true then the player will not deviate from whatever he has been instructed basically that is this one. So now this if we try to simplify it we have used already what is P by and what is PD put substitute these things here and then we can say that this is true if and only if P less than equals to half. So one can calculate this very easily it is not hard substitute the values of P by and PD PD and then you will get P to be smaller than half. Now under this thing we now each player's utility of course the same thing holds with the other player also this is a symmetric game and the symmetric behavior happens. So therefore now the players each player's utility under this correlated strategy is going to be 3 P plus 5 into 1 minus P by 2. So this is the utility that they will get under the correlated strategy if with 1 minus P by 2 probably they are choosing Y by D and with 1 minus P by 2 probably D by they are choosing P probability Y by now calculate the utility of utility that they are getting under the strategies for each player that is going to be 3 P plus 5 into 1 minus P by 2. Now this is nothing but if we calculate this one so this is going to be minus 5 P by 2 that is 3 P so 6 P minus 5 P that is P by 2 plus 5 by 2 then in fact if this utility will be maximized when P is equals to half therefore at P is equals to half the utility is nothing but 2.75. So therefore under this correlated strategy the payoff vector is 2.75, 2.75 so both of them will get 2.75 each in fact as we already proved it as if P is less than percent of no one is going to deviate from the instruction so therefore this is also a correlated equilibrium once you suggest this correlated strategy the players have no incentive to deviate from this correlated strategy. So now here there is an interesting thing let us look at in the mixed equilibrium here they are getting 2 and 2 and in a pure Nash equilibrium Y 0, 5 and 5 0 here so what they are really getting is higher the total payoff which is greater than in any Nash equilibrium. The players will get higher than any Nash equilibrium in this. So this correlated equilibrium has several such interesting features. So the most important thing we would like to point out here is that the correlated equilibrium is a joint strategy the players are not choosing their strategy independently there is some third party who is advising them what to play and that advice is through some common randomness. So now let us formally define the correlated equilibrium. So let us consider game G let us say we write everything for 2 players but we can actually extend everything for multiplayer case. So the player 1 strategy set is S2, S1 and S2 and then their utilities are U1, U2 so because we are considering only 2 players so S1 is the strategy set of 1 and S2 is the strategy set of player 2 and we assume S1, S2 are finite. In other words we are considering a bi-matrix game. So now what is a distribution, what correlated strategy is nothing but a probability distribution U on S1 cross S2. Basically S1 cross S2 is the possible strategies pairs that they can choose and then you are saying advising them both of them should choose same thing. Of course they do not know what the other player is doing it they will be advised. So mu is a probability distribution on S1 cross S2. Now what is a correlated equilibrium? It is said to be correlated equilibrium if the following happens for every i for every Si Ti in Si so the following condition S minus i belongs to S minus i, S minus i is basically suppose if i is equals to 1, S minus i is S2 and S minus i this is nothing but S2. So you are taking mu S minus i Si with probability S minus i and Si people are choosing this with this probability and then the i player will get S minus i Si this is the under this correlated strategy mu the player i gets this much. Suppose if the player i deviates from this so instead of playing Si he plays Ti. So then what he will get is S minus i, S minus i the player 2 is not deviating it so therefore this will be simply S minus i mu S minus i Si this is the probability with which this thing and but instead of ui S minus i Si he will now get Ti this is the utility that player i will get because he is deviating from Si to Ti. So this is the utility that he will get by this. Now this should be if this correlated payoff that he is getting under the correlated strategy mu is bigger than this quantity then I say that this mu is a correlated equilibrium. Now this is something exactly like a Nash equilibrium but in the difference between the Nash equilibrium and correlated strategy is that in the Nash equilibrium people are choosing their strategies independently the same inequality similar looking inequality applies there also but here the thing is that this strategy is a correlated strategy it is a jointly chosen strategy in a way jointly chosen through some randomness. So that randomness is known to both the people but they do not know what exactly the other player is choosing. So such a strategy is called correlated equilibrium and in fact in the previous examples whatever we have seen is that those things we can show them that they are the correlated equilibrium with respect to this notion. Now let us look at few interesting points here. So what exactly is this definition giving us? So the thing is suppose if mu is one correlated equilibrium let us say mu is another correlated equilibrium then if you this inequalities with these two inequalities with mu and mu will hold now it is not very hard to see that a convex combination of these correlated strategy which is also another correlated strategy. So they also that convex combination will also satisfies this inequality. So in other words what I am saying is that there is a proposition I will put it if mu mu are two correlated equilibrium then epsilon mu plus 1 minus epsilon mu is also correlated equilibria for each epsilon in 01. So this is not very hard because this follows from this inequality in fact the most important to note here is that this is a linear in mu that is what exactly comes here. This is that this means that set of correlated equilibria is convex this is the main interesting consequence of this one. Then there is another thing another proposition I can put it is if mu n are correlated equilibria and mu n converges to mu then mu is also a correlated equilibrium. So here we are using the S1 and S2 are finite. So therefore this convergence becomes easy if S1 S2 are not finite this convergence becomes little more technical but nevertheless the result will be true but we will not go into those details. So this proof again follows from the same thing if mu n converges to mu if this inequality is true with n's and the continuity of this if mu n converges to mu then the same inequality will hold true it is straight forward comes from this inequality. So therefore this proposition is also true this implies set of correlated equilibria is closed. Remember that set of correlated equilibria is a subset of this probability distributions on S1 cross S2 and S1 and S2 are finite. So therefore the set of probability distributions on S1 cross S2 is compact and hence this is also compact. So therefore for the finite games the set of correlated equilibria is convex as well as compact. So these are very useful results in this thing. In fact there is another thing we can easily prove is that every Nash equilibrium is a correlated equilibrium. Again this is not hard to prove it I will leave it as an exercise. So the most important thing to realize here is that a Nash equilibrium is also a correlated strategy only thing is that the correlation is actually there is no really correlation it is just simply the joint distribution that becomes a product of the individual things if you look at the densities. So therefore every Nash equilibrium is a correlated equilibrium. So therefore the set of correlated equilibrium is always is non-empty. This is trivial from this proposition. Now few points I would like to mention here is that can we prove the existence of correlated equilibrium without using the Nash equilibrium because Nash equilibrium requires a fixed point argument and set of correlated equilibrium the correlation these are defined by certain linear equation nice looking linear equation you should compare this with the inequalities corresponding to the Nash equilibrium this is certainly much nicer than them. So now can we prove it. In fact the answer to this is true we can prove it directly using zero sum games the min max classical min max theorem we can use it to prove this it is done by Hart and Schmidler. So I will not go into the proof of this one. So let us not worry about the proof now it is not very hard. Now another point before closing this session is that why this set of correlated equilibrium is interesting. Now in the previous sessions we have seen certain learning algorithms for example fictitious play BNN dynamics and best response dynamics these are a few of the things that we have seen it and we know that these dynamics need not converge to Nash equilibrium but many of these dynamics we under some reasonable assumptions we can show that they converge to a set of correlated equilibrium they converge to some correlated equilibrium but from there bringing to a Nash equilibrium is very hard and many a times it would not converge to Nash equilibrium but we can show that the limit point will always be in a correlated way. So in that sense this correlated equilibrium is a very of a fundamental importance and sometimes people say this is as fundamental as Nash equilibrium. So these are two very important concepts. We would like to point out that even though correlated equilibrium concept is defined for the non-cooperative games this correlation is a cooperative aspect of the game and under certain assumptions we can show that the if the Nash equilibrium is unique the correlated equilibrium will also be unique but these are all much more technical and we will not discuss. There are also several interesting geometry related aspects here where does the Nash equilibrium lie on the correlated polytope. So this is again an interesting question again there are some beautiful results available in the literature and they are all more technical we will not discuss about them in this lecture. With this I will conclude this session and we will continue cooperative games in the next session. Thank you.