 Welcome back to NPTEL course on game theory. In the previous session we have seen fictitious play in this session we will see another method to solve zero sub games. This method we use differential equations. So, this method is called brown Neumann Nash dynamics, BNN dynamics. So, in this we start with a zero sum game. So, A is the matrix corresponding to the zero sum game then we assume A is symmetric. This is a symmetric game. What it means is that A is a skew symmetric matrix. It is going to be a skew symmetric matrix. Therefore, such games are called symmetric games which we have seen it in the earlier session. So, is this assumption a restriction? Here is a question an interesting exercise. Let A be any matrix game. Consider another matrix game given by zero minus A transpose A0. So, zero or the zero matrix appropriates order depending on the sizes of A and minus A transpose of course that means A. Look at this one let me call this as a B then B is a symmetric game. B corresponds to symmetric game. So, the exercise is to see that the saddle point equilibrium of A and saddle point equilibrium of B are related. At this moment I will not explicitly say what they are which you should try to calculate from looking at the structure. So, this is going to be an interesting exercise. So, therefore computing the saddle points of this B is sufficient to compute the to find the saddle point equilibrium of A. So, therefore assuming that A is symmetric is not a restriction. Now what is the so we assume symmetric game that means the matrix A satisfies now minus A transpose. Now what is the advantage of this one? The important advantage is that value corresponding to A is zero. So, the value is going to be zero that means the optimal value is there. Now what is going to happen with this is that suppose a player if he does not get a zero value somehow he will try to choose something which makes forces the value to be zero. So, this is the idea behind this BNN dynamics. So, in a sense let us say a player 2 chooses y let us say at time zero. If player 1's value corresponding to y is not zero then y will be perturbed. So, some force which brings this y changes to y so that the value of the player 1 is going to be zero. So, this is basically the idea in some sense at any point of time if some strategy is chosen you try to enforce that the value is going to move towards zero. So, this is the idea and let us explicitly mention it what exactly is going to happen. So, there are M pure strategies and of course there are two players. Now what is really important is that at any point of time when a player chooses a pure strategy and you want to look at the way that the player is giving to that particular thing. So, this is basically the idea here. So, let us say you take any EI is a pure strategy and Y is any other strategy for player 2 this is the strategy player. So, EI is the pure strategy of course I can be anything between 1 to M. Now the UIY, UIY that means this is nothing but EI transpose AY. So, this is the value that player 1 is getting when he uses the pure strategy EI. So, this is the value player 1 gets. So, when is Y min-max strategy? Y is min-max strategy if UIY is less than or equals to 0 for all I. So, recall what is min-max strategy? Min-max strategy is the one which minimizes the player this thing you recall what is the mean you recall this minimum Y in delta 2 max X in delta 1 X transpose AY. So, anything which minimizes this that is going to be the min-max strategy of the player 2. So, Y is the min-max strategy we know that by because it is a symmetric game this is always greater than or equals to 0. So, in fact this has to be because the value is 0 this has to be equal to 0 that means minimum of this is 0 therefore that will happen only if UIY is less than or equals to 0 for every I if UIY is less than or equals to 0 then Y is going to become Y becomes a min-max strategy because the minimum over of this is less than or equals to 0 but this is always greater than or equals to 0 therefore both become equal to 0 and hence this happens. So, this is necessary to now let us define some terms we define phi IY to be maximum of 0 comma UIY and define phi Y to be phi 1 Y plus phi 2 Y plus so and so phi M Y I am taking this sums. Now phi IY is giving return of player 1 if UIY is greater than or equals to 0 whenever UIY is bigger than 0 because UIY is the payoff that player 1 is receiving therefore this phi IY will give you the value that player 1 is getting whenever this happens. Now the BNN dynamics are given by the following thing. So, this is given by DYIT by DT is phi IYT minus phi YT into YIT where T greater than 0 of course I is equals to 1 to M and Y0 initially starts with any strategy of course delta M here is the set of all mix strategies for the players. So, I have been writing delta 1 and delta 2 here both are same so here delta 1 is equals to delta 2 all of them are same. So, this is basically the dynamics that we have it. So, what is real is YIT is essentially telling you the strategy that the player is going to use when the player 1 uses the pure strategy I so that is it is giving. So, now in this dynamics phi we do not know a priori whether YT remains in delta or not. So, therefore I need to extend these things outside this thing. So, we extend fees to whole domain by phi of Y is nothing but phi of Y by mod Y if Y is not in delta and of course if Y is equals to 0 we just simply take phi 0 to be 0. So, this is basically a of course I am using the same values here same phi but without loss of generality and with an abuse of notation we are doing this but we should be careful in looking at it. So, outside this delta if you have another point Y another vector Y then phi Y I will define simply to be phi Y by mod Y and of course when Y is equals to 0 this phi 0 is simply defined as 0. Now, this is basically the dynamics. So, let us let me write down once again the BNN dynamics is nothing but d phi I d YIT by dt is nothing but phi I YT minus phi YT into YIT t greater than 0 and YN 0 is in delta. Now, the most important thing that would like to point out here is that this phi I fees they are all continuous functions. So, nice functions not only that the definition if we look at it here phi IY is nothing but the maximum of 0 and U I Y and U I Y is the payoff that player 1 is receiving it that is this that means it is linear in Y that immediately implies U I Y is linear in Y and maximum of 2 linear functions therefore phi I Y is going to be a Lipschitz continuous functions phi I Y is a Lipschitz continuous function and of course automatically phi which is sum of phi I is is also Lipschitz continuous function. Therefore, being a Lipschitz continuous functions we can immediately say that by applying this Cauchy Picard's theorem this differential equation the system of differential equations is I is equals to 1 to m here the system of differential equation has a solution and in fact it is unique. So, by Cauchy Picard theorem BNM has unique solution. Now, the most important thing that this does not this does not guarantee that Y t is in delta. So, we still require because the solution should remain inside delta because delta is our mixed strategy space and Y t is Y t is basically Y 1, Y 2, Y m Y m that should be a mixed strategy that means this they must sum to 1 that it does not guarantee that requires a proof. So, we will now try to prove this one. So, for this we consider an auxiliary body which is given by dxit by dt which is phi i x t and x at 0 is nothing but Y at 0 delta. So, I am simply considering dxit by dt is same as phi i x t. You look at this simple what we did is this particular term we have avoided there. So, without loss of generality we can assume phi i x at 0 is greater than 0. So, why are we assuming it? If x at 0 is 0, if phi i x at 0 is 0 that means the ui x at 0 is less than equals to 0 therefore that is automatically becoming I mean max strategy. So, we do not need to worry about solving any further. So, we restrict to this case if this case is not true then it is obviously x 0 is going to be the strategy. So, therefore we without this is a without loss of generality we can assume this. Now once again phi is Lipschitz continuous. So, this auxiliary equation has unique solution and of course we are going to have everything in r m minus singleton 0 here because phi i is greater than 0 all the time whatever it is because by the definition of phi i this is always greater than equals to 0. So, this term is always greater than equals to 0 therefore x i t as t changes the derivative is non negative therefore x i t this is always increasing therefore x i t is non decreasing. So, this is a important fact therefore x i t is always greater than equals to x i 0. Now if I assume x i 0 is greater than 0 because that is our without loss of generality x i 0 is strictly greater than 0 implies x i t is also greater than 0 that is a important thing and all of them are Lipschitz continuous. Now consider e alpha t by dt is summation i is equals to 1 to m x i alpha t equals t greater than 0 and alpha 0 is 0. Now we are considering another ODE which is given by t alpha t by dt is nothing but summation x i alpha t. Now x i t are always greater than 0 that is this thing therefore x i x i is a Lipschitz continuous function why x i s are Lipschitz continuous function they come from this fact x i s are the solutions of this and the derivative of this is given by phi i and what is phi i phi i is defined by this and we know that the payoff phi i s are all bounded this is a bounded thing. So therefore phi i is a bounded and that requires another this thing we have extended this phi i outside okay I have not written here in the previous slide but we extended phi i y is nothing but phi i y by mod y we have extended this outside the simplex. So therefore phi i is always bounded in respect of what why it is because of that this is bounded term therefore d x i by d x i t by dt is bounded therefore x i is a Lipschitz. So x i s are Lipschitz continuous therefore there exists again unique solution alpha. So Cauchy Picard theorem we have applied again third time here. Now this is also greater than 0 so that all of that we have. So now let us look at it d alpha t by dt is this is nothing but summation x i alpha t where i is equals to 1 to m and x i we already proved that they are monotonic therefore they are greater than equals to sigma x i at alpha 0 that is 0 this is nothing but 1 this implies alpha t is greater than equals to t this for every t. So now we define y i t to be x i alpha t by summation j is equals to 1 to m x j alpha t now I have normalized this x i s with this thing therefore this y t now becomes delta. So this happens for each t greater equals to 0. Now climb is y t is basically is solution of BNN dynamics. So this is not really a hard thing to prove it what we have is the following thing what is y i t into summation j is equals to 1 to m x j alpha t this is nothing but x i alpha t by the definition of y so from here so this is there. So now if I differentiate both sides with respect to t then what we are going to get is y i prime t summation x j alpha t plus y i t summation x j prime alpha t into alpha prime t this is into this is same as x i prime alpha t into alpha prime t. Now this is this thing and we use all this what is x j prime t alpha prime t on all these things if you put it then little bit of arithmetic here algebra little bit of algebra that gives you that y i prime t plus y i t into summation phi j y t is nothing but phi i y t. So I have excluded the algebra here and once you do this algebra here just simplify because we need to use what is x j prime alpha t alpha prime t every things using the previous equations start putting it then we will get this one this implies y is solution of BNN dynamics that is all. So that is the first step next we will take without with an abuse of notation we simply use phi i t in the place of phi i y t instead of repeating phi i y t we simply use phi i t. Now let us define psi t to be summation phi i t square. So psi t is nothing but this phi i t squares of course we have suppressed this y t here. So that is a understandable thing here now then what we can do here is that suppose phi i t is greater than 0. In fact if you recall this we have made as a without loss of generality assumption then d phi i t by dt this is going to be e i a dy t by dt. So this y is this coming it is coming from here from here because I have assumed phi i y is greater than 0 that means phi i y is nothing but u i y therefore the derivative of phi i y is nothing but the derivative of u i y and the derivative of u i y is exactly what is written here. So that comes from there now this is same as a i j phi j t minus phi t summation a i j y j t. So just substitute whatever y is there and write down all the things and this is exactly what we will get it. So once we get this what we have is the following thing the 2 phi i t into d phi i t by dt what is this? This is nothing but d phi i t square by dt we are calculating this one. If I calculate this one this is going to be the following thing I will just write it 2 summation j of course here a i j phi i t phi j t this is summation over j only here minus 2 phi e t summation over j a i j phi i t y j t this is what this thing. Now in this inequality even if phi i is 0 if phi i t happens to be 0 then this side it is 0 and you can easily verify that phi i t is here so therefore this also becomes 0. So therefore this inequality is true even if phi i t is 0 this equality is true even if phi i t is 0 that is very important. Now with sum over all i then that gives you summation d phi i t square by dt this is going to be summation 2 into summation i comma j a i j phi i t phi j t minus 2 phi t summation a i j phi i t y j t of course summation i comma j. Now we know that it is a symmetric game a is equals to minus a transpose if because a is minus a transpose a i j is nothing but minus a j i we use that one then what we will get here is that this term the summation the second term here is going to be same as summation phi i t the second term is equals to summation phi i t into summation a i j y j t this is going to be psi t psi t we have already defined here. So this is going to be the psi t so if that essentially means that the following thing we proved we proved following lemma the lemma is psi t satisfies d psi t by dt this is nothing but minus 2 phi t psi t. So here before going I have only said about this second term what happens to the first term the first term becomes 0 because of this condition only this second term remains that is that comes to be minus 2 phi t psi t. So d psi t is equals to minus 2 phi t psi t now using the ideas from the differential equations what we can now prove here is that from ideas in differential equations we can actually prove that root psi t is less than or equals to phi t which is less than or equals to root m into psi t. So this is it exactly comes from the this one you can use the integration by parts or whatever way it is you will get this value and take this complete this details here the details. So once this happens now what we have here is that phi is non is a non negative things therefore this is a negative thing psi t is a non psi t is a decreasing function. So psi t is a is decreasing that is first thing and another thing is that phi t is greater than 0 as long as psi t is bigger than 0 that comes eventual x from here itself if psi t is greater than 0 phi t has to be greater than 0. So if this is greater than 0 this also is greater than 0 and from this equation this is a negative term so therefore psi t is a non increasing so that comes. Now using this bounds what we can of course why this is coming this also comes from the definition of psi so you can use this from here also you can see it that this particular inequality is coming and use this inequality and to say that the following happens d psi t by dt is going to be less than equals to minus 2 psi t power 3 by 2 and we can also say that this is what is coming this immediately actually using the ground walls lemma kind of stuff we can say that this psi t is going to be less than equals to psi at 0 by 1 plus root of mod y 0 into t this square this comes this comes from ground walls lemma using the ground walls lemma you will get of course this inequality is coming from this inequality use this inequality here and if you put that we will get to this and eventually you will get this one. So what we have if psi t happens to be 0 then even this particular thing this is also true this is true even if psi t equals to 0 it does not matter now. So therefore this inequality is true always in fact we can prove psi t is nothing but psi 0 into e power of minus 2 integral 0 to t phi s d s this is there from here also we can see that psi t is going to 0 as t goes to infinity. So this is coming basically from because there is a t here as t goes to infinity this term is going to 0 therefore psi t goes to 0 as t goes to infinity. So what is psi t? Psi t is this so summation phi i t square summation phi i t square is going to 0 as t goes to infinity that means each phi i t is going to 0 phi i t is nothing but phi i y t is y t therefore what we have here is that phi i y t is going to 0 as t going to infinity that means what is phi i by going back to the definition of phi i is this phi i y t is going to 0 that means u i y t is going to 0 that means y is going to be the min max strategy that means y t converges to a min max strategy as t goes to infinity. So this proves the following theorem thus what we have is the following theorem. BNN dynamics are asymptotically stable any limit of the trajectory is a saddle point into d f. So this is the theorem that finally we have proved. So if you really look at it the idea that what we have done is first we showed that the BNN dynamics has a strategy unique trajectory then to show that the trajectory converges something we use the Lyapunov function idea. So we have used that the psi works as a Lyapunov function and that proves this one. So this is a slightly technical result but the proofs are not really hard but it is a very interesting method to show the to compute the saddle point of asymmetric zero sum games. In fact any zero sum game can be symmetrized and hence this works with any general zero sum games. With this we will conclude this session we will meet again in the next session.