 theory started with ecologists who were essentially interested in explaining animal behavior or say behavior in economic, say in biology. And so essentially the idea is that the main question is why do you find a certain type of behaviors in an animal population or in a population of individuals of a given species? Typical behaviors are like typically mating behaviors. So you know often in the animal world you find animals who are fighting for females, okay, for mating. And essentially there are other types of behaviors which are that are very particular. And the idea is that if one particular type of behavior is a behavior that is most effective for reproduction, for passing on the genes of an individual to the next generation, then evolution will select for this behavior. So essentially the idea is that you think the main issue, the main object in evolution is what is called fitness. And you think at fitness as being a function of both the genes and of behavior. So, oh sorry, I think I should start recording also. Okay, so registration start. Okay, so now there is an issue of, so let's make an example. So a typical example if you look at the mantis. So the mantis is an insect and there is a female and a male. And when the male meets the female, essentially they mate and then after mating the male lets himself be eaten up by the female. And you would ask, I mean in principle you would say well this is not a particularly successful strategy for a male mantis because essentially every time he meets, he mates is going to die. But actually this is a very successful strategy for the genes of the male mantis because in this way the female does not have to go and look for food. And instead she can just stay where she is and lay the eggs and with the resources so that she doesn't have to worry about getting food. Now, so what this essentially, so what this essentially says that what is essential is that in evolution the game is not played by the individuals but is played by the genes. So we have a question on do male mantis know that by experiments of other male mantis know? I mean the real issue is that for a male mantis it is very hard to find a female mantis. So in his lifetime the chances of meeting a female mantis are very small so that when he meets a female mantis that is his only occasion to pass on his genes. And this is why this strategy is successful but somehow this strategy is encoded in the genes or this behavior is encoded in the genes or in some other form. I mean maybe it's encoded in epigenetic markers of the genes that are passed from one mantis to the other. But in the animal world you find a lot of this type of strange behavior. So another one which is discussed a lot in this video of John Minor Smith is the one of fighting and display by animals so that essentially the males who mate with a female are those that endure most in fighting with other males. So there is this game that is played within the population that also has this feature of passing on as this function of passing on most effectively the genes of the strongest individual. So then how do we discuss this in terms of in terms of evolutionary game theory? What is the idea of evolutionary game theory? So the idea is that of defining in a game what is called an evolutionarily stable strategy or ESS. What is an evolutionarily stable strategy? Well you think about a population that is of individuals and these individuals are playing a particular so they are interacting. I mean and this is you think that at any time two of these individuals meet at random and they play a game and each of them plays a particular strategy and so if you let the system evolve in this way what you will find out I mean the idea is that what you will find out is that the strategies that survive along this series of random encounters within the population are those that are essentially the most successful ones. Okay so so the idea is that if this so an evolutionarily stable strategy is a strategy such that if all the individuals of a population play this strategy and at a certain point you have the appearance of some mutants that play a different strategy so then this mutant will not invade the whole population. Okay so essentially the mutant the payoff of the mutants playing against the other the other individuals will be smaller than the payoff of the individuals among themselves. Okay so are there any questions about this? Can you repeat how you introduced the mutants? So the mutant is just any other strategy that may appear because of random mutations. Okay okay thank you. So it's but the issue is that at the beginning you have an homogeneous population and at a certain point you have a mutation in this homogeneous population and so there is a small fraction of individuals in this population that plays a different strategy and the question is will this small fraction will be able to take over or not? Okay and you have an evolution a stable strategy if this does not happen. Okay whatever is the mutation. Okay so I was asked also to explain better what is fitness so fitness is generally defined as the expected number of as the expected number of offspring at the next generation so essentially all bob springs that reach reproductive age. Okay so essentially a species or a strategy that confers a higher fitness will have a population that grows faster than another which has a lower fitness. Okay so so can any two meet twice or more? Yes they can so in general when we discuss evolutionary game theory we think at the limit of an infinitely large population so that every time you pick two individuals at random but the chances that you pick two individuals at random at different times is very small. So could you explain a little bit more about strategy in this specific case? So strategy in this case is just a behavior. Okay so the mantis study you can think of in the population of mantis there are some males that have this particular behavior that they let themselves be eaten up by the female and someone and some males that do not. Okay so these are two different strategies and the point is that those that do not let themselves be eaten by the female have a lower fitness than the others and so this is why we don't observe them in in the population. Okay is the ESS defined within the species or can we express it among species as well? Okay so in the real situation species interacts both within the species and with other species but essentially if we are interested and essentially fitness also depends on how you are efficient for example in predation so in predating over other species or how you are efficient in escaping predators that predate on your species. Okay but here we only consider fitness so but this in some sense say for example if you consider different species this is a different game and here we are only talking about games that are played between individuals of the same species. Okay so in terms of competition for common resources beyond just okay yes common resources are also another issue yes yes please. Is it possible in our system we have a three ESS yesterday you said in our system we have a three national data beyond and if I correct is it possible in our system we have a one bigger one and we have two or three ESS? Yes it is possible it is possible to have in the same game more than one ESS so what do you for example yesterday okay now we are going to discuss more formally in terms of game theory but yes it is definitely the case that you can have more than one ESS. Any other question? Okay so very good okay so how do you define an evolutionally stable strategy in a particular game context? Okay so the idea in a game is that essentially you have a payoff of a particular individual i that is playing with strategy si against the opponents okay and so we are going to think about the fitness of individual i that is playing a strategy si in a particular population okay as as the sum as essentially expected value of the payoff that he may get playing with opponents that play a certain strategy a certain opponent strategy where this is the fraction of individuals that play strategy s okay so this is essentially the the the idea and essentially so the if you have so let's for example make an example so let's take this example which i discuss in the notes where essentially which is essentially a prisoner's dilemma where you have two strategy for player one which is say here i call them s and l and one and eight eight and one and three and three okay so in in this case well you can study the Nash equilibrium of this start of this game and what you can easily find is that essentially the best response for player two uh if player one play s is is eight and is is l and is l also if you place l and the same is true for also player one so that this is the Nash equilibrium you know it's the Nash equilibrium of the prisoner seven so now we can ask so is l an evolutionally stable strategy and in order to do this you will have to think about a mixed strategy where essentially you have now you change your population and you introduce a small fraction say epsilon of people that of individuals that play the strategy s okay so so this mixed strategy will play strategy l with probability or say it will be the one minus epsilon you have a fraction one minus epsilon of individuals of type l and you have a fraction s of individuals that play s okay very good so now you what you have to do is to look at what is the payoff of one individual that is playing l in this modified population with the mutants so the mutants are these guys here and compare it with uh with the payoff of the mutants the mutants are the s in this population okay so and if you do the math this is very easy you find that this is three plus three x whereas here you find one plus four x okay so you see that the payoff of the resident population is larger than the payoff of the mutants so in this situation the fitness of the mutants will be smaller than the fitness of the resident population and uh and the mutant will not invade sorry here i'm not using x so i'm using epsilon thanks donessa okay so uh for example if you want to compute uh this one uh you have there is eight times epsilon uh plus three times one minus epsilon sorry times one minus epsilon and then you get this number here okay so um excuse me yeah can you explain how how you calculate these uh by uh having these um one minus epsilon and epsilon i didn't quite get it uh okay so this uh so you one of l and sigma so you want to compute what is the payoff of player one uh against uh a population that uh uh is playing s so so this is so you have a fraction epsilon of people playing s and a fraction one minus epsilon people playing l okay so then uh this means that your payoff will be this payoff will be eight times uh epsilon okay thank you because you are playing l yes uh plus three times one minus epsilon uh huh yes yes yes and so yes so sorry and then uh uh what you have is actually uh three plus five five epsilon yeah thank you yes okay very good so um now this uh so in this case sorry sorry i cannot hear very well sorry i can there's some unexpected noise sir you can mute him okay okay let's see okay very good okay so so then the question that arises here is uh so um so is so you can ask the same question no so you can ask uh is uh is uh uh s an evolution stable strategy and uh you can do the math and you you can see that uh it is not okay because uh a a population of individuals playing s can be invaded by a small fraction of individuals playing l and this is because l is a dominant strategy okay so the question then uh uh is uh uh is uh uh uh a population that plays an evolutionary stable strategy uh equal uh to a national equilibrium or a population that is playing a national equilibrium so uh if you have a population that uh uh plays an evolutionary stable strategy then uh this must be a national equilibrium okay so the converse instead as we will see is not true okay so um if you have a um uh uh uh um and and say in order to see this we can analyze a different game you can take this game here for four zero four four zero three three this is similar to the other game and uh now if you look at what is the best response of player one when player two plays l whether it is this one or this one and if it is playing r then it is this one and you can do the same for the player two and uh so what you find is that uh here there are two national equilibrium one is this and one is this okay however uh if you uh if you do the math you will find that uh say uh sorry uh r r so r actually uh r uh is an evolutionally stable strategy because essentially cannot be invaded but l is uh uh not an evolutionary stable strategy okay and uh you just have to do the math so so this is uh uh this is a national equilibrium uh but the strategy that are played and the national equilibrium are not evolutionary stable strategies okay are there uh questions yes professor so uh for the prisoner's dilemma if you could go back just one slide so we could see that the payoff for uh one of the persons not cooperating is eight versus one so that's a very large payoff and if you calculate so the e the u one of s sigma so what what happens if um when we're looking at whether s is uh an evolutionary stable strategy you could see that it's um not for all values of epsilon you get something yes yes three minus six so i'm epsilon is small so you want to understand whether you have a mutation you have one individual in a large population that has a new uh gene which is a mutant and you want to see whether this takes over so epsilon is small okay yes but it turns but if you calculate u one uh for the two cases you get like uh that the difference between uh u one by by a mutant and you won by a resident using always s it's like three minus x so that's positive for all values of x okay so let's do this so now you are saying so let you want to see whether say s is uh an evolutionary stable strategy right yes yes so essentially we are going to the the population with the mutant will be s with probability one minus l epsilon and l will probably be epsilon okay okay then uh what we have to do is to compute this u of s against this sigma okay and uh yes now we have that this is the other way around so this is one minus epsilon and this is epsilon okay and uh so uh what is the payoff of player one if it plays s against s so it's five times one minus epsilon uh plus uh one times epsilon right is it okay whereas uh for if uh a mutant the payoff of the mutant will be eight times one minus epsilon plus three times epsilon okay so this is uh uh five minus four epsilon or epsilon and this is uh eight minus five epsilon okay so uh if epsilon is small then uh these uh the resident population that's the s people will have a smaller payoff than the mutant and so the mutant will take over okay are you with me Carlo yes yes yes i'm here yes so exactly so you so you get a three minus uh epsilon is the difference between u one l sigma and u one s sigma so that means that it holds for all epsilon um yeah could you give an interpretation of that perhaps no i mean uh what you care is only small epsilon okay so okay but in this case uh say uh it is a very stable uh uh okay uh nash equilibrium okay it's a very very stable strategy yeah but it's also but it's also dependent on the coefficients in the matrix right because yeah of course if you get everything depends on yeah okay okay thank you okay so the next thing uh uh that i want to discuss is that uh uh so you would think uh that then uh well uh evolution uh should always uh uh uh lead to one strategy or the other but uh this is not so uh uh in uh this is not uh necessarily so so and the idea uh in order to see this uh um uh let's consider the another game which is uh a game which is very popular in uh uh this um evolutionary game theory which is the help of game uh help no game so the idea is that uh every individual can play hope or dope so this is typically uh in a animal conflict between two individuals in an animal population then uh uh the question is how aggressive you want to be and uh uh the idea is that if you are very aggressive you play as a hawk if you are not aggressive you play as a dove and if you are a hawk and you meet another hawk then essentially the fight is going to last for long and maybe uh there's going to be some injury okay so we will put the payoff of minus one for a hawk that meets a hawk instead if a hawk meets a dove then the hawk will win very easily and the dove will get zero okay so the the idea I mean if you look at the video of the lecture of uh john minas meet I mean the the idea is that there is a resource and uh that has to be split and and also that is the cost of fighting and so a dove will not pay any cost of fighting but will also not get any any resource okay and so uh uh here you get uh the opposite of uh of of of of these and uh if a dove meets a dove instead they just share their resource and so they have one in one okay so now if you do the if you look for a Nash equilibria then uh the Nash equilibria are either when a hawk meets a dove or when a dove meets a hawk okay so there are two Nash equilibria but in the Nash equilibria the two players play different games okay and actually if you look at this game there is also a mixed strategy Nash equilibrium which is essentially playing half of the time as a hawk and half of the time as a dove okay I leave you as an exercise to uh to prove this okay so you can uh prove that uh this uh sigma uh is uh an evolutionally stable strategy well and why is this so well imagine that you are in a population where half of the people is a hawk and half of the people is a dove now imagine that uh you are a hawk and you decide to become a dove then uh the chances that you are going to uh meet uh uh a hawk increases uh sorry uh so your your payoff well so with probability uh one half minus epsilon you will uh uh uh meet a hawk and with the probability one half plus epsilon you will meet uh a dove okay so if you uh switch uh to a dove so if you have uh say epsilon more doves so then the payoffs of those of those will be equal to um with probability one half uh uh minus epsilon you will meet a hawk and then uh you will have zero and with probability one half plus epsilon you will meet a dove and you will get one okay if instead you are a hawk then uh uh with probability one half minus epsilon you will meet a hawk which is minus one with probability uh one half plus epsilon you will meet a dove and you will get two okay so uh here uh if you see then um uh this is uh one half uh plus uh three half times epsilon and so you see that uh hawk get a higher payoff than doves okay so in a population where uh there are slightly more doves the payoff of hawks is larger which means that hawks will reproduce more than doves okay and so this uh logic is uh uh say is a little bit uh the same as the one by which in animal population typically you have a sex ratio which is one half one half okay or in human population you have a sex ratio one half one half because essentially if you have a more uh females than males then females males will pass more likely they are genes than uh females okay so do you have any questions on this professor yes please so still in the idea of the epsilon the epsilon must be small but is there a threshold where the epsilon increases and then the the the certain strategy is not it's not a ESS anymore is there a threshold uh yes you can uh if you do this uh calculation you can figure it out I mean it's uh it's uh sort of uh easy you know uh you you do this calculation and uh you you can find out easily no okay other questions okay if not so let's uh go back and discuss uh more uh the mathematics of this uh evolution again theory and this is essentially what is called uh the replicator dynamics so uh replicator dynamics is essentially a way to put all these considerations into differential equation for the populations of individuals okay so again we think there is a population of individuals and in this population that are each individual is playing a particular strategy s okay or is of type s and the fitness of these individuals is a number fs okay and so essentially if ns is the number of individuals of type s in this population then uh the the the evolution of of the number of individuals is given by the fitness times ns so this tells you that essentially the fitness is the how fast uh the species s reproduces okay so essentially if this is all that is going that is going on and if the fitness does not depend on on uh who you meet i mean these these are not interacting then if you plot the log of ns versus t you just get the straight line okay and then uh if you have a different type of uh individuals the red individuals they will have a different fitness and they will go with a different rate okay and uh if you have another type of individuals maybe they will go at a different rate okay so what you what you see is that then uh if you look at the fraction of individuals of type s uh that are there at a given time then uh it is clear that uh over time this will be dominated by the type of individuals that grows faster okay and uh essentially you can write down an equation for the xs in dt and uh figure out that this is going to be equal to uh xs times the fitness minus the average fitness in the population okay so if your fitness is larger than the average fitness of the population you are going to your fraction of the fraction of individuals of this type will increase if it is lower then it will uh it will decrease okay uh is this clear for everyone okay so i assume it is clear for all of you okay now uh let's now instead assume that agents these individuals are interacting they interact with each other and so the payoff of an individual uh actually depends on uh who uh on what the others individuals are doing okay so uh then um uh we have uh uh that uh um so the fitness of an individual at time t in a population is going to be equal to uh the average payoff that uh is going to get in this population uh if he plays this game with strategy s against the opponents that play strategy s prime drone at random uh from uh the from the population okay and then uh uh in this replicator that says a question in this replicator dynamics individuals are divided into types based on what strategy they use however what about individual with mixed strategy so yes so in this case uh i'm thinking uh uh about the situation where each individual is um in the population is playing a mixed strategies committed to one strategy okay and uh you have you can also think of individuals that can randomize their behavior but for simplicity i'm just thinking about individuals who are either oaks or doves okay and uh excess is just uh so in some sense the mixed strategy is a result of an average over the population okay very good so uh uh are we supposing the total population uh of an s is fixed no we don't need to uh assume that the total population is fixed indeed uh in this case you can see that the total population uh will also increase in time in this particular case okay okay so if the fitness is defined defined in this way uh then uh uh uh we uh we can write down what is uh the equation for the fraction of individuals that play strategy s of type s in the population and this is just uh the equation that i've wrote before it's just uh uh that uh the fitness of individual type s is just this one uh you one of s s prime times x of s prime and the average fitness now is a sum of s prime s double prime of x of s double prime you one of s double prime s prime x s prime okay so this is called uh the uh replicator dynamics and uh and essentially uh what you can uh so it's one form of replicator dynamics so in general you can think that uh you can have a different uh relation between fitness and payoff and uh the different model that have uh different make different assumptions but this is the simplest uh assumption uh that you can make okay now uh let's see uh what is the behavior of this equation in simple uh say two by two uh games so symmetric two by two games uh like the games that we have studied just two players and uh uh two strategies then uh in every case you can write these games uh in uh um in a form in a form that is uh say uh with uh uh payoffs that are a one one a one two a two one a two two these are the payoff for this is strategy one and strategy two and uh these are the payoff for player one and uh the payoff of player two are just uh the uh transpose of this matrix okay okay uh very good so uh now uh if you uh write down uh this uh uh equation I cut a little bit uh this story short because we just have five minutes so uh if you write down uh the uh this uh dynamics replicator dynamics for this case well first of all you can say that x one is just equal to x and x two is equal to one minus x because notice that uh the by replicator dynamic by sum over x d x s divided by dt then I get zero so because uh uh when I summarize here you see that uh uh the uh sum over s here gives me just the average payoff so um when I sum over s here I get exactly this term here and when I summarize this term well this is a constant and I just get one here so essentially if the population is is normalized and keeps being normalized by this uh dynamics okay so uh the equation that we get for x is just that uh dx dt is equal to x times one minus x times uh a constant a plus b times x okay and then uh well uh you can study these dynamics very easily uh if you uh write dx dt as a function of x okay now for example if you have uh that a and b are positive then uh you see that uh uh dx dt is also positive okay it is also non-negative so you will get something that goes like this okay and so this is the dynamics by which uh if uh your x is at this point at some time the x dt is positive so you will move in this other in this right direction and essentially you will get to this point so this x equal one will be the evolutionary stable strategy okay now depending on the values of a and b you can have either this situation or the situation where if a and b are both negative uh then uh you can have this behavior here but you can also have a behavior like this one and the red case will would be one where the ESS evolution stable strategy is here and it is a green case you will have that the evolutionary stable strategy is here because if you are on this side you move on the right if you are on this side you move on the left because the x dt is negative but also you can have a situation like uh like this one okay well essentially you have two stable points okay two ESS which are this one and this one so essentially uh the point I want to make is that the asymptotic behavior of this replicator dynamics so if you take the limit as t goes to infinity of these replicator dynamics then you get these evolutionary stable strategies okay so questions so we have one question in the chat uh okay no no question is everything clear so uh not any any any way you can find all these things in the on the website as well as a remarkable lecture of John Minor Smith who is uh um one of the founding fathers of evolutionary uh game theory and essentially uh the other thing that discusses is uh also the issue of evolution of co-operate of cooperation so the other issue that you can ask yourself is why is it that uh in a context like uh Britain's dilemma so you um you see uh a lot uh more cooperation in society than you would expect okay and uh so the the the the reason uh for these uh is that essentially in uh one day as we have seen yesterday is that uh when interaction is repeated then uh cooperation becomes uh a Nash equilibrium of a repeated game but you can ask yourself uh well is the uh strategy that we discussed yesterday is uh um trigger strategy an evolutionally stable strategy and the answer is no and the answer is uh uh well when you ask yourself about um uh evolution stable strategy in a repeated game uh because the strategy space is so large then you'll have to uh uh I mean it's a very difficult question so what Robert Axelrod did at a certain point was to uh set up a tournament a tournament between strategy playing the repeated prism dilemma and what he found out is that uh even in uh a finitely repeated game there is a very simple strategy which is evolutionally stable although it is not a strategy that you will find uh by backward propagation as we saw yesterday but it's evolutionally stable and the strategy goes is called tit for tat and goes like uh uh the following so uh you start cooperating and then at every point in time you do what your opponent did the the previous time okay so if your opponent uh also starts cooperating then you'll cooperate and then this means that uh if your opponent also plays tit for tat then you will cooperate forever okay so if your opponent at a certain point defects then you will defect after but if your uh if that was just a mistake so your your opponent starts cooperating again then uh you also start cooperating again okay so this is the difference between with trigger strategies that tit for tat also uh as a picture of forgiving your meeting or misbehave i will punish you as long as you misbehave but after that i will continue cooperating and this is uh uh an evolution you can find out that well what Axelrod found is that this tit for tat was the best strategy was winning against any other strategy it was not uh invadable um if you have a population of players that play tit for tat it cannot be invaded by any other uh strategy so i think uh uh we need to stop here and um um if there are no questions are there questions no questions okay so let's take uh uh 15 minutes break before uh before uh going back to before before the next lecture thank you very much