 Hi, everyone. Thanks for taking the time to watch this recording of the lecture that I was supposed to deliver in Berlin. I'm sorry about the itinerary mess up. Well, let me get straight to it. I like to frame this talk in terms of a Greek myth, the myth of Tantalus, who was cursed by the gods to forever be reaching for food that he could never actually grasp. Every time he would reach for a piece of fruit in the branches above his head, it would recede from his grasp. Often in the study of social behavior or social heuristics, as a subset of that, we often feel this frustration that we are tantalized. The theory recommends particular heuristics that we should find and yet when we see empirical verification, they recede as it were from our grasp. In this talk, I'd like to try to shed some light on why I think that happens and propose some advances, some ways to proceed to help us actually grasp some fruit. Let me get into that by changing the topic from Greek mythology to lions. Lions are social carnivores. They live in groups. It's not entirely clear exactly why they live in groups. There are many options. For example, people often say they live in groups because carnivores hunt better in packs, but that alone doesn't mean that they should live in groups. It merely means that they should hunt in groups, and yet lions actually hang out as seen here in groups almost all the time. And one of the probable reasons why is because they defend territories and they're young from other lions. In particular, lions like most mammals are infanticidal. When males enter new groups, they kill offspring that were sired by other males. This brings females back into cycling faster and helps them spread their genes. So females have a collective interest in keeping new males out of the groups. This favors territoriality as the story goes and group living. However, there's a dilemma to be solved. The lions need some sort of social heuristic by which they can stop lazy females from cheating on their defensive territories. And of course for a female lion to repel a male lion does require some assistance because they're highly sexually dimorphic. This has been studied in some depth in biology over the last several decades, and studies are still ongoing. One of my favorite moments in the history of this empirical study is this newspaper article from an Ohio newspaper in 1995 entitled Lions Don't Play Tit for Tat. And this was a big scientific result. I was just starting graduate school when the science paper that this newspaper article is based on came out. And it generated something of a tizzy in the evolutionary biology community as we were so puzzled that we did not find tit for tat in lions. What could they be doing if we can't find tit for tat here? Shouldn't they be using reciprocity? And some people actually inferred, including some of the authors of the study, that this evidence suggested that reciprocity is not part of what maintains cooperation in lions. I think that this is a sad episode actually in the history of the study. It's one of these examples of being tantalized because we have the wrong target. I'm going to argue. We didn't find tit for tat and we never should have been looking for it. Instead, the models suggest something quite different and in this talk I want to develop what they actually suggest. There's a story here of the misreading of theory, a story here of empirical frustration, but in the end it's a happy story because out of this episode in 1995 a lot of broader understanding has arisen. And now I think most people who study reciprocity in wild populations aren't looking nor do they expect tit for tat to be what animals use. What is replaced it is going to take the next 40 to 50 minutes to develop, but I want to come back to tantalize for a moment and talk about some of the reasons the fruit keeps receding in the study of social heuristics. Divide these for convenience into three categories, the historical obstacles, empirical obstacles, and conceptual obstacles. I'm mainly going to focus on conceptual obstacles in this talk, but the other two categories are going to get woven in as we go through and really you can't separate these three things. They're all part of the same complex of issues. On the historical side, I'll start by showing this image of Fortuna, the Roman goddess of chance or fortune, which was a fickle and somewhat wicked personification to the Romans, the antithesis of Minerva, the goddess of wisdom, who was deterministic and sought to bring understanding to humans. Fortuna on the other hand was fickle and dangerous. The wheel of fortune that she holds in her hand there brings you up. It will then inevitably bring you down as the man who is sitting on top is about to experience. This is part of the historical obstacle to studying heuristics, is that we have a tradition in the West of not looking for them, of expecting optimization or deterministic solutions that are reliable in all contexts, that we can we can somehow extract knowledge from its context and have it be useful. And it's taken us a long time to develop intellectual traditions that have gotten out of that problem. Gigerins and colleagues have used this taxonomy of visions of rationality, which I think is still quite descriptively accurate of the debates across the social and biological sciences, that one of the historical obstacles we face in studying social heuristics is that lots of people aren't looking for them. They're looking for demons, as described here. That is the sort of beings of infinitely computational power and that we want to imitate them somehow and seek the kinds of solutions that they would find. Instead, we find ourselves as boundedly rational agents and we have to seek other things. But many of the tools that we have developed in social sciences and biology, economics in particular, are focused on demonic solutions. And they don't help us solve these problems, but it's often all we have to work with. Part of that historical baggage is the empirical methods that are available to us, but let me cut them apart from pure history for a moment. In this talk I'm going to weave in some of the empirical challenges by which I mean how we make models with data to study social heuristics. Part of this empirical challenge, as I'll try to explain as we go, is that it's often quite difficult to differentiate social heuristics, even when we know the truth of what they are. And I'm going to try to pin that in. But this means the statistical tools that all of us were taught as undergrads and often at graduate school are absolutely not up to the task of studying social heuristics. And part of that is also experimental design, which is a point I would like to close with today. But as I said, for most of the talk I'm going to talk about conceptual issues. The conceptual issues are that I think you can get a better grasp on historical and empirical obstacles in studying social heuristics by understanding the broader conceptual nature of the environments that they live in. And these environments are social environments that are highly dynamic and in a sense cybernetic. I hesitate to use that word because it lacks content for most native speakers. But all I mean is that in, for example, the scene of a school of fish with some sharks, most of these fish don't know where the sharks are. And yet the population of fish certainly does know where the sharks are and avoids them very well. And that's because each little fish with its tiny bit of information can signal to its immediate neighbors. And the immediate neighbors can start a cascade of information and behavior that helps the entire population of fish avoid dangers and forage while doing so, even though each individual fish lacks the resources or cognitive abilities to do this on their own. And so the social heuristics of the fish, and of course of the sharks here too, depend upon this environment. They depend upon being in a school of fish in order to function and absent the population. The social heuristics make no sense at all. And that is the general conceptual truth of all social heuristics is that they only make sense in the context of a complex population. Social heuristics live in complex populations, and that means that the study of them should be different than the study of simple heuristics that work in static environments. A little bit of unpacking what I mean by that, what I'd like to convince you of is that the population does a lot of the computation that makes social heuristics work. So at the individual level, social heuristics ignore a lot of information and they do better often by through that ignorance. And they use simple rules to integrate the information they have. And a lot of people, especially at Max Planck have studied the details of the design of such heuristics. At the population level, social heuristics depend upon social dynamics that integrate information across different agents using perhaps different social heuristics. And this integration lets the population do things, compute things that individuals cannot. Like for example, can leverage division of labor and comparative advantage and discover fantastic things that individuals lack the resources to be able to do. The population really does compute stuff. And understanding the design of social heuristics I want to convince you depends upon understanding the design of the population as well. And not only its design, of course, but its dynamic design, how it responds to changes in the frequencies of behavior and so on. And of course, this is not really a new point. It's part and parcel of the study of social heuristics for everyone who really does it. But I want to argue that a lot of our methods and our concepts about how we approach and analyze social heuristics have not yet fully embodied this realization. I have to say that Herbert Simon, of course, thought of this before the rest of us. Here's one of Simon's famous quotes that human beings viewed as behaving systems are quite simple. The apparent complexity of our behavior over time is largely a reflection of the complexity of the environment in which we find ourselves. Here he's talking about this fact that the strategies people themselves use are often quite simple and often quite myopic. However, the integration of them in a complex social environment means they can do powerful things for people to serve human interests. But it's not that the complexity of design is embodied in individual people. It's embodied in the social environment of the complexity. And I'm merely unpacking this point and trying to develop it further. So let me try to do that first by being a little bit specific about definitions. Definitions are necessary. Everybody has to have one, but none of them are perfect. So I want to take a working definition of heuristic, which will be familiar to this audience, that a heuristic is a behavioral strategy that ignores some information in order to do better. It does not ignore the information because it has to or it must, but rather by ignoring the information that can actually outcompete strategies that use more information. And there's been a successful research tradition of doing that. Social heuristics are a special case of this. They also ignore some information in order to do better. But what they have added to them is that their success depends upon behavior of other agents. Now there's not a static environment to be adapted to, but you have to play against everybody else in a game of life in which there's imperfect information, asymmetries and information. And so getting the social environment right is now critical. In a broader sense, what I'm talking about is that social heuristics always have feedback, and often quite powerful feedback. So let's take the quintessential case of feedback, which is audio feedback, now basically omnipresent in popular music, which almost always has some dissonance added to it through a feedback mechanism, or at least how that was originally generated back in the lo-fi days. Audio feedback is a case where the sound coming out of a speaker can enter a microphone and then be amplified through the loop so that the microphone re-receives it again. And if it receives it at a louder pitch, the volume will escalate, and we will get an unpleasant screeching experience. This is used in heavy metal music, but simply it can be created simply by bringing a guitar, pick up close to a speaker. You can generate as much feedback as you like. These days there are fancy devices, like pictured on this slide, which are designed to create custom feedback. It can be wet or dry, moderate, normal or rampant. Feedback is managed and used to create pleasing auditory, I don't want to say illusions, but auditory artwork. Feedback is a powerful feature of many natural systems. It isn't just used to sell music. All simple heuristics, of course, may generate feedback, and so understanding them may depend upon understanding those feedbacks. The recognition heuristic, for example, which has been previously studied at the MPI, can generate powerful feedback if people base choices, for example, where they invest their money or where they live, based upon things they recognize that will feedback and lead others to recognize those things more. And therefore the heuristic can create the environment that makes it successful. But social heuristics, I want to argue, nearly always generate quite strong feedback. And that's because their payoffs, what determines their success, is frequency dependent, meaning it depends upon the mix of heuristics in the population. And there are some heuristics that play well with themselves, and so they do better when they increase in frequency. There are other heuristics that don't play well with others, and they do best when they're at low frequency. But in real populations, any particular heuristic will have payoffs that are quite frequency dependent. The information it needs to function is created by the behavior of others, and therefore there's a strong feedback effect always. Those feedbacks can be positive or negative or other categories of feedback, but they're always there. So let's think to the classic, successful simple heuristics, something like take the best, abbreviated TTB on this slide. This is as if the success of take the best depended upon how many people use take the best. And in a specific sense, that's not true in the context in which it's been studied. But of course, there have been people recently who've been studying Q-order learning for take the best, and that's a case where the more people who use take the best, the more people will be studying the proper Q-order, and it may actually be frequency dependent. So perhaps heuristics we haven't previously thought of as social really are, because some of the parameters that need to be tuned in them can be tuned socially by populations. So I've finally gotten to the actual outline of the talk. I'm going to go over three points here, three specific points that I want to pluck out of this point that the population does a lot of the computation and that therefore changes the way we should conceptualize social heuristics and that they live in dynamic social environments. The first of these, that simple social heuristics generate many different equilibria. That is, there's path dependency, initial conditions matter, and this makes it difficult to predict exactly which simple heuristics we should find in populations because many different configurations can be self-reinforcing. Second, many, many heuristics can coexist in a population. There may not be any unique heuristic we should expect, like for example, for Tad and Lyons. If you don't find it, does that mean there's no reciprocity? I think not. And then third, to reinforce the second point, not only can many heuristics exist in populations, they probably should. There are strong reasons I think we should expect coexistence. And that is because sometimes coexistence is better for the population. That is, diversity can enhance the functioning at the population level. So from these three points at the very end, I want to extract some lessons and leads that can help us achieve a more satisfactory empirical and theoretical study of these social heuristics. Okay, first point. Let me ease into the description of many equilibria by talking about a classic dissertation by a Norwegian from 1921, pictured in the lower right is Torley Shuler of Ebbe. His dissertation in 1921 on Dallas domesticus was a fairly amazing dissertation actually by modern animal behavior standards. He created the modern study of social behavior in the domestic fowl. And in this dissertation, what Torley did was did a number of semi-controlled experiments with largely what we would call now free-ranging chickens, and studied their pecking order. That is the minor aggressive actions they take towards one another that over time establish a rigid dominance hierarchy, which determines access to food. At the time, it wasn't well understood that this was quite common in the animal world, although the details turned out to be very diverse. So this became a fairly famous dissertation in short order. One of the neatest things about this dissertation study is that Torley did experiments where he separated, he would let the chickens interact for a while, studying the dominance hierarchies that formed, and then he would separate them and isolate them in cages for a couple of weeks and then put them back together. And he discovered that was sufficient for the chickens to forget one another and forget their dominance hierarchy, forget the pecking order. And then they would reform a new one, which he would study. This was not always the same pecking order. It was correlated with the previous one, but the tiny differences in events and agonistic interactions would stabilize the population of chickens on a new dominance hierarchy which had incredible material consequences for them because they competed for food according to this dominance hierarchy. So this is one of my favorite empirical examples of multiple equilibria in a system where you would not anticipate in the first place that it should exist. It is still not, I have to say, completely understood how chickens learn dominance hierarchies. It used to be an active area of research in animal behavior, but it has become unglamorous somehow. But it is an interesting case where there is some low hanging fruit that may recede from our grasp or maybe not. But certainly the chickens are using simple rules and yet the population ends up stabilizing on a pattern. Let me broaden this back out away from chickens for a moment. Feedback tends to generate sensitivity to initial conditions. It doesn't always. It depends upon the details, but it can quite often. So the water wheels on the right hand column of this slide is a famous example of a kind of feedback that generates sensitivity to initial conditions. These are Lorentz water wheels. Lorentz, often regarded as the father of chaos theory, modern deterministic chaos theory, you can construct these water wheels yourself with some cups that you poke holes in the bottom and then put on a wheel, a mounted wheel. All that is required is that there is water pouring out of a single source on the top. The water fills up individual cups. It is leaking out of the bottom of the cup at a slower rate into other cups. What this water wheel ends up being is a random number generator. It oscillates and changes direction seemingly at random. And yet it is entirely deterministic depending upon the initial conditions. It is a simple, truly chaotic oscillator. An example of the Lorentz oscillator, actually. These sort of things can occur in natural systems in quite unexpected cases. You can buy these Lorentz water wheels for your garden, actually, if you go online. They're quite fun to stare at. You can lose a lot of time doing it. Here, I just want you to think of them as an anchoring in your mind of this idea of sensitivity to initial conditions. How much water is in each cup, the angle or angular momentum the wheel has when you turn the fountain on will generate a distinct sequence of random numbers that are produced by the random time series of angular momentum. Cooperation in multiple contexts is an example that I want to unpack over the next several slides that illustrates the sensitivity to initial conditions in a different way. That is, even in quite simple contexts, relative to, say, the Lorentz water wheel, it is easy to get multiple equilibria. Even when some of those equilibria, some of those steady states in the population, that is, configurations of social heuristics that come to be stable in a population, even when some of those configurations are bad for the population, they can be stable. And this is a powerful point that, of course, is not new. It's as old as game theory, even older. But it's often overlooked because, I think, partly the history we're looking for some sort of optimal design. And one of the themes I want to weave through this talk is the processes that create social heuristics in populations are themselves, in a sense, satisfying. There are dynamics to it that aren't like optimization. We need some other kind of metaphor to understand it. So think of it this way. In the context of cooperation in multiple contexts, what I want to illustrate to you in the next couple slides, repeat interaction itself leads to the generation of a large number of alternative stable states in the population. Repeat interaction does that in conventions. And conventions are self-reinforcing. And it doesn't often matter if the convention is stupid. You're forced to do it, even if you're a savvy agent, because otherwise you get punished by deviating from the convention. And I want to give you an analytical example of this. The consequence is that this impedes prediction in empirical context, because it's not clear what to predict. It's one of those cases where if a game theory model predicts everything, all right, so let me unpack this. This is a story that I've abstracted away from a paper by Robert Boyd published in 1992. This paper is hardly ever read because it was in an edited volume. So partly I'm just trying to advertise it for that reason. It's a unique point. I don't think anyone else has made it mathematically in the context of studying the evolution of cooperation. What I want you to imagine is that there are two contexts in which you could have a personal cost to help someone in your community. So let's think of the standard iterated prisoner's dilemma context. Individuals are sorted into pairs in the population. Each individual in each pair has the opportunity at a cost of themselves to create a benefit for the other person, and vice versa. There are some contexts in which it is mutually beneficial to cooperate. That is, as long as both individuals are cooperating, the pair as a whole are off, and this is the classic case of thinking of a prisoner's dilemma. So let's think of a context label number one in mutual cooperation is beneficial. This could be something like taking turns, taking one another to the airport of dilemma that academics routinely have. In this case it costs you perhaps quite little. It does cost you something to drive your colleague to the airport. Your colleague gets a quite large benefit from it, and over time if you're reciprocal, you can achieve a very significant mutualism by taking turns helping one another to the airport. Over there are other contexts in which it is so costly to help the other person that it's not in your self-interest to do it, not only as well as never in your self-interest to do this, but it's not even in your long run self-interest to do this. If it is more costly for both individuals to provide the good for the other than it helps the other, then over the long term you cannot get ahead. You're basically the pair is destroying themselves over time by quote-unquote cooperating in such contexts. Search your memories and think, are there cases where you help people even when the benefit you generate for the other person is smaller than how much it costs yourself? I think it's probably true. The model I want to explain to you here offers a potential explanation for why we might do that. There are many other potential explanations, but potential explanation is they're reputational consequences. As long as people care about your cooperating in any context then they can enforce your cooperation in any context even when it's in a sense and I use this word with irony for this audience, irrational to do so in context number two. However, there is an evolutionary or social rationality to doing so because of the power of convention. So the diagram at the bottom has just been to illustrate this model. There are two contexts, numbered one and two. In context one, the benefit, B sub one is greater than the cost, C sub one is still a prisoner's dilemma but over time a pair of individuals exchanging benefits will do better. In context two mutualism cooperation is not favored because the benefit is now smaller than the cost. The pair continues to another round of an opportunity to cooperate in these contexts with probability W. This is just a standard choice of variable labels in the iterated prisoner's dilemma literature. I'm not going to do too much math in public here because I think that never goes well, but I wanted to be a little bit specific so you get a feel for what's going on. So let's define some strategies for the example that I want to pull out of this. Let's think of a set of strategies which are list the context in which they cooperate and expect the other individual to cooperate and they will withhold cooperation if an individual defects in either of the context, any of the context in which they themselves cooperate. So strategy one and two at the top bullet here, specify cooperate in both contexts one and two, again you withhold cooperation in both if your partner defects in either. In other words, there's a reputational heuristic here which links reciprocity that is the return benefit in the future to the individual having cooperated in both contexts in the past, not just one. You could also have a strategy only one cooperating context number one, but not in number two, and you withhold cooperation number one and only number one if your partner defects in number one. This would be the delivered with irony rational strategy because cooperating in context number two remember does not help the pair in the long run. It is self-destructive. It diminishes the group's benefits in the long run. Both of these strategies can be evolutionarily stable and this is true even though number one always has a higher payoff once it's common. The population is better off if the strategy of only cooperating in one and withdrawing cooperation if the other individual doesn't cooperate in one. If that's the common strategy, the whole population is better off. Nevertheless, both of these can be evolutionarily stable. The intuition behind this is one and two can be stable as long as number two is it too costly. If the self-destructive aspect of cooperating in context two is really big, then it'll drive the population out and that strategy out and only cooperating in one can come in. But there is a quite easy to satisfy condition which I reported at the bottom of this slide that you can prove analytically that both of these strategies can be stable under a wide range of conditions as long as the cost C sub two is not too big relative to the some weighted sum of the benefits of cooperation in both contexts. The reason this happens is because of convention. It doesn't matter if the population would be better off if everybody only cooperated in context one. You aren't the population. You don't have control of the population. You only have control of your own behavior. If you could get everybody to switch, that would be great and sometimes societies figure out ways to do that. But in these distributed sorts of situations like in this model you don't have that power in convention rules and this creates a large number actually of self-reinforcing stable states for the population and this will lead history to matter quite a lot. I've extracted this away from the general analysis in Rob Boyd's paper which actually studies a quite large expansive set of possible strategies like this and he shows that it's quite easy to structure a game such that there are thousands and thousands of equilibria and many of them will not be socially efficient. Models like this lead me to think that perhaps we should be a little bit more hesitant about what we predict animals should be doing. So with that let me bring up another animal example. These are elephant seals and I think they provide a nice naturalistic case study in how heuristics coexist or can coexist in natural populations in their social ecological environments. Elephant seals have this fairly spectacular mating system in which large males defend entire beaches herems they're sometimes called not a metaphor that I like but I'll use it anyway. Herems of cows which is what they're called in seals and against rival males and therefore they monopolize the matings and father a bunch of children. However in elephant seal populations it was learned decades ago now there are a large number of smaller males like the one who is foolishly trying to fight this larger bull who are sometimes called sneaker males because they sneak up and steal matings from the harem in a sense. They hide on the periphery of the harem and when the slow large bull male who has driven off his competitors isn't looking or taking a nap or mating with some other female the sneaker males run in leave some sperm behind and father some kids and it turns out that these two strategies interestingly further study reveals encourage one another each creates a social environment in which the other does well so you think about the limiting cases if every male is trying to be the biggest bull on the beach then the first sneaker male is going to do well because they won't die so you can do quite well and harvest up some descendants that way and the other extreme if every male were a sneaker the first male who tried to drive them away could get more matings that way instead of trying to compete in the mad dash to sneak up and steal copulations from no one who's guarding the harems these sorts of competitive systems arise quite easily I think and they have interesting dynamics but the key point is that we should expect coexistence of different kinds of social heuristics different ways of achieving ends rather than looking for any particular uniquely optimal social heuristic again this is not a new point lots of people this audience will already have this in the front of their minds but it's worth saying over and over again because there are historical reasons that scholarship tends to look for uniquely optimal solutions to all kinds of behavioral problems and I want to discourage that actually I want to search for solutions but not uniquely optimal ones perhaps satisfying ones if you will so here are the points I'd like to make in this section in the next couple of slides mixes are often superior so we should expect them to be around they can do mixes can do things for populations that single social heuristics maybe cannot in these mixes there are a lot of strategies that frequently will behave the same when we go out and we measure behavior it will look like every individual has the same heuristic but they actually don't and that's because together the combinations have encouraged the social environment in which the differences aren't expressed and this may this will generate all kinds of interesting dynamics I think so I'm going to unpack that in a detailed example so hang on the example I'm going to use is the evolution of direct reciprocity otherwise known as the study of tit for tat coming back and picking on our lions again so I think the study of tit for tat really should begin with hamarabi hamarabi's code specified a large manner almost 300 specific punishments for different crimes but the guiding principle is an eye for an eye a tooth for a tooth that is a crime should be punished with the harm that was caused by the crime of course famously hamarabi adjusted these punishments in line with the social class of the victim punishments were much worse if a thief stole from a rich person then from a poor person but the general principle of tit for tat exists in hamarabi's code and this leads ties into the literature on tit for tat it's been this famous result usually attached to Robert Axelrod's fantastic book on the evolution of cooperation the tit for tat is an evolutionarily stable solution to the problem of sustaining reciprocity that is if you're nice to begin with that is you give strangers the benefit of the doubt but then you're retaliatory in a quite trigger happy way that is if you have no tolerance for defection but immediately respond in kind to defection by defecting then a population of cooperators can persist this is a false result and I say this with no qualification this is a result that nobody should cite in the present day tit for tat is not evolutionarily stable in any evolutionary model this is an old result it is not due to me but I would like to explain it because it is hardly ever discussed and I think this is a shame because it's one of the things that leads to say looking for tit for tat alliance and then being shocked when we don't find it for example I'm going to unpack this in the next several slides the strategy always cooperator all-see which is an unconditional nice guy cooperator all-see is the golden rule unlike hamarabi's code which is definitely not the golden rule the golden rule is do unto others as you would have them do unto you that doesn't mean retaliate necessarily it means being nice so think of all-see as the more biblical kind of strategy and in a population of tit for tat it will have the same fitness as tit for tat why because there are no defectors around in a place where they fleeced tit for tat actually creates a social environment in which more tolerant strategies can evolve and we should expect them to be there I think because they pay smaller costs and that's the argument I wanted to develop in the next few slides and so empirically many many strategies end up behaving the same in the absence of non cooperators if there are policemen social heuristics out there they create an environment in which social loafers who are nice can persist a little challenge of discovering social heuristics more difficult but these things can have effects on the social dynamics even if the events that differentiate these strategies are quite rare so I want to talk about that in the context of mistakes in the iterated prisoners along the game so let's extract away for a moment let me give you the cartoon version of the study of tit for tat as a strategy that can sustain cooperation in the simplest model and in fact the original one that was published by Hamilton we think of only two strategies there's all D which means always defect on the far left of some state of the populations can be in on the far left the entire population is all D on the far right the entire population is tit for tat the strategy that starts by cooperating but then copies the move the previous move of its partner that makes it perfectly retaliatory a trigger happy strategy and in the classic game theoretic analysis of this there is some bifurcation point in the middle an unstable internal equilibrium in this space in the population so once tit for tat is common enough evolution dynamics will make it even more common and remove all D from the population making it seem like a potential solution to this problem now of course we have to get past that unstable equilibrium they are internally in the population but there are ways to do that which are not so far fetched so so far so good it looks like tit for tat is an evolutionarily stable strategy in response to all D and could sustain reciprocity however there's a lot more to this game let's just add one more strategy to it we're going to add all C this turns this line into a triangle some of you are familiar with these kinds of plots sometimes called very centric coordinate plots or mixture plots or let's just call it a triangle this triangle plot it has a coordinate system such that if you're in any corner the population is entirely the strategy labeled on that corner so if you're in the far right corner the population is all tit for tat if you're at the top the population is all C if you're at the red dot in the middle the population is a third of each strategy what this sort of diagram lets us do is plot the evolutionary dynamics in this game in a way that doesn't bias our attention to favor particular strategy so I want to show use this representation to show you what happens when you add all C to the classic game here I'm going to focus in on the far right in the moment the lines in here show you the evolution dynamics if you pluck a population down at a point in this space you can say where it goes and that's what the lines are tracing out the evolution dynamics of the population over time so if you're in the far right say you're in the bottom right corner and the population is all tit for tat this entire margin which I've now shaded in orange along the lower right hand side is a bunch of mixes of tit for tat and all C and all of these mixes are evolutionarily stable that is if there's any mutation or migration which introduces the social heuristic always co-operate into the population it will behave exactly like tit for tat because there are no defectors no disadvantage at all those processes can eventually drift you far enough up such that all D actually invades unwinding the whole thing however it may not the model doesn't say that is to say what this model predicts already you'd have to say well it predicts a bunch of mixes stable mixes of tit for tat and all C and we could wave our hands rapidly about other forces like mutation and migration and drift you might actually destabilize those mixes by eventually removing enough tit for tat from the population such that all D comes in you don't have enough police officers eventually and then all C is like food for all D and it just feeds on it and then the population goes to the lower left however we can make it even more problematic take us beyond the point that this model actually predicts not tit for tat but some mix of tit for tat and a bunch of other nice strategies but we can think of a bunch of strategies like tit for two tat, tit for ten tat suspicious tit for tat all kinds of strategies which as long as they're nice will be sustained in this population at some frequency and that makes the empirical challenge fairly difficult actually that is that the strategies comprise a social environment in which they're co-adapted to one another in a sense but the evolution dynamics over time might be unpredictable so let me problematize this a bit by talking about errors too I think you have to train your attention to errors in the study of the iterative prisoners dilemma because in the absence of errors you can get thousands and thousands of equilibrium and almost anything can be stable and all sorts of forces matter that aren't in the model and that's not a sort of situation I want to be in as a theorist of telling people that theory predicts anything adding errors helps a lot actually it helps you reveal differences between strategies that otherwise look identical and this can help us in the task of prediction as well in particular let's talk about implementation errors as Mahatma Gandhi intuited an eye for an eye can be a very bad legal kind of precept because it makes the whole world blind over time why would make the whole world blind even if everybody is nice there will be mistakes you will think the other person defected or sometimes you will defect when you didn't mean to how could that happen well let's focus on a class of errors called implementation errors and implementation errors when you want to cooperate but you accidentally defect this would be something like you meant to take your colleague to the airport but you overslept if you alarm clock had gone off you would have taken your colleague to the airport that kind of error seems quite likely because cooperation is an ordered state in the universe the opposite sort of error that is cooperating when you meant to defect seems much less likely so I think we can safely ignore that at least on the first pass in the analysis as a consequence of this tit for tat gets into feuds it has no way of knowing that the other person made a mistake and didn't mean to defect it merely copies their behavior and therefore as a sociographic it's very sensitive to errors it's intolerance that makes it look successful in the absence of errors is this Achilles heel in the presence of errors so even if errors are rare they can be extremely important in the aggregate dynamics in the population and favor other heuristics so let me reanalyze the tit for tat all the all see game to pull this out just by adding a little bit of implementation error the consequence of this is that social environments are really complex even though social heuristics may be simple but we have to understand the complexity of the environment to understand how the strategies can be simple that is how their simple design fits the social environment they're in then a sub point error can actually be adaptive it's not something that we can just sort of hope averages out of the analysis strategies social heuristics may be adapted to produce errors because it helps differentiate them from other strategies one of my chapters from my dissertation over a decade ago now made that argument mathematically here is the tit for tat all the all see game now replotted the dynamics replotted in this triangle graph with a 5% chance of an implementation error on every move that which means if an individual meant to cooperate there's a 5% chance that they defect instead the exact number 5% is not important here what's important is that it completely changes the game now there isn't there aren't any stable mixes of all see in tit for tat tit for tat gets into fights and over time it gets removed from the population if you place the population of playing god in the lower right of this figure you would crawl up the right hand side to an all see world because all see isn't getting into feuds all see is very forgiving right so in a world of cooperators all see actually does better now than tit for tat does even if errors are rare that process could be quite slow though but once there are a bunch of all see in the population the population is going to quickly move towards the all D state internally the dynamics can be very complicated and take a long time with these sort of swirls and transient states so now imagine going out in the world say studying prides of lions and looking for a particular strategy what should you expect well I would expect some mix of things perhaps I wouldn't be too surprised by diversity which is a point I'm going to echo over and over again to get here and that isn't I don't think what lions are doing is is well represented by this game necessarily but the lesson this game delivers I think it's directly applicable to the study of those sorts of empirical contexts okay we're getting along here we're almost done I promise third and final point not only can many can many heuristics exist I think they should co-exist that's because they can often be complimentary in a sense the story I've just told is one in which different strategies exploit one another or they compete or the evolution of all see undermines tit for tat or some such and that does happen but I think there are other cases where the structure of the population harvest the diversity recruits the diversity among social heuristics do things that individual heuristics cannot and as a result the computational ability of the population if it were actually helps the social heuristics achieve something greater than the sum of their parts and so we should be looking for these co-adapted complexes of social heuristics populations I think where the skills and abilities of different individuals since they're different require different social heuristics at the individual level and therefore at the population level we can do things like achieve complementarity recruit the benefits of division of labor and comparative advantage the example I'd like to use to explore this is not doing the dishes in the upper right or ants building a bridge although both of these things illustrate the general principle division of labor about who's doing exactly what in doing the dishes or taking turns or some ants make a bridge while others crawl over all this requires diversity and differentiation of behavior which benefits the group I want to use social learning as an example get away from cooperation a little bit as you'll see there's cooperation story to be told here too okay I'm not going to spend a lot of time explaining the structure of this model I want to give you a cartoon sketch of it that's because the chapter I recommended as a companion reading to this lecture has this in painful detail so you can go look at the details there the citation to that is at the bottom of this slide let me start by talking about the problem that individuals have to solve the problem that individuals have to solve in this model which is meant to help us understand the logic of when simple heuristics for copying behavior rather than doing your own research and development could be favored is an environmental challenge where there are a range of behaviors here called phenotypes because most of the time I'm a biologist and we call behavior phenotype there are a very large number in fact an infinite number of different alternative phenotypes but at any particular point in time there's only one that's optimal they're equally non-optimal, equally bad so you might as well just say there's one good one and all the rest are bad and the red time trend on this plot with time on the horizontal axis and phenotype on the vertical is meant to represent that it is changes through time so for some number of generations the same behavior is optimal but then there's a switch in the environment and in this model that's purely exogenous although there are cool models where it's not the first model we want to do so let's just stick with the exogeneity story and it's changing over time so as an individual you're born in the middle of this time stream you've got to figure out what to do we're going to study a case where behavior has to be learned you can learn it either by studying the environment you were born into yourself that is trying to innovate or you can copy behavior from the previous generation from the parents who are around but your own parents or somebody else's parents and the question is when are these different heuristics favored let me do let me extend this cartoon metaphor a bit and think about this population here's a population of naive individuals each circle representing an individual they have no phenotype yet because they're born naive but they do have a genetically inherited learning strategy this is a point where I meant to insert I think all these points apply whether or not the social heuristics the learning heuristics are genetically in fact I think it's an interesting point about social heuristics that they're self-modifying or can be self-modifying but it's easier to think about the case where we have the transmission system separated so they're empty of color in these circles because they're born naive let's fill in some of them with green which is to indicate that they have the optimal behavior for the current environment and all the others have a non-optimal behavior not only for the current environment but forever we're assuming they're an infinite number of behavioral states so as soon as the environment switches none of the past behavior is optimal that's a mathematical convenience for analysis you can relax that assumption and get the same qualitative results from these kinds of models so let's put that population up at the top and think of them now as parents however they've learned their strategies however some have become green and some have not they're going to have offspring which is a new level of empty circles now at the bottom learning is active in this model so some individuals now indicated by the red borders are individual learners that is their social heuristic and some are social learners who copy individual learners pay a cost to do some research and development and then they have a chance but only a chance social learners copy when they copy they can't preferentially copy individuals who have the optimal behavior because they don't know what the optimal behavior is they can only copy at random in this so this is one of the simplest in fact I would argue the simplest social learning heuristic that anyone has ever written down copy at random there are much better ones as my chapter goes into so some of the individual learners will turn green because they successfully innovate some social learners like say the one there will be lucky enough and randomly select an individual who's green remember they can't see the green they're going to copy a behavior but they don't know if it's good or not they only know it's a different behavior and they become green themselves sometimes in this model because the environment is changing there's a perturbation which makes all of the parents white and then social learners do very badly immediately after a perturbation and that's the risk involved and why the information economics of these different social heuristics is quite different individual learning is a very low risk strategy but high cost and social learning is a higher risk lower cost strategy so they're different in some interesting ways to generate the dynamics I'm going to spell out for you so making a quick point that's spelled out much better in the chapter because I had a lot more space if we focus on only these two heuristics learn individually and learn socially there's no actual equilibrium because the environment keeps changing and right after a change in the environment as shown in the graphs in the bottom the blue time trend is showing you the frequency of innovation that is individual learning as a social heuristic in the population just after a change in the environment it increases rapidly because social learning is doing poorly so there's this brief period after every switch in the environment in which innovation is favored once most of the population is doing well which is indicated by the red kind of shark fins in the same graph labeled behavior that is the frequency of currently optimal behavior in the population once that reaches a high enough level social learning is way better than innovation because it doesn't cost you as much and whoever you copy you're going to get the right behavior so there's a time trend to this but in the long run there is an attractor a kind of average level of innovation which is adaptive in the long run and the population floats around that what you don't get is either pure type you can push the parameter conditions to be very extreme in this model such that there is only innovation at the limit but even that is a fairly rare event what this sort of model predicts is actually a combination of the two and it is the combination of the two of innovation and social learning in these models that lets the population do things that individuals can't do what is that? it lets them accumulate innovations across generations it lets the population build complex behavior that no individual can invent in their own lifetime so I always tell the story in talks but that's because it's so good Isaac Newton famously said that if he had seen farther than other men is because he stood on the shoulders of giants and we should amend that to say that if he had seen farther than the other men is because he was a midget standing on the shoulders of a vast mountain of midgets everybody being just as short as he was we have all benefited from the hundreds and thousands of generations of people before us and the tiny innovations and accidental sort of discoveries that they have made that have been transmitted to us through copying over over human time no individual human could invent calculus but Sir Isaac Newton could because well he had all the other stuff in mathematics at that point and a social environment that allowed him the free time to do it and so on so it's not just that this model produces a mix almost always neither strategy neither sociographic innovation nor social learning is best in any real sense but also that whatever the individual incentives for individuals to adopt one or the other the population does things over the long run then individuals can't do because of the combination because of the mix of the two does that make some sense alright so next couple of slides I want to go over quickly just to draw your attention to some deeper points that are developed better in the McElver's Welling and Pasola chapter in the edited volume we predict a mix as you allow more and more social learning heuristics in this model the mix depends upon things for example we could make the environmental variation spatial such that in different places different optimal phenotypes are favored and individuals can migrate between different places in this case there are different heuristics that are favored and not only that but you get diversity across individuals that is selection will favor a lot of diversity where some individuals are always using one heuristic and other individuals are always using another that is individuals embody different heuristics under temporal variation however selection favors individuals randomly using a portfolio heuristics at different times temporal variation means that the environmental state is changing through time like in the original story I told you in that case selection actually favors individuals somewhat randomizing their strategy use for reasons again I don't have time to develop it here it's well developed in the chapter lots of other configurations are around some combinations of strategies within or between individuals actually again help the population they help innovation across generations and make the rate of adaptive behavior increase higher and part of the design problem from our perspective if you take our job as to eventually apply these insights to designing human institutions is coming up with structures of populations that recruit individual heuristics and the diversity of them so that it creates these positive population level effects and I think that's a very hard problem but we approach it seriously by recognizing and studying in detail cases like this last point again this is developed well in the chapter I won't have time to do so here but it echoes the point I made with the iterated prisoners dilemma lots of different strategies often have nearly the same payoff and can coexist over time that's just as true here in social learning you can add a bunch of social heuristics to these models and get a bunch of interesting mixes lots of transient states where a large number of quite different learning strategies are around but they all result in the same behavior and so not only will it be hard to predict that there should be only one heuristic in the population it will be hard to even recognize them as being different unless we have exactly the right level of understanding perhaps we have to study for example information search to differentiate these strategies and not just choice which is a point of course that isn't going to be new to this audience but it's a point that still needs to be stressed again so at the bottom I highlight this hairline differences which I mean is in the time trends in these models there are typically many generations under which once the social behavior has become common because of the action of some social heuristic lots of social heuristics do well almost any social heuristic will do well and therefore tiny payoff differences are all that you see and our empirical apparatus for studying these things may be quite frustrated this is echoed by a fairly high profile set of publications from Luke Rendell in Kevin Layland's lab but Rendell and Layland and their colleagues did was organize one of these tournaments to study the evolution of social learning this is modeled after Robert Axelrod's famous tournament to study the evolution of cooperation which made TIP for TAP famous and in this tournament just like in Axelrod's a bunch of independent researchers and teams submitted computer programs which embody social learning heuristics and then they created a big melee in which the teams competed and they studied the design features of them relative to one another and how that affected their success in the social environment embodied by the heuristics that were submitted so one of the things I like about these tournaments is they reinforce the point that social heuristics do well given a particular social environment and the dynamics it generates you can't conclude from these tournaments that the winning strategy is the best in the real world and of course Layland and colleagues don't think can be pulled out of the analysis nevertheless for example I'm just showing you a couple of plots here the payoff differences among the strategies even though behaviorally these strategies are really diverse they use information in many many different ways there are hundreds of submissions on the left this is a plot of the mean score think of this as aggregate payoff against the rank in the tournament the bigger part of the graph here shows only the first 10 and you'll see after the first two they're all basically the same this is especially true in the full view that's in the upper right of the left hand plot where you see all the strategies ranked there's a very little differentiation in neighborhoods that is there are hairline differences between many of these strategies and so many other forces that operate in the real world can lead to a persistence of diversity in the details of how information is used and then on the right what you're looking at is on the horizontal is one particular dimension of behavioral difference one of the most important in the tournament this is the average number of rounds between learning moves learning here means innovation, individual learning so on the far right on the horizontal axis there what you're seeing is our strategies that did a lot of social learning that is they use individual learning infrequently and on the left you're seeing strategies that learn for themselves quite a lot now there's certainly a correlation here strategies that learn for themselves a lot are going to be worse on average but in the orange band I've highlighted at the top what you see is across the entire range represented tournament of how much individual learning is done by a strategy you can get comparable payoffs in the same kind of range and that is a factor of how the social environment the information environment is being generated endogenously by the action of these strategies and even a quite crummy strategy you can do quite well as long as there are savvy strategies in the population creating a benign information environment for them and that's a general feature of studying social learning I mentioned Axelrod's famous tournament was started it I want to bring that up again just very quickly to show you that that's not just true of social learning this was also true in Axelrod's tournament although this is not usually what people report about it the left here is just the book cover this is the definition of the book in which this is reported on the right is a time trend you can go back and you can simulate Axelrod's tournament as many times as you like and what always happens is there are a bunch of strategies that do just about equally well and they co-evolve always a cloud and why is it because they're nice strategies and they do well together and Axelrod recognized this and talked about it in the book but it isn't the point that is often brought up because it doesn't say there are strategies which have a certain feature given that the other strategies also have that feature it's a hard thing to talk about because we don't really have vocabulary for it but it's not a strategy prediction it's a kind of social environment prediction and as we develop the language to talk about these things it'll get easier I think to understand social heuristics okay let me try to close this up now you've indulged this remote talk long enough bring it back to the problems and opportunities and the problem of Tantalus reaching for the grapes here and they're always receding from his grasp I think the lesson with many of the social heuristics and the sorts of contexts that I study at least is that if you reach for individual fruit you will be forever tantalized you will be like the people studying lions looking for tat, you will not find it you may find it in some individuals but not others what we should be looking for is something completely different another way, Tantalus shouldn't be reaching for a fruit he should be reaching for the whole tree stop trying to grab the fruit and grab the branches of the tree and pull yourself up and get the fruit later you have to sort of think about this as a metaproblem about a population of heuristics and not an individual sort of problem of identifying a particular heuristic so let's get away from the Tantalus metaphor because of course it has its limits like all metaphors do and come back to the actual substance a population does computation so you don't have to social heuristics, as Herbert Simon said, are quite simple but that's because they exist in complex social environments that recruit their simple capacities in adaptive ways they fit together this is a point that is often made heuristics always fit their environment when the environment is dynamic and highly frequency dependent the analytical problem becomes quite different in this sector in particular you get many equilibria especially in cases of repeat social interaction as in all human societies there will be many thousands of interactions with the same individuals in short periods of time in human communities and this must have an effect on the design of social heuristics social heuristics can coexist and not only that, but they probably do we should expect them to quite often let me try to boil this down to some lessons now as well we should expect history to matter and this ties in to the questions of multiple equilibria we should expect that the initial conditions of how things started or past social environments for example may have powerful effects on present social environments and the design of social heuristics even those of those social environments are gone why, because they provided initial conditions they selected for a configuration of heuristics in a past context that made a certain equilibrium reachable when the social institutions changed and so differences among societies or groups will not be explicable solely in terms of the design of present social environments or even their dynamics but will depend upon past states and this is frustrating, but hey human behavior depends upon the study of history that's just the truth of it I'm just re-emphasizing a point that's been made by many others in that regard second we should expect and measure diversity looking for tit for cat in lions look for sets of strategies which may coexist or the reason that some lions are more vigorous in defense maybe because they're differently abled than other lions maybe there's division of labor in other ways we don't know, maybe we've got the payoffs all wrong, maybe we haven't represented the game right but even if we have represented it right, we shouldn't expect every individual line to be tit for cat and we shouldn't expect that immediate retaliation is necessarily the best strategy because it's easy to get equilibria without that third we should expect integrated and antagonistic suites of heuristics that is we should be looking for cases where the diversity among social heuristics in a population somehow does something for the population that individuals can't do for themselves that there's in a sense a heuristic function of suites of social heuristics as in the case of the complementarity of different learning algorithms or heuristics as I explained earlier and I think that's something that's actually quite understudied and there's a lot of room there to think about that, how diversity enhances function every one of the social sciences and the biological sciences and the physical sciences has examples of systems to behave that way but they aren't knitted together very well across disciplines into a study I know that complexity folks do think about that a lot but they're still segregated out from many of us unfortunately so a couple of really pointed empirical comments here and I'm just going to label these here and I'm going to show you a slide to kind of dwell on in a moment because I think it's an important point because it ties together the historical and empirical and conceptual obstacles to grabbing the fruit that I started this talk with we should be resisting highly controlled experiments I think and that may sound like blasphemy in a scientific audience so I want to unpack that and instead we should be embracing endogeneity and developing research methods which deal with these these three expectations on this slide head on what do I mean? Well, let's talk about history again for a moment I think most of the empirical and statistical methods in the social sciences especially in psychology come from Rothamstead research manner many of you will know about what Rothamstead is it is still going Rothamstead research manner was started in the early 20th century the most famous work to come out of there is probably Sir Ronald Fisher's work on research design as statistical methods this is where analysis of variance is born in this book, statistical methods of research workers and things like block designs as shown in the stained glass window which memorializes Sir Ronald Fisher's contributions there in the right of this slide what does Rothamstead do? Well, they study how different fertilizers and moisture content and crop orientations affect the growth of different crops, mainly various grasses like wheat and barley so many agricultural fields done in random block designs Rothamstead has been incredibly successful and a lot of the productivity of modern agriculture worldwide depends upon results, long-term environmental studies at Rothamstead the problem with Rothamstead is that it has polluted the model of what is good science in all of the social sciences and in much of biology and what do I mean by that? I said well look animals and their behavior are not wheat plants that are stuck where you put them random block designs and analysis of variance and full factorial treatments are great for studying the growth of wheat under fertilizer regimes they're not so good for studying human behavior and in particular let's think back to the points I tried to make in this slide in this talk when heuristics are adapted to a particular dynamic social environment the information they need to function correctly is endogenous to the behavior of the system to the way others behave in the particular mix of heuristics you can't realize that if you're controlling all the variables and randomizing them you're really endogeneity to study the phenomenon you're interested in and by creating the sort of randomized block design experiments and studying them with analysis of variance which is probably only appropriate if you use these sorts of randomization techniques we're actually preventing ourselves from seeing the phenomenon that we're supposedly interested in and again for this audience this will not be such a controversial thing to say because with this audience you're used to critiquing the sorts of standard experiments but in the study of social heuristics this is a very important thing and it's one of the reasons the other paper I recommended to read to accompany this talk my experimental paper in Proceedings to the Royal Society makes a complicated loosely controlled experiment where behavior is endogenous and all the information that's available to individuals in that experiment is endogenous to their behavior it's not highly controlled the only thing that's controlled is assignment of individuals to treatments randomized the information available to them I don't control any of that because I don't believe we can understand the fit of social heuristics to their social environments if we control the social environment the social environment is part of the endogenous process and we have to create a set of experimental and observational and empirical methods which fit the phenomenon of interest instead of taking Rothamstead as the model of successful science and applying it in a procrustian way to everything that people do all right, with that I want to thank you for your indulgence and leave you with a very meta-thought of course all of us are scientists or part-time or full-time and we study social heuristics I think one of the things we have to realize is that the study of social heuristics itself is conducted, it must be conducted using social heuristics scientists or individuals with all the same foibles and population of every other decision maker and of course we reason with social heuristics as well and this isn't a novel point other people make it, but we don't usually dwell on this and so I think it's a healthy thing to do to remember that we're embedded in environments that make our social heuristics either useful or not to what extent is the structure of the society of science co-adapted to the social heuristics or psychological foibles of individual scientists, how does that system work and this is an area of important applied application of social heuristics. I haven't worked on this myself back in the 1960s and 70s, Donald Campbell and Herbert Steinman and Carl Popper did think about these ideas, but it sort of got dropped for reasons that I never understood, so I just want to take this opportunity to plant it in the minds of the audience and hopefully someone will run with this with that, thank you for your time