 The technical person was asking whether you hear the speaker on the back If you hear fine, we try to reduce the volume a little bit further so that we try to avoid that very high pitch Thank you Good afternoon everyone So what I wanted to share with you today is a new way of looking at task allocation specifically in social insects which we at least think is starting to yield some quite interesting results and This is joint work between myself and my colleagues Julie and Gathia and Richen at Monash University in Melbourne So we do of course all know that social insects lead terribly busy lives Every colony has lots and lots and lots of different tasks to do simultaneously Say in nest tasks like for example brood care or nest excavation all that kind of stuff out of nest tasks like foraging defense and attack and whatnot And all of these tasks need to be done simultaneously and of course balanced according to the internal needs in the colony according to the external conditions of the environment and so on and so forth and what that means is that Possibly the most important task in the colony is actually a meta task which is task allocation Regulating who does what right? How many do I allocate or how many get to do foraging? How many get to do brood care? How many get to do thermal regulation and so on and so forth and Because of that central role a whole lot of research has gone into task allocation already And quite a few different pieces of the puzzle are quite well known But on the other hand we're pretty far away. I think from getting a consistent picture of everything So some things that are known in a lot of cases physiological factors play a big role not in all but in a lot of cases So for example size and age right in many species like for example weaver ants The younger ones smaller ones tend to perform in nest tasks bigger older ones tend to perform the risky out-of-nest tasks But always but that's often the case and that leads to something that's per age polyethism So as you go through their development you perform different tasks, and that's quite a slow dynamics Other factors play a role as well in neurotransmitter levels Morphological factors genetics that influences the morphology morphology and so on and so forth Then of course spatial distribution plays a role so When a task occurs over here, it's quite unlikely that others that are a kilometer away are going to engage in it quite simply And that's all encapsulated in this foraging of work model Again big influence, but then there's a whole other level and that's the level that doesn't have that slow dynamics of say age Polyethism, but it's much faster dynamics where the colony can adjust quite flexibly to the conditions of the environment And that is thought to be mostly self-organized Right and what plays a role in there are among other things are social influences. What do the others do and Learn task preferences in some sense So what's Probably the most widely used the best established model of the self-organized task allocation I mean many of you will be familiar with it Which is response threshold models, but just to make sure that we're all it's a diverse community that were all on the same page I'm just going to very quickly summarize that The basic idea is quite simply that every individual has a task related threshold and then there's a task related stimulus and The response of the individual whether or not an Indian engages in that task depends on the relation between that internal threshold and the stimulus level Basically to make it simple the higher the more the stimulus exceeds the threshold the more likely is the individual to actually respond and pick up that task Now where the magic really happens is when it comes to adapting that threshold so-called reinforced threshold models and the idea again is relatively simple The idea is that when an individual engages in the task It lowers its threshold for that particular task And that means it becomes more likely next time to pick up the same task again Conversely if it doesn't engage in the task the threshold rises and therefore it becomes less likely And that generates this positive feedback loop where Individuals that start to engage in a task engage in it even more further lowering their threshold Engaging even more and so on and so forth and it's well studied model It's well known that that can lead to specialization where basically those who started push the others out of the task That group takes over this task and another group takes over another task So this is a pretty well known and well investigated story But if we step back from that for a second What is the overall picture that's happening here the overall picture is pretty much like that on the top there See what I can work this point Yeah, like that one here The individuals don't actually really interact you've got a whole bunch of individuals acting in parallel Independently the only way that these individuals actually communicate is via the stimulus So if I engage in a task I fulfill the task partly the stimulus level obviously sinks and that can be read By other individuals and that's the only communication that goes on in that model What we do know however is from empirical studies that social interactions actually do play a role in what happens And what would what would like to model is actually a scenario like this where the group engages Collectively in a task execution because that's actually what they're doing And then from that task execution Receives some sort of reward benefit whatever you want to call it The important thing is that in many cases these rewards actually go to the group and not to the individual On the other hand think about foraging for example The food is shared at least in social insect colonies So what I bring back also goes to the others So it's a community benefit on the other hand the costs obviously I incurred to the individual if I go out and forage That's energetic cost to myself So that's what we'd like to model The other part is that we'd really like to understand how the environment interacts with the whole thing Right. What are the conditions in the environment? How does that influence that is foraging hard or not? What's the impact of that and that's also quite difficult to do it's possible But difficult to do in a special response model because you need a whole other section of a model to map How does the environment translate into that single input variable that you have which is the stimulus level? Right, so if we want to model this kind of thing what comes in very handy Is that there's actually a theory that can help us with that well established which is evolutionary game theory? May sound strange because obviously and as the name implies evolutionary game theory comes out of evolutionary biology Not so much out of behavior, but it was created or learning I was created to explain how evolution shapes behaviors that govern the interaction between animals, so that's a good starting point For those who are not familiar with it Just giving an example probably the best known example is that Hawk dove game So why do some animals engage in a conflict immediately go into full-on? Fight potentially to the death whereas others just engage in signaling behavior display behavior and then withdraw without anything happens Why do these two strategies continue to coexist wise one not just better, right? And that question was successfully answered by game theory and it celebrated a whole lot of other successes Evolution of altruism of collaboration signaling mechanisms including deceptive signaling and so on and so forth What basically happens is on some level. It's quite simple You've got a population and in each member of the population got a particular strategy how it engages in those interactions Then the interactions actually take place. They come together in interaction groups, if you will or games They play these games according to their strategies They receive a certain payoff from that and then after they've received the payoff This is an evolutionary setting the strategies propagate and basically in an evolutionary setting the more successful strategies the higher payoffs Propagate more in the next generation of course There's also a process of strategy innovation, which is essentially mutation and then the whole thing repeats over and over again Now that's an evolutionary setting But it has been translated before and that's quite easy to do into a learning setting And what that means is quite simply that the strategy adaptation there isn't genetic propagation anymore But instead a learning step takes place there Or we'll get to that in more detail in a second So this is essentially what we're doing or what we're trying to do We're taking evolutionary game theory reinterpreted as learning game theory and applying it to task allocation in social insects So make that a little bit more precise For starters just to keep things simple we set up a game Which is just a competition between two tasks two tasks compete for the attention so to speak of the colony One we have a foraging task, which we assume is a maximizing task the more food we bring back the better The other one is of quite different characteristics, which is a regulatory task think thermal regulation Where the optimum level of effort is somewhere in the middle Over-regulating as a waste of energy might even you might over cool your colony or whatnot and Regulating not enough is a serious thing. So you have to have an intermediate level of effort just right adjusted Our strategy parameter is quite simply a probability for each worker to pick up task a or b or rather task t as Task R or F here Which you can also interpret as the expected and amount of energy on average invested into that task and Then they engage in these games in these Interactions performing the task get a payoff out of that with a benefit that is shared across the colony and Of course, that's incurred to the individual and then we put that into a learning game theory setting So we've got a population here with the individuals that all have that strategy parameter X I They go into the interaction groups. They perform their interactions. They play the game in IGT terms They receive a particular payoff after that they can perform a learning step by Innovation so they basically here in this setting what that means is they can explore the strategy space by slightly varying the strategy and Comparing with their own memory the previous strategy to the current one if the previous one was better The previous one stays if the current one is better It's it gets adopted for now And that's basically a form of reinforcement learning or if you will it's stochastic who climbing actually and that Then is the next in this case not generation because we're not in an evolutionary setting That's the next step in what happens in the colony on colony lifetime The new population then goes through the same process over and over again And what we're asking what game theory or evolutionary game theory obviously asks always is in the end What is there? What are the strategies that occur in the population in something like a steady state? And here's the result So if we look at these small diagrams here what they are they're really scatter plots of strategy values against time So what you see here in this specific setting? Let's take that one is that the whole colony from having a spread of strategies over time converges on a single strategy So reinterpret what that means is everyone actually does the same thing. They're all generalists Everyone does both tasks one of them This is 0.4 roughly one of them with a higher probability or with more energy investment But everyone does pretty much the same thing Now this diagram up here is a phase space of the environmental conditions So it has two parameters here for our setting, which is B and R and be essentially tells you how Effective foraging is the higher the B value the more benefit you get out of foraging per unit of energy invested On the other hand tells you how hard regulation is how much do you need to regulate? Is it a hot day or a cold day? And what you see is that that phase space falls apart pretty clearly into three regions in one region You turn out with generalists in the colony where foraging is really easy You turn out with generalists in this setting Everyone doing the same thing behaving in the same way As foraging gets harder they actually start to specialize I should say it's not really specifically foraging That's just how we imagine it as this one task gets harder They're starting to specialize you see the colony splits into those that only do one time the task a and the ones that only do task B As it gets still harder you're getting the blue region where they just fail to coordinate completely Everyone does only one task. That's always the same task So we're only going foraging but not regulating anymore or the other way around and the whole colony dies But what's interesting here is that we haven't changed anything in the mechanisms how the adaptation happens Nothing at all only the environmental parameters and that alone drives what kind of specialization here emerges or whether it does that at all All right, so you might say This of course is just an artifact of the particular type of learning mechanism that we've put in So let's go to the other extreme of the spectrum Let's go to social learning and we're going to an extreme form of social learning because we know that social learning does play a role in insect colonies But we don't know very much about what the mechanisms are in general So we go to the extreme form of social learning, which is the normal form in EGT Which simply means the following we start with the same kind of population They meet in the same kind of interaction groups the yellow circles there receive their pay off But then what happens is strategy copying. I'm learning from someone else I'm copying someone else's strategy and the better that strategy was performing The more likely is it for me to adopt that strategy, right pretty simple assumptions for a basic social learning setting And then of course there's still the innovation space where you get to Innovation step where you get to explore the strategies and the whole process gets reiterated So what happens if we switch to the other spec end of the spectrum of learning mechanisms? Well surprisingly or maybe not pretty much the same thing Same phase space you get the green region Monomorphic population Everyone is a generalist on this side Middle space here as foraging gets harder the population splits into specialists on that task specialists on that task And then unsurprisingly as the word gets even harder to die Fail to coordinate that is Okay, so that sounds pretty good the nice thing about EGT is that It's a switch back to conventional technology good the nice thing about EGT is that's very well explored And there's a pretty rich mathematical theory behind it and one part of that theory We can use adaptive dynamics to actually do this as a theoretical prediction Everything that you've seen so far with simulations, but we can analytically Predict what's going to happen and what you see down here is the analytic prediction What you see up there as a reminder is what happens in the social learning setting and pretty much the same what happens in the individual learning setting and you see that that Qualitatively and even quantitatively quite well agrees and that's exactly what the theory should be telling us So this is only really a sanity check to some degree But the theory clearly applies and what's nice about that is the theory also tells us another thing namely these outcomes are reasonably robust Against the type of learning mechanism that we put in that's been widely explored in different settings We can vary the learning the adaptation step quite a bit and the ultimate outcome remains the same So that gives us a reason of confidence that what we're doing here is actually relevant to some degree It's not because we're choosing particular mechanisms that particular artifacts arise The learning mechanism doesn't matter that much. It does within limits. It doesn't matter that much What matters is really the structure of the environment. What kinds of benefits and costs do I have? The fact we can also ask of course the question about colony level efficiency, right? How effectively do the colonies perform now? You see the way we do this is we can theoretically assess what is the efficiency the maximum that they could achieve and then just Build the ratio to what they actually achieve in the simulations here where you have weak specialization or generalists They're almost running at optimum if you're coming to actual real specialists, sorry weak specialization or generalists If you're coming to strong specialization Learning mechanism actually suddenly matters the individuals there Perform optimally almost which means they're sitting on the Nash equilibrium Here in the social learning setting they're getting slightly off that and so it's a little bit different So here the learning mechanism actually starts to matter But what's really important to appreciate is the fact that optimality here isn't a driver in the framework It's not built into the framework not colony level efficiency. We can assess that neutrally Afterwards, you know Because it's not driving the process in any form And that also now enables us to ask another really interesting question in a new form And I think that's probably the most interesting outcome so far It enables us to ask what is the function if any How do we get lazy insects in a social insect colony? You'd all be aware that in any so in most social insect colonies shouldn't say any a large proportion of the Workers is usually inactive. They're not actually doing anything, right? It even appears that they're specializing in that being inactive The question is what does that contribute to the colony? Is there some sort of benefit from that? Is that a functional component and therefore there's evolutionary pressure, for example There's an emergency workforce Now we can ask that question in a different way because as evolutionary game theory does for evolution Would you simply ask do they simply happen as an artifact as a byproduct of the process and The answer is yes, they do you see here That's the red line in certain regions of that phase space You actually get lazy workers and that brings me to the end that Is actually something that is underpinned by game theory as applied to human settings quite frequently In anywhere not anywhere, but in certain scenarios where you have communal benefits, but individual costs generally you find that some individuals just become free riders and Thrive on the communal benefit without actually contributing to it and that's a very well understood phenomenon in evolutionary game theory Okay, that's pretty much it to wrap that up. I hope I try to run you very briefly through this different kind of framework for task allocation I think there's some interesting results already Which is? Particularly that the environment alone can drive what kind of specialization arises without changing the underlying mechanism And of course what we just looked at the laziness that can come about without any functional role Next steps a lot of theory. We're doing that But of course also what we're now really trying to do and that's not an easy task is to get empirical validation of that Thanks. Thank you