 Hello everybody. Welcome to the Active Inference Lab. This is live stream number 29.0 on September 14th. Is it September 14th? Not anymore. September 17th. At least where we're at on September 17th, 2021. Welcome to the Active Inference Lab. We are a participatory online lab that is communicating, learning, and practicing applied Active Inference. You can find us at the links here on this slide. This is recorded in an archived live stream, so please provide us with feedback so that we can improve our work. All backgrounds and perspectives are welcome, and we'll be following good video etiquette for live streams. At the short link on this slide, you can see the past, present, and future live streams. And just to call your attention to the tabs on the bottom, the first tab are our regular Tuesday live streams. So for these ones, we do the .0 contextualizing video. That's what you're watching now. And then we have a .1 and a .2 where on two successive weeks, we try to have the authors, try to have a lot of different people on a panel just talking about the paper. The guest stream is sort of a wild card slash anything goes. We have a lot of different topics. Model streams reflect machine learning, different kinds of modeling and walkthroughs of code. And the math stream is for talking about math and different formal frameworks. So if you are interested in co-organizing or in presenting for any of these different kinds of streams, then just get in touch with us. Today in Active Stream 29, we're going to be having the goal of learning and discussing this paper, Active Inference in Active Inference Framework for Ant Colony Behavior by Friedman, Shantz, Ramstead, Friston, and Constant. And just like all of the other .0 videos, this is just an introduction to some of the ideas. It's not a review or a final word. And it doesn't change too much even when the first or any author is on the stream. We're going to go over the keywords as well as the aims and claims, the abstract, all the usual things that we cover in the .0s. And in the coming weeks, we're going to be discussing this paper. So everybody is welcome to join from whatever area they're excited about the paper from. Just get in touch with us. So we can begin with little introductions and warmups. Blue, how about you can go first and then we'll pass it to the first author. So I'm Blue Knight, an independent research consultant from New Mexico. And I'll pass to Daniel Friedman, the first author of this paper. Thanks, Blue. So I'm Daniel. I'm a postdoc researcher in California. And yeah, usually we have everybody except for the first author introduce themselves. And then the first author steps in and gives a little background on the paper. So it's fun. It's the first time that we've discussed one of my papers on the stream. And we'll be talking about a bunch of different aspects of it. And it's awesome to have you here, Blue, with a lot of expertise in collective behavior and a bunch of other areas. The big question that motivated not just this live stream, but this whole line of research in this paper is assuming that each agent and as a quote from the paper, assuming that each agent, NESTMATE, has only limited access to incoming information. How can a group of active inference agents solve complex group foraging problems? So in the case of ants, that local information is going to be the pheromone density, the chemicals that it can perceive. It's local envelope. But in the case of information foraging, it's what we see or it's our perspective or the people who we can communicate to. And so this is a topic that's come up in many different guises, which is, again, when you have a multi-agent system using active inference to model each agents, how do we think about the way that the perceptions and cognition and action of each individual active inference agents, how do those compose or interact so that the group can have adaptive or interesting behavior? The big question of the paper and also, again, just a big open question, hardly one that gets answered in its finished form by the paper or by this discussion. Anything you want to add? Not only that, I was really excited to... Actually, I picked this paper and I've been kind of pressuring Daniel to do it for a little while, maybe three months or so since I first read it. So I picked this paper because complex behavior, complex group foraging problems, it's really linked to my interest in collective intelligence and in scaling intelligence, which is also why I'm drawn to active inference. So I'm just curious to see your insight into ants and the deeper, like when we see a paper, we see only the surface, the final product. And so it's really going to be fun to take a deep dive into the little nuanced things that kind of maybe led you to the birth of this paper. Great. Visual with sort of what's on the surface and what's beneath the surface and in fact foraging is such a fun task to study, not to get ahead of ourselves, but it's one of the few tasks of the ant colony that you can see outside. You just stand next to the nest entrance just like this picture, which is taken while I was doing field work in Arizona. So there, where all the ants are coming in and out of, this is the nest entrance. And so this species, the red harvester ant, Pogonomirmex Barbatus, it's like there are ants coming in and out doing some trash work, mid and work, but mostly what you see on the outside is foraging. And so what happens underneath? Well, here's this painted ants with the three yellow dots. So we can take a little tracer down underneath and we can see what is happening beneath the surface of the foraging. The paper is active in France in active inference framework for ant colony behavior. And we read the authors previously. It was a really awesome collaboration. So thanks a lot to Alec, Maxwell, Carl and Axel who each brought really important insights here. And two of the main aims and claims and then blue, you can provide any thoughts. In this paper, we develop and implement a Bayesian simulation model derived from the active inference framework that captures some of the underlying dynamics and emergent phenomena of ant colony foraging behavior. That's what we did. And the second point is by integrating the Bayesian ant colony perspective of hunt and others with stick merge regulation and the scale free formalism of active inference fields in Glazebrook 2020. We provide an integrated framework for behavioral ecological and evolutionary modeling in ants. Such an integrated approach could facilitate the transfer of modeling insights from ant colonies to human design systems and I guess back. So cool aims and claims. They weren't very forthcoming with or you guys weren't very forthcoming with the aims and claims of the paper but I found like lots of them like nested like little gems in the paper. So this is just a couple of them but I think that this paper was very, on the bleeding edge we say instead of like the leading edge cutting into some novel areas and making some new connections that we haven't seen before or at least I haven't seen before. Cool. There's what we did which is write a simulation based upon theoretical grounding and then there's the fields that were integrated which was a sort of Bayesian forager multi-agent perspective Stigmergy which we'll talk about soon and of course active inference and that is going to hopefully develop the field of ant research and other areas that stand to learn from either the Bayesian multi-agent perspective Stigmergy or active inference so that is quite a large swath. Do you want to read the first part of the abstract? Sure. In this paper we introduce an active inference model of ant colony foraging behavior and to implement the model in a series of in-silico experiments. Active inference is a multi-scale approach to behavioral modeling that is being applied across settings in theoretical biology and ethology. The ant colony is a classic case system in the function of distributed systems in terms of Stigmergy decision-making and information sharing. The second half of the abstract is here we specify and simulate a Markov decision process model for ant colony foraging. We investigate a well-known paradigm from laboratory ant colony behavioral experiments the alternating T-Maze paradigm to illustrate the ability of the model to recover basic colony phenomena such as trail formation after food location discovery. We conclude by outlining how the active inference ant colony foraging behavioral model can be extended and situated within a nested multi-scale framework and systems approaches to biology more generally. How from that abstract do we forage ahead? That's why we have a roadmap. First in the introduction section we talk about Bayesian multi-scale modeling and Stigmergy and then introduce the need and basis of an active inference model of the ant colony. In the methods section we talk about the foraging task specifically which is the alternating T-Maze but that's really just a map. It's kind of like Minecraft or SimCity like you can specify any kind of map you want. We used an alternating T-Maze for a few reasons and then we talk about the active inference model for ant colony foraging specifically. We present some results and we didn't have any live ant recordings in this paper so the results are more like schematic. They suggest what kinds of things can be measured and be discussed for multi-agent active inference systems specifically the colonies foraging performance and swarm coherence metrics. And then in the discussion we talk about three scales of analysis and implication for the work which is at the nest mate the colony and the population scale. And we use those words intentionally because they're all ants. Someone says, oh, there's an ant on my window sill and they mean a little six-legged nest mate but if the colony is the organism then wouldn't the ant be the colony and just it's a piece of ant tissue on your window sill and then we end with a concluding remark. So it's a relatively sparse roadmap but maybe we don't need that many signposts to get where we're going. Any roadmap comments? No, good. Let's forage ahead. Exactly, exactly. The keywords were ants, foraging, just why even continue? Ants, foraging, active inference. Oh, yes, that's right. Behavioral modeling, collective behavior, evo, eco, divo or eco, evo, divo, different people order differently and stigmergy. So these are going to be keywords that open many doors. So let's go into them. We'll start with sort of the grand Gaia level keyword which is eco, evo, divo. That's just how I say it but we might have done evo, eco, divo in the paper. Eco, evo, divo stands for ecology, evolution and development. So this is one way to see it with these three interacting domains. So it's like a tridisciplinary or transdisciplinary approach that is going to, just like the name would have you believe, just like action and inference come together in active inference, eco, evo, divo is going to integrate across these three fields. So what's happening as organisms develop in their ecosystem and how evolution shapes that. But it's also, and this is drawn from the Gilbert et al paper and a good text to read on it is the AboHeath et al 2014 paper. It's actually a historical lineage. So those who study the history of evolutionary theory might be familiar, for example, with the modern synthesis of evolutionary biology which happened in the first half of the 1900s and integrated, for example, population genetics and quantitative genetics with evolution. Well, here's a cool timeline from the Gilbert paper talking about how from the 1800s with Darwin as well as antecedents moved through sequence of really important developments after the first half of the 1900s. So these are post 1950s post the discovery or elucidation of the structure of the DNA molecule, things like symbiosis, the way that microbial interactions shape development. A lot of cool things and on the bottom here growing awareness of beneficial symbiosis. So eco evo divo is referring to a historical lineage that is becoming increasingly recognized as an important integrative take on the systems that we're a part of. What would you have to say about eco evo blue? Just that it's constantly, I mean, we're becoming so much more aware of the different facets that go into eco evo divo and how interconnected and fundamentally important they are in a triad. It's like for me, the clear like I remember being at the neuroscience meeting and I don't remember the name of the professor, but she was a toxicologist from Johns Hopkins and had been working in her field for, I don't know, like 30 years. And this is like when they're doing the microbiome sequencing, which was maybe like 2012 2013 somewhere around there. And so they're doing like the massive microbiome sequencing and she got up and said, we're gonna have to redo all of the toxicology work that I've done my entire life, which it just was so overwhelming and it just really blew my mind to hear such a renowned professor stand up and say, we don't know what's toxic and what's not because we've not considered the whole. And so in relation to toxicity and ecology and evolution and development, like we consider developing, like of course we consider children needing lead paint and so forth. And I think we're, you know, we've looked at evolution but really that this the ecology of our own survival is something I think that people are just starting to to become more aware of and attuned to. And there's a famous quote in biology by Dubjanski. Nothing in life makes sense except in the light of evolution. It's like, but doesn't nothing make sense except in the light of development and ecology and physics and so on. So it's kind of like, right, we should have integrative frameworks, whatever their English titles are so that we can understand all the light that's being cast on the phenomena that we care about. And I'm reminded of the Holobiant too that Chris Fields brought up, right? All of that. Yes, absolutely. What are some of the ways that we can think about the phenomena we care about? Well, one of the big keywords here is collective behavior. That's one of the keywords of the paper. So here's a Lewis Thomas quote. It is in our collective behavior that we are most mysterious. Individuals are collective too. Hashtag we were never individuals. And here's a nice map from Wikipedia of complex systems. Some of the key terms, a little fractal with complex systems and complexity itself. And then like a palette with the art we have the different key terms like systems theory and evolution. And then here's collective behavior up here. So collective behavior is one of the fundamental pillars of complexity because collective behavior is just studying systems interacting. What do you think about that, Blue? I want to say it in the next slide. Sounds good. What is the difference between collective behavior and collective intelligence? Right, and so this is something that we've talked about offline, but really like when you look at collective behavior and collective intelligence, and this was just a map I also pulled from Wikipedia's definition of collective intelligence. And so here, I mean, this is clearly framed in a human situation or is it? Do ants have open source software? Maybe we can talk about that. But collective intelligence, it's these three branches framed in this image. It's coordination and cooperation, which both of those come into the collective behavior like play when I think about fish schooling or like birds flocking, right? This is like a behavior that we behave and coordinate to behave collectively for each other's mutual benefit and for the benefit of the flock or the school or whatever. But the only thing that's left out in this triad is cognition. And so like what is cognition? Is that input output or is cognition intelligence or what's the difference between cognition and intelligence? So for me, there's like a very blurry boundary between what is collective behavior and what is collective intelligence? And so I mean, we've also talked offline and so I just want to know, you know, my thoughts on this are evolving and I'm curious as to maybe people that are listening to this or maybe anybody that wants to come on to our next live stream and talk about it. Like if they have strong opinions on collective behavior versus collective intelligence, but I'd like to hear if you have any, you know, overwhelming thoughts. It's definitely two different perspectives on intelligence and behavior. There's sort of like some who would say that intelligent behavior, it's defined by its outcome and it doesn't matter what's happening internally and then there's sort of the possibility that intelligence could exist in the absence of behavior. Now frameworks like we discussed in live stream 28 with mental action, with mental and cognitive processes as themselves being action and being modeled as action in active inference framework that sort of blurs the distinction but definitely that's going to be a question. Like is collective behavior the same thing as collective intelligence? Is intelligence simply behavior? And then something I always wondered about with collective behavior. Oh, well, you know, collective behavior, groups of people. Okay, but aren't people groups of cells? So if collective doesn't mean anything as opposed to individual, then we're just talking about phenomena or behavior. So, you know, let's drop the adjectives that don't really matter. So what are we signaling when we're talking about collective behavior as opposed to saying, yeah, I'm talking about individual groups, you know, individual crowds of humans, for example. I definitely think that, you know, there's definitely, if you could have no, you could have intelligence without behavior, of course, because babies can't do anything, but like lay there in bed and like observe, you know, like they just, you know, can suck their fingers or something, but there's some intelligence there that doesn't have necessarily an associated action and a tremendous amount of intelligence because they're learning and, you know, building their own model of the world. And so what is collective? And I was listening to the internalism, the externalism live stream, like number 5.2 for anybody wants to go back in time and we visit that with me. I was listening to that last night and it's like, you know, what is the preferred level of analysis, right? Like from the, the noble paper, like where do we get this, you know, preferential level diagnostic? And, you know, we are as individuals, collectives, collections of human and other types of cells. So what is this collective behavior versus collective intelligence? I don't know. It's something that's always mushy to me. There's a Kantian perspective on that for another time. One way that we can talk about a specific type of collective behavior is by introducing this topic of Stigmergy. So sometimes when people talk about collective behavior in the first slide of their talk, they'll say, you know, flocks of birds and here's a group of people walking through a landscape. And those are examples of coordination amongst agents. But Stigmergy is actually pointing out a really specific and important different type of collective behavior. So here's from the Wikipedia, describing that Stigmergy was introduced by a termite biologist, Pierre-Paul Grosse in the 1950s. So it's a recent term, but of course people have seen this phenomenon before and he defined it as stimulation of workers by the performance they have achieved. No, not performance-based benefits for jobs, but more driving from the words mark, stigma and Aragon or work, like an air, the unit of work. It captures the notion that an agent's actions leave signs in the environment and signs that it and other agents can sense which determine and incite their subsequent actions. So here we're not just looking at sort of interactions amongst subunits of the birds looking at each other or the school of fish, but actually the modification of the environment through the actions of the agents and then the ways that the environmental conditions modify how the agents behave. So here is like the agents in feedback with the environments and what Grosse was looking at was how termites, how one termite would pick up a little clod of dirt and then put it somewhere probabilistically and then that would change the likelihood that it picks up or puts down certain other pieces of dirt later. And so you could imagine like one termite building a nest by itself or many happening at the same time, maybe even moving pieces of dirt in the same way. So Stigmergy is actually a mechanism by which extended cognition occurs and by which cognitive processes can be scaled not just by increasing the bandwidth or the type of interactions laterally amongst agents but actually by externalizing the work to marks on the environment. And in the paper it was written here we present the first active inference type model of Stigmergy and the first active inference model focusing on insect behavior. So think about that. There's been a lot of navigation work in active inference but when the mouse is moving through the TMAs for example or when the eyes are scanning over the piece of text which are active inference models that we've seen they don't leave traces. Even when there's multi-agent active inference simulations the interactions have been lateral and the environment has been taken as fixed. So that's why we brought EcoEvo in because now we're thinking about the way that agents are in feedback with their environments and each other and how Stigmergy as a topic reflects that. This is pretty cool. You added this second slide on Stigmergy. So what does it show? I did. So this was another just an image that I pulled off of Wikipedia and it's taken from this is someone's PhD thesis Stigmergy collaboration a theoretical framework for mass collaboration and again it shows here coordination cooperation and collaboration as opposed to cognition in this collective activity or collective behavior model and one of the things that I just really thought was cool to point out was on the right side under collaboration okay so collaboration is distinguished by co-created emergent shared representation which manifests as creativity. I thought that was super interesting because creativity applies implies novelty in some way to me right it implies like either a new use or or or you know developing something that's entirely new making new connections as well as co-created emergent shared representations result in divergent production and evaluation. So how does this co-created emergent shared representation result in divergence? I thought that this was just it was an interesting figure and all these different like components and end results are just an interesting way to think about Stigmergy and like can we go through our environment without a trace we've all seen like murder mysteries or like I mean how hard is it actually to not leave a trace in the environment behind you even if it's hairs or fingerprints or whatever it might be so none of us is without this leaving a mark on the environment and what how does Stigmergy contribute to human niche construction which were incredibly adept at and and how do we learn from that the traces that are left by others? We certainly Stigmergically generated this presentation because I also drew from the 2006 work of Mark Elliott and Elliott wrote many questions arise when confronted with the streamlined efficacy and apparent lack of organization and motivation of these new global enterprises like Wikipedia and others that were being mentioned in 2006 not the least of which how does this work? Stigmergy collaboration provides a hypothesis as to how the collaborative process can jump from being untenable with numbers above 25 towards becoming a new driver in global society with numbers well over 25,000 pre blockchain pre Bitcoin pre whatever it is that we're going through now but these four tenants from 2006 really say a lot which just like Blue's image reflected collaboration depends on communication and communication happens in networks so that's kind of collective behavior 101 that's something that you'd see in the flock of birds or the school of fish collaboration is inherently composed of two components without out without either of which collaboration cannot take place social negotiation and creative output if there aren't lateral social interactions there isn't an interacting group and without the creative output then the productivity is low so maybe systems like that can be designed but probably not that interesting or important this is for humans collaboration small groups two to 25 rely upon social negotiation to evolve and guide its process in creative output so that's like the startup model or the small team model however Elliott's proposal is that collaboration in large groups 25 to however many is dependent upon Stigmargy you need to have if you have 25 or more people in a meeting and it's an hour meeting it's like everybody can be taking notes on a shared document everybody can be editing the wiki everybody can be typing in the chat but if people are talking linearly each person is going to only be able to speak for two minutes on average so just think about how that scales up if you had a thousand people working on a project each person is only going to be able to speak for a few seconds per hour so what are the mechanisms by which instead of just lateral ephemeral interactions amongst agents how can we modify a single source of truth in no sense so that productivity can actually occur and it's important to note that it's not that social negotiation doesn't take place when we're doing Stigmargy in big groups so it's not like Wikipedia editors aren't aware of each other as social entities it's just that as Elliott wrote the negotiation takes a back seat in terms of the creative drafting process most if not all Stigmargic wiki collaborations have discussions associated with the content being developed but it's possible to contribute to wikipedia.org for instance without discussing what you're contributing to or creating so that's kind of a cool insight anything else to add on Stigmargy? just the group size is super interesting because we are like I'm writing a paper on central place for a gym right now actually like an unrelated paper but it's interesting this large groups of humans because like we're looking at hunter gatherer societies and the formation of camps and like what did human society kind of look like way back and really the estimate that's pretty agreed upon in the field is you know 500 people in a camp which like you know it's like a hundred families of five is like the upper limit and like you know mostly like at least maybe 250 people for a camp to maintain right for it to stay cohesive for it to stay bound together and I think it's interesting because what what was the Stigmargic process back in like the hunter gatherer foraging days like what did that look like it's just interesting to to wonder about and think about if it's dependent on Stigmargy I mean like now I think about we stack rocks right like if you you're marking a trail like everybody's been out hiking and you see these rock stacks it's like what are those like when you first start seeing them and then you see that they're trail markers and you know I don't have to know like no one told me that rocks are stacked there to mark the trail but it's like this is a clearly human signature nobody else stacks rocks up that's weird right and so you see these Stigmargic like things that are just left as traces in the environment that could be used to guide like large large groups of humans across time even yes and even in the definition of Stigmargy we saw marks and signs so semiotics and bio semiotics comes into play. Let's get to the ants and potatoes. The you social insects you social is a cool word we're not going to go too much into the difference of what is you social versus other social forms but some of the exemplar you social insects are ants all of the around 15,000 ant species in the world are you social so they all live in colonies that there's no solitary ants and they live in colonies where there's you could call it a reproductive division of labor also honey bees are you social with the queen and worker distinction and termites are as well. Most bees are not you social so honey bees are a bit of an outlier in that respect and just like there are you social wasps not all wasps are you social and there's sort of six images here because I want to show that there's some related species like ants and bees and wasps are hymenoptera that's an order of insects but termites are in a totally different order more closely related to cockroaches and you social insects are able to accomplish tremendous feats of Stigmargy and niche modification. Here's a leaf cutter ant colony being excavated by tiny little humans big colony. Here's a bee hive with honey and the cells and the amazing geometry that they can construct or that they can work with once constructed by humans and also very large nests for termites so you social insects are really cool. How do they do what they do? Well, I wanted to pull from a few slides from my PhD work which was on ants so these are just some fun slides about ants and if you want to ever jump in blue then go for it but this was from some early work that was in the middle school age range talking to people about ants. So one of the first things about ants is there's no leader on the left side we see the without getting into sex and gender politics. There are female reproductives called queens usually again a whole another topic why and should they be called queens there are males who are sometimes called drones they don't play as active a role in the day-to-day tasks of the colony and then there are sterile females which we call worker nests mates and the image on the right shows the queen laying eggs which are being nursed and tended to by the nurse task group of workers so she can just the queen can just pump out eggs and the nurses are taking care of it and they're also feeding the queen and then you have foragers outside of the nest making decisions for themselves so there's no Bluetooth there's no long range communication there's no hive mind need we just have independent agents making decisions for themselves locally and there's no leader ants. How do they do that how does it work without a leader the way it works is that individual ants make probabilistic decisions about behavior based upon the rate and type of interactions they receive so here we can start with just this ant on the top left and it's walking along and it starts encountering a high rate of trash and so it thinks or acts as if it thinks I should start picking up trash maybe the next one I see I'm more likely to pick up. Encounters a piece of trash can identify that by the scent picks it up and can continue walking until it encounters another scent maybe the midden pile where there's a bunch of trash piled up walks along for some more time before it doesn't encounter any more trash and then reverts to a more generally poised state so this can be modified with a drift diffusion model or response threshold model. There's a lot of mathematical models of how different rate and type of interactions probabilistically can update behavioral tendencies but this is just a general way that we can look at how different ant behaviors play out we can always ask what is the individual interacting with and how are they responding to that interaction. Well how do you go from just responding to interactions to adaptive behavior. The nature and what the adaptive behavior looks like is going to depend on the ecology. Again ego Evo Devo so here we see two possible ecologies of food on the left is a colony where food is found sparsely but in very clumpy piles this is like a picnic in the Argentine ants in this case it makes sense that there would be a trail that might get laid down that helps direct nest mates to the location where another nest mate previously found the food. However there's also ants that are looking for food sources that might be mobile like looking for live prey or maybe just scattered dead prey but like just dead insects in the desert bringing them back and in those cases you're not any more likely to find food where another nest mate found it and so in that situation you see a lot more independent foraging activity and you don't see a pheromone trail so just because we're talking about pheromones there's a lot of different kinds of pheromones there's nest marking and queen pheromones and larval pheromones it's not just trail for pheromones and they're not just making these kinds of nests though there are ecologies that favor these types of collective algorithms and then across ecologies there's this lose it to find it component which is the balance in the sense between explore and exploit or between pragmatic and epistemic values and goals which we talk about in active inference because if every single ant were to follow the pheromone trail then they could exploit locally but they would have a challenge discovering new food resources whereas if none of the ants followed the pheromone trail then it wouldn't be informative and so it'd be a totally different game as well so there's this balance between following traces that have been made before and going off and discovering new resources so that applies to seed foraging and also information foraging which is kind of a parallel that we'll be looking at again and again so those four pithy little statements there's no leader it's about the response to interactions and the way it looks depends on the insert on the ecology the interactions with the ecology and that across tasks there's this lose it to find it balance between explore and exploit those capture a lot of the dynamics of ants the life cycle of an individual queen is that she is also born as an egg and develops into usually a winged fertile form that's called a guy like a princess she will mate with males and found a colony but there's other lifestyles for example that don't require mating with males or in which case the queen doesn't found a colony by herself but rather breaks off with some workers there's a lot of diversity in the ant life cycles but once the queen is installed in the colony she pretty much just is fed and lays eggs many of them dozens to thousands or millions depending on the species the other sexual type of ants is the male the male lives to breed they have a larval development where they're tended to by extents nurses and then males usually a little bit leaner and meaner they try to have sex they sometimes succeed and then they die so they only live for in adult phase that's very very short so these are the two kinds of sexual ants the female and the male queen and the male but what we see when we're looking at ants out there in the wild is usually the workers so workers start their life by being born as eggs and they have a different developmental trajectory than the queen even though they're both diploid females in most cases and they can live for months to years depending on the species and there's a process called temporal polyethism by which a worker once it's born the first tasks that it performs are inside of the nest and it performs nursing work so it's like as if right after being born the first thing you did right there in the the neonatal unit was take care of other babies that's kind of like how the ants work when they're middle-aged ants they tend to be doing nest work like architecture and trash cleanup and then in their elder worker phase they do foraging which is the most dangerous task as well as the most water loss in certain ecologies so this is temporal polyethism so it's not that ants are born into being a nurse or born into being a forager all the workers progress through a stereotypical sequence of tasks in fact that progression through a stereotypical sequence of tasks by the workers as well as the fact that workers and queens are both from the same genome in many cases a diploid female genome is what has led many to propose you social insects as emerging models for behavioral epigenetics so we know that they display fascinating individual and collective behaviors as well as niche modification and stigmargy it's also display behavioral plasticity they can learn and they have memory they can also integrate census like vision and all faction and vibration and again this is arising in the context of similar or even identical genetic backgrounds because there are clonal ants so here's Waddington's epigenetic landscape showing how this developmental ball depending on how it bifurcates epigenetically can lead to different developmental outcomes like this first bifurcation is are you going to go the fertile route with developed ovaries and a different kind of thorax and wings or will you go the sterile route at which point about 85% of ants have what are called monomorphic workers whereas the minority of ants have what are called polymorphic workers like there's two discreetly different or more types of workers like majors and minors so a lot could be said on this of course blue can I just ask a quick question so I see like flying ants around all the time I'm not seeing like queens but are there like species of ants that fly also there aren't fly workers there aren't flying workers but during mating flights you will see flying ants and those are guines and drones flying around looking to mate especially in the Southwest US where you are got it okay so that makes sense that I see flying ants I'm not crazy like they are ants that are flying okay just checking people have used many different approaches to study how ants work and how it is the case that you end up getting all these different behaviors out of the same genome one of the approaches that's been taken is just not looking at any of the molecular details but purely investigating from a behavioral ecological approach and some of the critical work here has been done by my PhD advisor Professor Deborah Gordon and so she has looked for many years at the relationship between the statistical regularities of the environments and the adaptiveness of different distributed algorithms so in environments that have as we talked about earlier like spread out food resources you wouldn't expect to see pheromone trails forming whereas if the food resources are distributed like this you would expect to see that and another example is if it's expensive to forage then it's going to be in a default off mode until it's stimulated whereas if it's cheap to forage and maybe even expensive to allow that territory to be taken by different species you'd expect to see like an on until disruption mode and these search algorithms are not just idle play things of mere macologists they've been proposed to be a bridge between organisms evolution and ecology bridging the search algorithms that exist on the internet on computers with the kinds of search algorithms that natural systems use because wouldn't it be interesting if we could have adaptive search algorithms that make all the tradeoffs and succeed that species do and this is an awesome paper by Michelle Denon kind of like a meta analysis across ants studying the characteristics of their environmental niche and then also their foraging strategy so this is a great work to compliment some of Professor Gordon's work behavioral ecology again is just looking at behavior and ecology together so we haven't talked about anything molecular but indeed people have studied molecular components of this from 2016 and well beyond we can use evolutionary genomics to investigate some of the genetic underpinnings of ant behavior evolutionary genomics gives us the toolkits to compare genomes across species so we can look at species that have behavior a versus be and ask if there are gene families that expand or contract in those species so that's about the composition of the genome like what genes are present or not conserved or unconcerned gene families and sequence changes in the genome there's also transcriptomics so that's gene expression analysis where we can look at the same tissue or different tissues and how gene expression is differing between the male and females as well as between the reproductive and the sterile females we can use transcriptomics to investigate the differences between nest mates so a common experimental protocol is for example to collect the brain or the antenna from the nurses versus the foragers and then ask about how gene expression changes are associated with differences in behavior and tissue specificity is really important you can't just blend up an ant and then say something about how tissues are expressing genes and then also most recently there's been some of the techniques developed in other species like fruit flies RNA interference which is like a way to knock down the expression of a certain gene and the first gene gene modifications and ants were published in 2017 with the mutants are here and right now there have only been like knock out which is easier to do than knock in experiments but the future of ant genome editing will be quite interesting so these are using genetic and epigenetic and transcriptomic tools to study ant behavior another approach that relies on neither behavioral ecology or genetics although of course it's related to them is studying physiology so in physiology we're interested in similar types of questions how are species similar and different how are the reproductive and sterile of females similar and different how are the different sterile nest mates similar and different and one of the most common approaches here has been to study the concentration of molecules in the brain like neurotransmitters dopamine serotonin which are common in us as well as well as some neurotransmitters that are trace amines and humans but really important for invertebrate neurophysiological function like octopamine and tyramine and there's been a relatively epic history of drugging ants in the lab going back to at least the seventies giving them opioids hallucinogens stimulants THC all kinds of drugs and more recently some of the first works where we've been able to provide ants with drugs in the field and this is a phylogenetic perspective by Kami et al looking at which neurotransmitters have been investigated across which ant species so it's pretty sparse we don't have a total picture but that's why we do evolutionary biology so we can study how the same molecules in species A B and C play a role and then maybe we can have a good hypothesis about how it might work for species D. So clearly I have no background in in ant you know species neuro transmission but I do know quite a bit about neurotransmitters and I'm curious I haven't read the paper but are the the use of different neurotransmitters species specific in ants or do they all pretty much use the same neurotransmitters to do the same things. They have the same neurotransmitters but species might utilize them differently. And that would be the whole cool thing to study. Let's talk about the ant brain. So on the right side is a Pogo head and this is what the dissected brain looks like it's zoomed in the brain at the same scale as the head is very small. So ants in their head have a lot of muscle for their jaw and a lot of glandular tissue and so the brain is taking up a relatively small fraction of the head. Let's see this video will play. Okay I don't think the video is going to play but Sasha has made some very beautiful immunohistochemical images of three-dimensional ant brains. On the left side we have just one half of an ant brain that a picture that Sasha took and it's juxtaposed next to a 2014 paper that developed a new ontology for the brain regions of the invertible brain because invertebrate neuroscience was a very discordant field in different species people would have different names for clearly homologous brain regions and so this paper in 2014 brought a lot of clarity to the field because it gave us standard nomenclature for talking about different brain regions so although the invertebrate brain uses a lot of the same chemicals and families of neuropeptides that are used in the vertebrate brain it's not organized like the vertebrate brain. So there isn't like a brain stem and a limbic system and a prefrontal cortex there are different brain regions. So just to kind of name a few. Here's like the optic nerve coming into the optic lobe. So the eye is like this sort of curved area that sends signals into the optic lobe and then similarly the internal nerve comes up from this side and projects into these kind of grape looking clusters into the olfactory lobe. So the layout is very different but there are some functional and computational analogies to vertebrate brains. And then this area up on the top is called the mushroom body the middle is called the central complex and the mushroom body is known to play a very important role in route memory. For example the work in Catechlyphus which is another type of desert ant so different brain regions have identified functions and what's cool about the invert and the ant brain is we can almost like visualize the whole thing at once because it's small. Some neural changes have been found to be associated with the bio with the bio behavior of foragers. So for example a lot of hormones that have been associated with foraging activity in solitary insects as well as hunger activity in humans have been found to be differentially expressed in foragers so although they're not foraging for themselves they're foraging for the colony it turns out there are some extremely deeply conserved neural circuits and peptides related to hunger neuro peptide why which is I think called neuro peptide F F in humans. There's changes in gene expression which again are found analogously in solitary insects like fruit fly as well as unsurprisingly in humans some of these questions about regulating hunger and about how the perception the interoception of hunger is linked to foraging activity like get up and do something if you're hungry. These are deeply conserved beyond ants even beyond invertebrates and invertebrates and then just like so many other behaviors have been found to be the case changes in the biogenic aiming neurotransmitters in their signaling properties and metabolism have been established in invertebrates. So these are like tyrosine and dopamine over here serotonin octopamine and then the enzymes that are kind of like the freeways between the neurotransmitters that transform them into each other synthesizing and degrading differences in the expression of these enzymes which we determine with transcriptomics can influence the levels so we can use some technologies like HPLC or GCMS to measure the amount of neurotransmitters in the brain and then we can do gene expression analysis to look at the receptors and at the metabolic enzymes as well. Why does it all matter? Because a forager doesn't know how much food the colony needs it doesn't have a dashboard telling it how many seeds there are how many larvae need to be fed or how much food is out there and so it is faced with this go no go decision stay home or forage which is influenced by environmental features that it may not have total awareness of that decision is much like the decision that for example an investor might face with looking at even when they have total information or partial information you have to make some sort of decision so foraging would be easy if you knew where the seeds were and you knew exactly when to go and you never made a bad decision and that's like sort of you know by low sell high like if you knew what the price would do then trading would be really simple but the whole question is about decision making under uncertainty not under certainty and so that is the question to forage or not to forage and that distinction will come up again and again anything on that blue. Yeah so since he didn't specifically go into foraging I know we've talked offline about different strategies that ants use for foraging and it's a common interest that that we have and so answer central place foragers for the most part with a nest but you've given you know where they they go out central place foraging just in case people don't know is that they leave the nest they go get food and then they bring it back and share the resources with the people that are living in the nest and you told me about ants that use other kinds of strategies like you've told me about the feed as you go ants that don't have a nest but but it's are they also central place foragers or do ants ever use other strategies for foraging other than than that. There are polydomus ants so they have multiple nests and they will so that's not center place because they have multiple places they're bringing it back to there's ants like army ants that I guess it could be considered central place but it's a moving central place nomadic yes exactly so lots of kinds of foraging behavior. Finally we get to active inference which is an integrative framework for studying perception and action of different systems. This is what was written in the paper active inference is a Bayesian theory of behavior that accounts for perception learning decision making and the selection of contextually appropriate actions all of which are cast as a form of approximate Bayesian or variational inference. In this framework perception is modeled as the formation of posterior state estimates. Learning is cast as the process of updating of priors as well as other parameters and decision making and action are processes that compare the evidence for various models about future states and observations under very possible courses of action. So we're going to continue this thread of having a unified framework for perception and action and we're going to apply that to the ant system. Anything on active inference here every every week. What could we say ahead forward so much to say about active inference and the last keyword before we get into the paper more specifically is the behavioral modeling word. So here was from the paper describing how active inference is related to behavioral modeling. Active inference framework naturally lends itself to modeling behavior of different systems across scales generally what changes across models available in the literature are agent action affordances. So what can be done by the agent visual scanning versus physical movement the semantics of the prior beliefs. So what are the belief distributions over and the hyper parameterization the prior distribution of our beliefs like are those fixed or are those learned from the point of view of the dynamics. I eat the message passing and update equations all active inference models are the same and so we've seen all kinds of behavioral modeling frameworks and while many people have sort of agreed that there's like an input output structure. There's some sort of events from the outside world that go to the system of interest to the agent whether it's a computer program or animal and behavior outputs that's kind of where the agreement stops. There's many many different ways to carry out behavioral modeling and totally discordant simulation approaches. So one of the aims of bringing active inference to bear here is that we can actually have a framework that not just integrates the perception and action stages but puts in the ability to specify arbitrary generative models in between that cognitive sandwich and look at the patterns across different systems and different scales because they would be looking exactly the same from a model outline perspective. So just to tack on to that like one of the beautiful things about active inference is it contextualizes the interaction between the agent and the environment. So there is no behavioral modeling like that. Behavior doesn't take place in a vacuum right and so one of the beautiful things about active inference is that it's always you know considering like well how does this plant grow and it's like well that plant grows depending on how much sun there is and how much water there is and so there's always this perception action loop when considering behavior which is just I think probably one of the greatest things about the framework. Here's the paper that we started on that we ended up modifying to introduce Stigmergy to it's a paper the acquisition of culturally patterned attention styles under active inference. So at first glance it's like wait it's it's literally a paper about cultural styles of creating pottery designs. So here's like what the pottery designs look like. There's it's kind of black and white art on pottery and then the eye can see a local set of cells. So like the focal point and the immediate periphery and motifs are visually detected by looking just at pottery and so different kinds of patterns can be inferred by the visual scanning over pottery patterns and so in the quest to resolve a culturally informed uncertainty. We have these scanning paths over motifs. So here it's like the motif was just like this white line. It's like the eyes moved across and I was like hey I'm kind of curious is there is it going to be a motif that looks like a Tetris piece scan up. Oh no okay and then the scanning happens like that whereas other motifs have different scanning paths. So that's going to be a lot like the foraging paths that we're going to be looking at but there's a few questions that we want to consider like how is visual scanning similar and different to ant foraging or in other words how does epistemic foraging as well as pragmatic foraging differ when you have skin in the game or exoskeleton in the game you have to move your body visually you can scan and then go back and then scan and go back and you haven't left a mark on your environments. So it's a foraging task but it's not Stigmergic because your vision doesn't modify the object of vision and so there's these similarities and differences between physical foraging and visual foraging and then there's the question about how these visual and bodily foraging phenomena are similar or different than conceptual or informational foraging. So foraging is kind of this concept that links all these different pieces from an animal physically moving on a landscape to our sensory modalities like vision scanning a landscape to informational landscapes and their similarities and differences across these. So we'll look at the some of the figures and formalisms the code and so on figure one has a layout of the map and the model and so the map looks like a T at the bottom of the T is the nest. So this is like the point where the ants emanate out of when they start foraging and then when they return that's the end point of their foraging trip and in the classical T maze paradigm food is placed on one of the arms of the T like the left or the right arm. In this case we used an alternating T base paradigm so the food would be on one end of the T for some period of time and then it would switch to the other side of the T. So this is the foraging task that we set up but this is just a map you could make it look like a labyrinth or it could look like anything in the simulation. What happens is the ants start at their nest and then they implement a model which will look more at soon but the ants action affordances are movement it can move in any of the directions that are surrounding it. So a future model could have ant inertia or it could only move forward to the sides like a pawn or something like that and chess could rotate but we didn't keep track of the ants orientation. We just said the cell is about as big as an ant and they can move to any of the adjacent cells. And the ants able to perceive and act in those local that local environment. And then we use the following to parameterize the model so here we go little bit blurry but there's a generative model which is just like in any other active inference model it's a generative model of outcomes and hidden states here the outcomes are the perceived pheromone and the hidden states are the true concentrations that are being inferred of the pheromone. The the factors that go into the the Markov decision process that's happening inside of each ant are its prior over hidden states the a matrix which maps from the hidden state to the outcome the B matrix which is describing how hidden states change their time and then policy which is suggested here to be influencing the transition between different states and the free energy minimization which dictates policy selection and then we discretized the pheromone into 10 categories so there was 10 possible concentrations of pheromone in this model. So that's kind of setup of the model is food is going to be placed and alternating between these two sides of the maze ants are going to leave the nest and diffuse around with no preference for a direction. Except when they find a seed they're going to start depositing pheromone on their return journey and then they have a preference for walking towards regions of increased pheromone and it'll be shown that that is actually enough to recapitulate some of the most interesting behavior of ants like the ability to form a trail to food and just one interesting piece that differentiates the active inference model from traditional reinforcement or reinforcement driven notions of foraging is in those models the reward would be the seed at the end of the food itself and that the cues would be the pheromone trail along the way the pheromone trail would be seen as like a means to an end and the seed would be like the goal but in active inference the reward in other words what the preference is over is actually over pheromone density and the seed is seen not as a reward but rather a semiotic cue to switch strategies so active inference really inverts the situation by making the reward about the moment-to-moment decision making of the ants and then sees what is traditionally seen as a reward the food itself as actually just a cue to switch behavioral strategies so that's one of the most interesting things that we figured out so can I just comment in here really quick just that that this switching task teammates is like it's a very simplified version of the switching bandit problem so we discussed the multi arm switching bandit back I guess it was maybe 23 I don't know if you recall which live stream it was 25 maybe but it was back I think before the summer so maybe 25 or 23 so that that live stream talks more about like a stationary bandit problem and a switching bandit problem but had multi arms like I think eight arms and here there's a link there's only two but I just wanted to draw the parallel between the two kinds of simulations it was I just looked it up it was 24 with empirical evaluation of active inference and multi arm bandits this is a two armed bandit it's a two armed teammates but you could imagine the different switching dynamics or different number of arms so it absolutely is related to that let's go into a little bit more detail on the formalism and the model itself so here is describing the process of variational inference and specifically there's a cue variational distribution that's being fit as a function of the policy and it's looking at the hidden variables of pheromone density through from the beginning of time to a given time step and of policy so those are the kind of the pieces that inference is being done over and then there's a generative model of the outcomes the hidden states and the policies so that's what the agent is doing variational inference on and that model doesn't need to be jointly considered all by all it can be factorized which makes it very tractable to compute and then given the factorization there can be a free energy minimization such that actions are selected based upon the optimization of this variational distribution in a way that it's selecting policies that are expected to minimize the free energy so that's in the appendix and these are some of the formalisms and it is minimizing the difference between the cue variational distribution hidden states at time t given policy the difference between that variational distribution and this P distribution the actual empirical distribution which is over outcomes states also conditioned on policy so there's again so much to say about the active inference formalism but really importantly the calculations are conditioned on policy so it's like we don't need to calculate the exact momentum or trajectory of the car we need to pick a policy condition on what affordances we have and so that in a way simplifies and sidesteps a lot of the questions about wanting to describe the world to the highest possible degree of precision because actually what we want precision on is our policy selection so that's one key difference is that conditioning on policy and then this ability to do variational inference on a simplified and factorized representational distribution cue and then minimize the divergence the expectation of the divergence between the cue distribution and the empirical P distribution including outcomes so those are some of the pieces of active inference that help make it a really scalable and tractable framework here's the figure in the appendix so we talked about the Markov decision process and what some of these variables mean but here is like the center point right with this is where the ant is located at a given time and then what it can perceive and its action affordances as well are the adjacent squares so just like in the visual scanning model that I could perceive the center point and the adjacent cells here it's pretty much the same code actually that describes how even though there is a computational object there's a variable that describes the actual pheromone density across the whole map just like there was an actual computational variable that described the whole pattern of the pottery we're able to use a remapping matrix that zeros out all of the other cells that aren't the adjacent so that is what ensures that each agent is actually doing a purely local decision-making so we kind of take a per agent approach to masking such that its action and perception are constrained to only its local neighborhood so that could be changed you could have ants that are aware of 10 cells around them but only act within one cell and those are the kinds of extensions to the model that could be done but we constrained perception of pheromone and action to just adjacencies. So this is one thing that like I wish you guys would have played with because what I would like to see is like okay is there something in my immediate proximity like I read like in the likelihood remapping if there's nothing in the immediate proximity then the ant moves around in accordance with Brownian motion as opposed to moving towards the pheromone and like I would like to see just an extra step of code if there's nothing in the nine let's look in the 25 and move toward the thing with the highest pheromone like that's one like because okay it's like a just a more distal like well like a trace of ascent or something like that. Yes, it could be done. It could be done by modifying the code. So here's alex github and just look alex chance and ants probably a coincidence that it's in his name the last few letters but you really never know these days and the main pieces of code to look at our ants dot py python that's the main script ants dot pi config dot pi has a lot of the parameters so if you want to explore having just two ants in the nest or five million ants in the nest you can just change a parameter in the config file and then when the ants dot pi is run it's going to like look at the config file to run and then figure one two and three are just for reproducibility to exactly reproduce the figures in the paper. Let's talk about the pseudo code so we can maybe go over the code during one of the dot one of the dot two but this is from the paper to summarize the computational activity of each forager at each time code pseudo code is provided here see the code for full details. The active inference model as implemented here does not presuppose or imply any specific neuro cognitive architecture for any specific ant species. I believe that was some reviewers comments about what this did or didn't say about what was happening inside of each ant and again that's why it's so fun to talk about active inference and these broader philosophical questions while realizing that actually we're just looking at how sensory information comes in and what actions come out and the internal generative model can be a hypothesis that's informed by cognitive architecture but in fact it isn't that cognitive architecture it's kind of like black box modeling but then we can go into the box and start building sub boxes and hypotheses about what is inside but we don't need to presuppose any specific cognitive architecture and in this case the ant has no memory and no anticipation so it's like kind of like a one step make a move towards the pheromone it's not predicting or remembering what's happened but what happens at each time step is it's like you know for each ant first the ant perceives the local environment that's the pheromone density it then updates or optimizes its beliefs about pheromone density with respect to the variational free energy it then after updating its distributional beliefs about pheromone density updates its belief about action so relatively given the affordances moving in different directions update my beliefs about how different action policies will relate to expected free energy so it's like let's just say there's a pheromone above but not as much below that would be perceived in in step one and updated in step two and then it'd be like well moving up will I can expect to be more like in my preferred state than if I were to move down and then action is sampled based upon the relative proportions of the relative free energies of the different action affordances so that action is sampled from that belief distribution action selection and then the agent performs movement and in the case of it carrying a seed so having already found the seed it deposits pheromone otherwise it doesn't deposit pheromone on the way out there are ants that sometimes deposit pheromones both ways but we were studying the sort of simplified case where the ants only puts down pheromone on the way home so can I come in really quick on the neurocognitive architecture so again like after listening to the internalism externalism live stream 5.2 I believe it was you know as somebody asked that question and Maxwell give this great answer which I thought was was really cool like okay so where in the brain is the generative model in this like internalism externalism debate and that's something that we see like with the visual cortex mapping etc so where is the generative model and Maxwell's answer was really cool like when you go to machine learning and you don't like look inside the computer for the generative model like where's the print of the generative model it's like software it's just running so I always think that that's a really neat and sorry sorry Maxwell if I really paraphrase you horribly because I may have but but but there's no need for the model to be necessarily inside the brain mapping to some specific brain region it can just be software that's just running yes figure 2 showed an example trace of just a simulation so the food starts off and has not been discovered yet so like the density is low and it sort of starts here pure no pheromone then it's kind of follows like a zigzag and you can see like the food was on the right side and so there was a pheromone trail that was forming down to the nest starting up on the top right with the food and then the it switches to the left side and so there starts to be pheromone laid down going towards the left arm but what's reinforced is still the trunk because that part is like good to have high density on no matter what and so that was our simulation this was a colony with 70 nest mates over 2000 time steps with three switches of the food location and we didn't optimize the parameters or try to like find the best performing set or it was really just to show that this phenomena of a pheromone trail leading to food that can loosely track location of food changing due to the natural decay of the pheromone so informationally this decay rate can be thought of as environmental perturbation or drift that counteracts the reinforcing effects of Stigmergic convergence or Stigmergic tightening we were just like this is the first pass model that displays that phenomena and it could be optimized to explore how it works differently what if one arm is longer or all these different pieces but this is kind of what the model outputs and then in the supplemental videos there's a lot of videos there's some some large file size gifts of you can see the ants themselves diffusing around then once there starts to be a trail they start like really start taking advantage of it one of the pieces that was most exciting to think about was how do we measure forging performance and this speaks to a common area in studying collective behavior which is like how do we think about the relationship between individual level behavior and group behavior again we know that as we talked about earlier like the individuals are groups and so on but just looking at a two level model where we have like agents and then the collective what are the difference between individual and colony level traits so colony level phenotypes also that's why we use Nesme and colony level so that we don't have to use individual or group because Nesme is always just going to be Nesme colony means colony colony level phenotypes are summary measurements of emergent properties in that phenotypes do not apply to separate Nesmates but they're enacted by Nesmates and these are good citations to check out for example each Nesmate has bodily phenotypes like the width of its head every Nesmate has a head width or has a mid gut gene expression profile but colony phenotypes are those that only exist at the colony level so there's only one measurement of those per colony like the total forging performance or the average interest in Nesmate distance or the sex ratio of larva so which sex each larva is is Nesmate level but this larval sex ratio only gets one measurement per colony and so we looked at two measures of colony foraging the number of foraging trips round trip completed through time and a swarm coherence metric so this is just the amount of foraging trips completed through time showing how as a function of the number of Nesmates in the colony you get different dynamics so for example 70 ants finds it faster the first return trip happens before the 500 time steps finds it faster and ends up getting more but potentially not you know multiple times is more so there's like some sort of a scaling relationship between the number of Nesmates the performance and then like look at 10 like it finds a few seeds this blue line but then it kind of flattens out so maybe the pheromone trail that the first few brought home just dissipates so then none of the other ones are really able to latch on to that trail so it has a very qualitatively different shape than the other ones and then this is the swarm distance coefficients which is like the average distance between the Nesmates and so just interesting that again it could change maybe if you run it from work time steps or you have it in a different structure than the teammates but it's almost like the distance increases as the ants spread out and then there's almost like a stationarity where the average distance stays relatively similar so these are just a few examples of the kind because you have in the simulation at least you have full information on the location of every Nesmate so if you want to do an individual Nesmate level phenotype you can calculate that trivially but then also there's this question about how do you summarize collective foraging performance so any thoughts on that so the choice to have full information on the location of every Nesmate is an interesting one and also like I wonder if it's a parameter that you guys tweaked or played with at all like what if you just have knowledge of half of the Nesmate locations like you could almost simulate being a behavioral ecologist the grad school simulator where it's like the computer is simulating a hundred Nesmates but then you can only see 10 of them or each ant can only see 10 of them so we have full knowledge or the ants have full knowledge so that that was the question that I have here we as simulators have the total trace of the simulation and one important piece that is not included in this was like interactions among Nesmates they just passed right through each other in this simulation so that's definitely not too realistic and in nature the way that ants know whether they're heading inbound or outbound they can look at the polarization of the Sun there's step integration and the memory there's visual scene recognition and then there's also importantly interactions with other Nesmates on the trail like if you're bumping into Nesmates carrying food you're probably heading out and if you're holding food bumping into empty mandibald Nesmates you're probably heading in and so they can combine all these kinds of sensory information to determine a lot more information than the kind of drift model that we instituted here well and it just reminded me of this model that we just built with this forging paper that I'm part of and in that model all the foragers know the location of all of the food a priori right like so they already know where all the food is so but learning where the food is is really like a whole different model right like going to get the food and modeling interactions like you know network formation in forging patterns is different than learning where the food is how we learn from others this collective like learning and that's something maybe that's the difference between intelligence and behavior like you can just follow what other people do that's maybe behavioral but like learning from others based on different cues and so forth I mean that that's a that's a maybe an intelligence metric let's return to this broader question about Stigmergy as a type of niche modification and how it plays a role in extended cognition so this is some nice writing by Axel probably and referring to some of his work the representation of the environment leverage to solve the team is problem i.e. the map is not the one entertained by nest mate brains but rather the one that is carved out of the environment and leveraged implicitly by each ant as they each forage alone together as a colony leaving cues that nest mates can use adaptively so I like that kind of like collective behavior as being alone together like each nest mate is perceiving its own umbelt its own local environment is all it perceives yet the way that the colony approaches this problem is truly collective as well as truly Stigmergic in active inference terms the answer endowed with only an extended generative model an extended generative model is one where some of the model parameters are encoded not by the agent engaging inference itself but by the physical environment in which it lives so that's kind of like a person maybe can have a few numbers in memory but now imagine if you're hearing a string of numbers and you're writing them down and then you're able to go back and interact or somebody else could have written the numbers down you're going back and your generative model is functionally including things that are now part of the environment like they're outside of you so that really speaks to this integrating internalism and externalism and it brings up big questions like what other kinds of systems whether they're currently modeled not in the collective behavior framework or whether they're being modeled as collective behavior but not as Stigmergic collective behavior what other systems can be modeled with Stigmergic active inference algorithms and then what kind of individual level Nesmate level behavioral heuristics contribute to adaptive collective behavior in the context of what kind of statistical regularities in the niche or regimes of Stigmergy because just it's not like Stigmergy equals adaptive you could imagine the switching teammates where they reinforce it too rapidly and then it is inflexible as the food changes location so it's not like oh collective behavior equals it's going to be adaptive or Stigmergy equals adaptive there's actually probably only certain combinations of parameters certain regions of parameter space in which specific environmental regularities can be capitalized upon. So the idea that the answer only endowed with the extended generative model in the environment and I know like that in the paper they mentioned that memory is something that could be played with but I'm reminded of the story of Hansel and Gretel so like where Hansel leaves a trail of breadcrumbs behind them so that they can find their way back home but or a trail of pebbles the first time that we're not eating so you leave the trail of pebbles is like modification of the environment but like if you can just remember the way right like that's that's an interesting dynamic between knowledge gained from environmental cues versus knowledge that's inherent and retained and I know maybe you have maybe more to say about ants and ant memory and where maybe these memories or if there's some genetic memory for ants in different species or in maybe even in just single colony right like you've talked to me before about colony foraging behavior and even in the same species the foraging behavior is different and so I'm curious as to your thoughts on we don't have to go in right now maybe for next week but your thoughts on ant memory and how that may come to play. Interesting question I think species where the foragers head out alone for longer distances and need to find pieces of prey and bring them back we'd expect them to have higher memory whereas the extreme case of just the pheromone follower you know pheromone respecter ant maybe there's a very limited role for memory and then the question empirically would be are there morphological differences in their brain is the mushroom body smaller are there certain genes that have differential expression patterns how do nurses and forager differ in the expression of those genes like maybe nurses don't need to have as much spatial memory but foragers do need spatial memory or step integration for example it's been shown in multiple species that nurses don't have a circadian rhythm they are just sort of low key working and chilling 24 hours a day whereas foragers have a circadian rhythm that's strongly in tuned by the light cycle so how does memory sort of start and then how does the temporal polyethism and the trajectory of what the nest mate needs to do for the colony how does that lead to the development of nest mate level cognition and then how have the regularities of the niche selected for colonies that succeed by virtue of being composed of certain kinds of nest mates that have certain kinds of memory this returns us to Stigmergy so what was cool about this I just thought it was cool. I mean I'm always looking for you know intelligence and behavior and how these two were linked together but this paper which I found like self published on Amazon but I think it's part of a proceedings anyway it was just like a neat kind of like Google find it says Stigmergy an intelligence metric for emergent distributed behaviors and I'll just read this little piece of it. It says the complexity of aggravated behavior often depends on Stigmergy Stigmergy occurs when behaviors by individuals modify the environment while being regulated by the environment state Stigmergy has generally been studied for the forward problem predicting the consequences of local behaviors. It is also applicable to the backward problem synthesizing local behaviors to fulfill a global need. The concept provides provides an objective measure of intelligence for natural and synthetic systems. A systems intelligence is measured by its amount of effective Stigmergy. It not only adapts to changing environment but also modifies the environment to suit the system's needs and goals. So I thought that this was just kind of a neat refreshing perspective on a lot of things we've been talking about today. Agree intelligence is all is often thought of as just being like inside of a brain or in one agent when we know that that doesn't take the brain body into account doesn't take the social into account and then even then that's where this paper came in was okay. So then even social cognition let's say you integrate brain and body as well as a social you're still just in this floating mob type simulation. How do we talk about the way that interacting agents modify their environment and how that updates their tendencies. That's where Stigmergy comes into play and that's why it was an important thing to do. But and we're increasingly seeing the links drawn between cultural evolution and collective intelligence because as a species I mean we're still obviously not discovering fire and learning how to you know make buildings right like we already know how to do that that's why we're not like living in these camps anymore and so we've evolved and you know none of our ancestors knew how to use an iPhone but maybe you know in nine generations none of our none of our what are they called it successors that that's what it is none of our successors will also know how to use an iPhone so it's just us right now but everything that we do we're leaving these Stigmergy traces behind in our environment that enables us as a species to evolve or become more intelligent for better or for worse. Anyway big questions so there's some big questions for next week that I'll give you to think about. Are you here so what is the relationship between collective behavior behavior and collective intelligence and I'd love to have other people's takes on this what are other possible measures of group synergy or colony synergy how are colony level metrics retained in ants over time in this model forgetfulness is critical so do you think that the ability to forget is as important as memory in human or ant behavior is Stigmergy a measure of intelligence or behavior or both maybe and then does trust play a role in use social insect synergy and I have some more but I thought that that was a good start. All great questions I would yeah I think I got some dot one questions I got some dot two questions got some dot three questions but this is an open line of research the paper was published only this year in 2021 so if this is something where you think oh I have a theoretical contribution or I study archaeology and I want to learn how Stigmergy might relate to those kinds of models or I'm way better at coding than this and I could make dot dot dot dot improvement on the code let's do it so these will be fun things to discuss blue thanks for all the help on the dot zero in this fun conversation we hope to see many of you on the dot one and dot two in the coming weeks on the 21st and on the 28th and Tucker in the chat wrote would love to hear and learn more about memory and collective systems totally agree and when we think of like the von Neumann computer architecture it's like where's memories like I mean there's two spots where it can be right it can be in the hard drive long term storage or it can be in RAM short term storage and then that has even percolated into how we think about short and long term memory in organisms but now what about the niche modification what does it look like and how do we think about it is it simply like data A is in short term and data B is moved to the environment or is something modified about the environment and something modified about the internal generative models such that the information could be recapitulated only through interaction with the environment how would we think about memory in that case and that's also why aunt colony optimization and swarm computation are part of the broader umbrella of unconventional computing architectures because they do challenge or move beyond some of the sort of classical notions of memory processing bandwidth clock speed we didn't have to talk about that but those are things that studying natural systems makes us wonder about fun times blue see you soon see you later see everyone next week bye.