 Hello, everyone. Welcome. It is September 13th, 2022. We're here at the Active Inference Institute with John Vervakey. So, John, welcome. Thank you for joining. I'm very much looking forward to this conversation. Hello, and to you. Thank you, Daniel. It's a great pleasure being here. I'm looking forward to this. Is there any introduction or hello that you would like to provide as we head into this journey together? I guess I can give just a little bit of background so people know where I'm coming from. I'm an associate professor at the University of Toronto. I'm appointed both in Cognitive Psychology and in Cognitive Science. I teach in both, and I am the director of the Cognitive Science Program. So, I do a lot of work integrating, especially psychology and ideas from artificial intelligence, large framework models, philosophical framework models for the nature of the mind. I especially do work on intelligence and consciousness, insight, flow, mindfulness and wisdom all centered upon a notion, well, not a notion, a proposal, a theoretical proposal that I call Relevance Realization, and perhaps we'll get into that at some point. Into it we go. So, to provide some context, these live streams are multi-directional roads. We're gonna be talking about different ideas for some people. It'll be their day-to-day life and the water they swim in. For other people, it might be their first time hearing about Relevance Realization or predictive processing or active inference. And so, we always wanna make it accessible and also provide depth. So, yesterday in speaking with some other participants at the Institute about how to frame this discussion, I was bringing a notion of a three-legged stool and the three legs of this stool that provided stability are, again, those Relevance Realization, predictive processing and active inference. And then a participant dean said, flip the stool, it's still stable. And so, that led to some discussions of what was the betweenness of these ideas and what were they really doing alone and what do they do together? And we're going to approach that. So, first, we'll be exploring Relevance Realization and predictive processing in terms of what they are and how you recently connected them in a recent paper. And then we'll bring in active inference as a third guest to the party. But first, what is Relevance Realization and what is predictive processing? How did you bring those two together in the recent work with Brett Anderson and Mark Miller? So, the two lines were working independently. I first got my first paper officially accepted in 2009, didn't get published until 2012 because they were holding it back for a special issue that took forever to come out. The reason I mentioned the earlier date is that's before, I think, most of Friston's major publications. And I'm not claiming precedent or anything, I'm just claiming independence. I didn't know about his work until after the theory was well-developed, and I assume he might even not know of mine even now, and that's fine. So, the idea by Relevance Realization is that if you take a look at many different problems within attempts to explain the mind in a non-homuncular, non-question-begging manner, you hit this problem of Relevance Realization. So, for example, how does categorization work? Well, we notice the two things are similar. What does it mean to say they're similar? Well, logically, it means they share many properties. And then Goodman famously pointed out that any two objects share an indefinite number of true properties. You know, aplomb and a lawn mower both have round, shiny surfaces. They're both found in North America. Neither one existed 300 million years ago. Both have an odor. Neither one is a particularly good weapon, et cetera, et cetera, et cetera. And what people will say to me is, they can't say those are false because they're all true. They'll say, but that doesn't matter. And that's the move, right? When you go from logical similarity to psychological similarity, they'll say, well, I zero in on the relevant comparisons. Ah, and how do you do that? And that, of course, is still the crux of the categorization problem. Let's take a look at inference. Well, inference is just to follow all the implications of a proposition in relation to other propositions. Well, how many are those? Well, let's take a proposition. It's going to be windy tomorrow. That has, you know, that would have implications for me if I'm sailing that are different for the implications for me if I'm flying a kite, or if I'm planning to have a picnic, or if I'm planning to stay inside, or if I'm planning to take an airplane. And do I check out all the possible connections between that proposition? No. I only select those that are relevant. And this was a point made very, very well quite a while ago by Christopher Cherniak in his book, Minimal Rationality. Problem solving. Many of you are already familiar with this, so I'll just state it. We know that the search space for most problems is combinatorially explosive. You can't search the whole space, so you have to somehow ignore most of the space. Well, nevertheless, reliably being able to solve a wide variety of problems in a wide variety of domains. I can make similar arguments for communication. I can make similar arguments for learning, et cetera. I won't go into them. I'm just giving enough to exemplify that independently all of these things keep crashing into this issue of zeroing in on the relevant information. And you can even note it in your everyday experience. What should you pay attention to right now? What should you draw from working memory? What should you draw from long-term memory? Especially the fact that you can combine different things from long-term memory. What sequence of actions should you perform? What possibilities you could consider, et cetera. I take it that this is at least plausible enough for me to go on. People can take a look at all the publications to see the arguments more in depth. So this crucial ability for relevance realization needs to be addressed if we're ever going to provide, I've argued since my thesis, if we're going to provide a naturalistic explanation of the mind. And to the degree to which we avoid it, I try to just pretend it's not there, et cetera. I think it undermines our efforts to give a naturalistic explanation. Now, this is sometimes called the frame problem. I don't call it that anymore because Shanahan has rightly argued that the frame problem was two separate problems. One was to create a viable logic for change that didn't get you sort of enmeshed in a bleed out of all kind of identities. And he argues that that challenging problem has largely been solved. He claims to be one of the people that solved it. Nobody seems to be challenging him on that. I don't either. What he says remains, and those are his words, not mine, is the deeper problem of relevance. That's exactly what he calls it. The relevance problem. So this problem is a key problem and I have a proposal that I've worked on with other people. Tim Lilacraap, Blake Richards, Leo Ferraro, Brett Anderson and Mark Miller in the recent paper. Gary Hovanesian in the paper we recently published. Also integrating relevance realisation theory and inactive big five personality theory. And the proposal, I mean the proposal has more sort of technical details, but just to give the idea. And we're not claiming that this, I'm not claiming, none of us claim that this proposal is completely novel. What we're claiming is you see it implicit in many different areas where progress is being made and you can draw it out and explicate it and therefore make more specific proposals of how to look for it or how to engineer it. And this is the proposal that the brain basically uses at many different levels opponent processing and what it's doing is it's doing something very analogous to evolution. It's opening up variations and then putting selective pressure on them and then from that opening up variations putting selective pressure on them and you can see this at many different levels. Your attention is doing it right now. Your default mode network is trying to make your attention wander off and introduce new ideas. Your task focus network is drawing it back in, killing off most of those options, but one of two of them might be actually relevant what's going on. You keep them. And the idea is you do this and there's many of these trade-off relationships. I'm just giving one example. There's many of these trade-off relationships. They're operating in a massively recursive multi-dimensional self-organizing process to help you repeatedly zero in on the relevant information. Like I say, there's more detail obviously, but that's as fast as I could make it a gist of what the theory is proposing. Thank you for the summary. Definitely a lot there. In one way, it's like moving from the sketch of the continents and the lay of the land, the space of the possible for cognitive science across these different domains and then asking how the relevant few combinations are selected. And then one can point to a broad range of so-called cognitive phenomena in human as well as in other cognitive systems learning, memory, planning, attention, compositionality, action selection, and policy inference. All of these areas which one might imagine could have bespoke custom theory or there might be a way to look at some similarities ironically or not finding the relevant similarities in these different cognitive domains and asking whether we could improve our inference and our action regarding those domains through the usage of an integrative framework like relevance realization. That's excellent, Daniel. And so just to add to why I think it's reasonable and perhaps even plausible that there is such a domain general central capacity is we have robustly replicated evidence for the existence of general intelligence. By the way general intelligence doesn't mean it can't be analyzed in components. It means at its functional level, it forms a causal whole that has, you know, reliable predictive relations and we have it's, you know, it's the most powerful psychometric in psychology and people don't like to hear that but it is nevertheless the case and so we have evidence that there is above all and not in place of, but above all the specific learning you have for specific domains, there is powerful evidence for general intelligence a domain general ability. The way we test for it, Leo and I made this argument in another paper, is basically we test for people's capacity at relevance realization again and again and again and so that's why I think it's plausible that there is this underlying capacity and what that helps do is it helps it allows you, it allows to bridge into more comprehensive patterns of explanation we have. So for example, the model I just proposed to you is explicitly an analogy, but a analogy, you know, to the way evolutionary processes produce adaptive morphology for organisms evolution variation selection and it doesn't require an intelligent designer, right? Evolution is clearly a process that is itself not intelligent and can produce intelligence and so that gives us the potential way out of any homuncular explanations of intelligence, like, you know, the central executive selects this or that and those kinds of explanations that just presuppose an answer to this question and so instead the idea is, no, no the brain at this general level, again I'm not denying specific heuristics although I'm proposing that perhaps relevance realization is how those specific heuristics might be generated. I'm not denying specific heuristics, I'm not denying domain specific knowledge, I'm not denying any of that, but I'm saying there is a general ability just like there is, right, there's general process of evolution for biological adaptivity. There is this general, if you'll allow me, cognitive evolution of your fittedness, your cognitive fittedness as a problem solver to your environment. Yes, in those ways it's very resonant with generalized Darwinian models or like neural Darwinism as well as the trend in eco-evo-devo, ecology evolution and development by way of looking at proliferation and that pruning back process in different spatial and temporal scales, different systems like the way that it might happen with synapses forming and then being pruned during development, the way that ecosystem assembly might occur with initial plants forming some sort of substrate that later forms ground for development and then even at slower and longer time scales with evolution and especially when we think about the neural aspects, learning and behavior are part of development, they're processes that happen within a lifetime and that is a great angle to consider. So relevance realization in our first pocket let's move to the second leg, predictive processing. What is predictive processing and how did it come into play and why did it come into play for this recent work? So I've been familiar with predictive processing and I would often be an annoyance that the conferences were predictive processing was being presented because I would ask where does the model come from and where does the innovations for the model come from and things like that because I was and the questions were genuine and I wasn't trying to sabotage I wanted to understand how this was integrating and I wasn't getting to be fair and the fault is probably also mine I was learning all about this I've learned about it quite a bit and I have my bias but Brett Anderson approached me and he said, look I think that what you're doing in relevance realization is really needed for predictive processing and at the very same time and this is one of those happy coincidences that are the shadows of God or something like that a former student of mine who went through the cognitive science program with me, Mark Miller who has worked with Andy Clark and a bunch of the other predictive processing people came and he said, John I want to start working with you again and I want to integrate what you've been doing and I've been doing and so the three of us were talking and when we went back and forth a lot and we came up with a way in which we saw these two theories converging so predictive processing there's a lot of technicalities and people vary to the degree to which they're committed to a strong Bayesian or weaker Bayesian everybody agrees that you can't be pure Bayes because it would be computationally intractable you have to do some approximation you have a massively recursive network that is basically doing something really interesting now again not to claim priority or precedence but independently in the 2012 paper we predicted that the brain basically tries to model itself in order to model the environment because this actually gets you through a lot past a lot of difficult problems and that was independently and again I want to give due credit the idea was no no the brain shouldn't predict the world the brain should predict itself in a multiply recursive self-correcting fashion in order to predict the world and for reasons that I won't go into right now that just gets you past so many of these roadblocks and then the idea is what should the brain doing when it's predicting itself there's all kinds of math but a good interpretation is the brain is trying to reduce the degree to which it surprises itself higher levels you've got a level of the brain level is meant to be taken as an abstraction not an anatomical region there's a level of the brain doing the sensory motor interaction with the world and the idea is this is a loop it's not input output it's a continuous loop perception is modifying action and action is modifying perception and so there's a level of the brain that is just registering that as patterns of activity in that causal interaction and then layers above it try to predict how that pattern is going to unfold and when there's error back into that layer and it is corrected to try and make a better prediction and then what you do is you do this massively recursively you have layers predicting the layers below them so on and you propagate error up and you propagate prediction down and what that does is it allows you to slowly slowly hone in on the causal structure of the world and it doesn't promise to get you the perfect causal structure of the world and it's not a strategic thing like that but what's really interesting and one of the things we talked about is you also have that recursive sort of stacking of layers of prediction in various models of deep learning and deep learning also does it was one of the inspirations for relevance realization theory it does the variation and selection in things like the wake sleep algorithm and we thought hey there's a lot of things converging here and then we started examining this with other people like Clark he says well what do you predict there's a combinatorial explosive number of patterns you could predict in the environment you predict the important ones or the relevant ones oops that's going to be problematic you predict the reliable ones that's not good yet enough and then you had the idea of precision weighting in which you're trying prioritize which patterns are given processing which is basically a model and many people explicitly claim this in the predictive processing of attention that's what attention is psychologically a prioritization process that's doing the precision weighting that helps the predictive processing avoid combinatorial explosion you can see the convergence coming in here but then and fairly recently I think it was 2017-2018 Clark was talking about this and he said well precision weighting ultimately comes down to task relevance and it was like oh no that's just homuncular we need a little man in there that's relevant that's not relevant and that can't be it and then what you see is the predictive processing framework is moving to trying to come up with a dynamical self-correcting way in which the trade-off relationships in the environment are being used in order to help trade-off relationships sorry in the way the organism is trying to survive in its environment that's more accurate can be made use of in an opponent processing fashion to help get a non-homuncular account of the precision weighting and we saw the two theories as they're converging they're coming right together and then it looks like they could actually help each other the relevance realization can help explicate what these trade-offs are like and the kind of opponent processing and I do mean explicate I think this is already implicit in the most cutting edge work within precision weighting and predictive processing and the predictive processing I sometimes like to use this analogy and don't push it too far but I think the predictive processing is basically giving us the Mendelian genetics of intelligence and relevance realization is giving us the Darwinian evolutionary theory and they need each other very deeply and there's a lot of things that the relevance realization theory can give to the predictive processing that it doesn't do as directly on its own these are things like helping to explain connections to a lot of phenomenological features, salience increase the linkage to theories of consciousness I can go into all of this in detail and so the two theories just feed together and they really strongly support each other I think a very strong advantage which is coming up now and this is something we're just starting to work on so I don't have too much to say that I'm very confident about but I'm confident in the intuition that I will have something to say which is that there's been a very strong critique of the predictive processing framework coming out of somebody who I hold in very high regard, Evan Thompson and it's complex and there's an ongoing debate I do think that Evan's main point if I can boil down a very complex argument I hope Evan will forgive me because I'm just trying to make sort of a promise here but make it a plausible promise is that the predictive processing can be sort of refuted with easy counter examples of pendulum clocks in training on a mantle piece and all kinds of so you get panpsychism all over the place if predictive of these predictive processings are supposed to be the model of the cognitive processing people despite that bullet but for me that seems implausible I think the answer which is almost forefronted in some ways in Evan's point is that predictive processing is not really as for easy claims to be it's not properly embodied, embedded, inactive and extended now I think the way out of that is by the connection, the integration with relevance realisation theory because relevance realisation theory grounds out in the bioeconomy of an organism and it grounds out in the fact that relevance is always relevant to a system that is auto poetic Montague put it very well different from computers in that they don't care about the information they process we care about the information we process and I propose to you we care about the information we process because we are fundamentally taking care of ourselves with our auto poetic beings and the theory of relevance realisation ultimately grounds out it says that relevance is deeper than representation it's deeper than inference it's deeper than logic it's actually running off the bioeconomics of the organism in terms of the auto poesis and so I think if predictive processing is integrated with relevance realisation it gets very clearly and explicitly and deeply integrated with embodiment and embeddedness and that would address I think many of the criticisms that are merging from Evan Thompson's critique so that's not only is there increased explanatory power outward in terms of explaining mental phenomena there's increased explanatory power in responding to deep criticism so that's sort of where we're at right now amazing many ways to go here and many great questions in the live chat that I know we'll be able to get to one footnote that we'll come to as well is connecting so many of the terms that we're using here which are being used in a technical sense scientific or even formal and mathematical to their everyday uses ranging from attention and relevance to all these different notions of prediction, explanation design, control and so forth so connecting these natural language terms to the way that they're used in cognitive sciences is a very rich area for the predictive processing architecture itself you absolutely conveyed it which is building from Bayesian formalisms around inference and uses a vertical nested model spatial metaphor it's not actually like a skyscraper or anything like that where higher levels are making ongoing predictions and lower levels are said to be feeding upwards observations and initially or at the most peripheral aspects of the agents we have what are most sensory like observations but those can become processed in a predictive framework step by step for example edge detection and vision, object recognition and so on and then very interesting how even at the very beginning asking how this came about you spoke to the way that Bre and Mark became involved in this ongoing line of research which I think speaks to the social and action of research and the distributed cognitive elements of development of theory and practice and then to bring to the very end that the relevance is and situational and that's kind of like the strange loop or like the quasi tautology like fitness in an evolutionary setting is situational and it is both the most mysterious and explanatory phenomena and it's the most circular phenomena and so relevance as well is so hyper situational that it can seem like it's going to evade generalization yet by combining computational architectures which are inspired by both advances in modern machine learning and artificial intelligence and deeper investigations of natural intelligences of diverse kinds ranging from ecological psychology and all these other fields that is what you pointed to in the title convergent solutions to the same problem and so anything else to add on predictive processing, relevance realization or there before we add more in? Yeah I mean there's an important point which is important to both Brett and Mark as well as myself I wanted to first of all try to lay out what you might call the scientific provenance of what we're talking about here and I want to also talk about a domain in which it offers a significant promise I won't go over the argument here I've done it in multiple places both in many different video series I won't even name them the point is in talks I've given at conferences the relevance realization machinery I think you can make a very plausibly strong case because the relevance realization is not cold calculation it's about caring it's about what grabs your attention it's about what calibrates your arousal it's about what binds you adaptively to the environment this is about the fundamental agent arena fittedness that makes you a cognitive agent it generates the primary affordances that are brought into your cognition through salience etc what I'm trying to get at is this is like a primordial connectedness of your agency and I think relevance realization and that kind of connectedness and constitution of your agency is very much what people are seeking when they're seeking this domain general feature of meaning in life it's a kind of connectedness the language that is used is mattering being connected they're having a fit between you and the world that you make a difference all of this language which is almost always metaphorical in fact using the word meaning is a metaphor we're taking something from propositions we're trying to talk about our relatedness to the environment and I think the relevance realization stop relevance realization when it is giving you what Marlo Ponte would call an optimal grip or in fact a meta optimal grip on the environment essentially affording that meaning in life that's one point I want to make another point then I'm going to bring the two points together so a bit of patience if you take a look at how relevance realization works and this is one of the advantages so I've done a lot of work in sort of insight and self-deception that have not been gone into as much in predictive processing you know because everybody's doing what has attracted their interest and what they find relevant to coin a phrase but what what I would say to you is the very processes that make us adaptively intelligent make us perennially prone to self-deceptive self-destructive behavior because we are intelligent by ignoring so much information we always run the risk that we are actually ignoring the information needed to solve the problem and you can catch this in the evolutionary dynamics of relevance realization when you have an aha moment when you go you have an insight and you realize oh I've been framing this we'll even sometimes use that word I've been looking at this the wrong way we don't mean visual perception I thought she was angry but it turns out she's actually afraid I've completely misinterpreted and you know what I'm talking about those insight moments and that's when the system is actually doing something like punctuated equilibrium like an evolution but the point is we don't always have insights we have the opposite of insight which is we get locked into blocked into misframing situations in a self-deceptive self-destructive manner we can do this individually and collectively echo chamber ring etc and therefore we need processes that ameliorate the self-deception without hamstringing the adaptive relevance realization that enhance insight enhance that connected that's meaning in life while reducing the foolishness the self-deceptive self-destructive behavior and I put it to you that across time and across cultures people have come up with ecologies of practices for doing that and that's what I would call wisdom it's a kind of rationally self-transcending rationality and therefore there are deep connections that can be made scientifically plausible and empirically testable some of the testing is already occurring about central aspects of what goes into a lot of falls under the name I don't like this term but we don't seem to have a better term for it spirituality where people are primarily in an oven nebulous fashion talking about wisdom insight the alteration of consciousness and somehow but if you think about it a wise person is a person that can go into a very messy complex situation and somehow zero in on the relevant information what really matters and appropriately fit themselves to that situation to make a difference to it in a way that brings about a movement towards flourishing for all parties involved and so an additional thing that the connection to relevance realisation provides that predictive processing doesn't have I'm not saying it couldn't have but I'm saying this just speeds it up and makes it a connection to a lot of the work that has been done by me and others collecting relevance realisation to meaning in life, altered states of consciousness insight flow wisdom and that is a powerful way to extend the explanatory scope of the theory while also increasing its existential and cultural relevance thank you very interesting there I want to pick up where you mentioned this may be towards what people can be seeking with questions around life's meaning, people's search for meaning and you use some terms there like things that matter and you've brought up in writing and in conversation about asking people questions around what matters and seeing that as an entry point into further investigation as well as like connection and fit and you also brought up from ecological psychology, the notion of affordances, capacities for action and optimal grip where we need to hold the sports implement enough to use it but it has to be flexible and adaptive and situational as well and this points in our language and thought to extensive spatial and visual and physical metaphors and figures of speech we talk about sketching ideas and even tangible aspects like comprehension and grasping ideas so we even use this kind of optimal grip language around cognitive ideas and this is being dovetailed in a new way with cognitive physics and the Bayesian mechanics that we'll be exploring more inactive inference so I think that's extremely interesting that even the formalisms are seeing convergence and similarity in terms of the physicality of things that might seem cognitive, mental, abstract, even spiritual and vice versa and I wanted to know where's action in this picture so I mean action for me is the um how do I say this without, I'm trying to not say too many things all at once there's an agent arena relationships set up affordances and that's a basic level of relevance realization a level I call participatory knowing where the environment the arena and the agent are co-shaping each other so that affordances open up and you don't make affordances they're not subjective, they're not in you they're not features of the environment they're real relations I say they're transjective relations that's what affordance is and we need to open up our ontology and this has been sort of a philosophical dimension of a lot of this we need to bring back the idea of real relations we've already been implicitly assuming them, adaptivity is the adaptivity of the great white shark in the shark no, put it in the Sahara Desert, it dies is it in the ocean, no if I put an elephant in the ocean it drowns so no, no, it's a real relationship a real relation of fittedness so we need to have an ontology of real relations and so the participatory knowing is and remember this is something to come out of philosophy I'm trying not to do too many arguments at once you don't have to know you know in order to know because if you do that you get into infinite regress and skepticism this is a kind of knowing that you don't know because it makes all other kinds of knowing possible for you it's a co-shaping by the physics by the biology even by the culture between you and the environment so that affordances open up so agent affordances are there perspectival knowing is layers up of recursive relevance realization in which the relevance is being translated as relevance to something like working memory and that's what salience is, I put it to you that salience is relevance to working memory I do not think we should be saying that salience is a property of objects or a subjective phenomena that again doesn't ultimately make ontological sense I'll just keep going perspectival knowing is knowing what it's like to be you here now in this state of mind in this situation and the situation is how the arena is made affording to you and your state of mind is how you are fitted to the arena but now you are salience landscaping it you're foregrounding and backgrounding you're sizing up some things you're leaving other things not as gestalted together more penumbra and it's dynamic and it's flowing you're always salience landscaping and the function of that salience landscaping is to select and I mean this like in an evolutionary sense to select which affordances are going to be enacted and that's your perspectival knowing when it makes that selection is it also taps into your procedural knowing your sets of skills your sensory motor routines for interacting with the environment your procedural knowledge this is your know-how this is your knowing how to do and then only within a subset of your knowing how you know how to make certain inferences you know how to make certain proposals and propositions etc propositional knowing so action is going up that stack that's my proposal it's the participatory knowing that is taking up by perspectival knowing that is taken up by procedural knowing that can be reflected and extended by propositional knowing one other aspect of action which I saw in the tension or compromise between thinking different threads and only having one self is action is like the eye of the needle because the embodied entity is able to only engage in one action selection at a time with potentially serious and even fatal consequence exactly exactly that that's that's important thank you and cognitive systems either as if as we model them or in their actuality can engage in all of these interesting phenomena that we've been exploring like counterfactuals planning all these different aspects that help one emulate simulate however we want to think about that but consider alternate pasts and presence which is like the perspective counterfactual and alternate futures which we must engage with to understand which sequences of actions we prefer to select and then in predictive processing the imperative becomes to among other things reduce our surprise about the world which can be an optimistic model we're reducing surprise about so it's not that we've expunged pragmatic value in fact we've balanced and as we'll explore formally soon balanced epistemic and pragmatic value through action selection by way of replacing this like monolithic reward pursuits and seeing curiosity and epistem as merely like a tributary to reward exactly and so that's another that's that's part of what I met with the the analogy I gave earlier about genetics and evolutionary theory because you know familiar people who initially encounter they say but I don't want to reduce all surprises I want and then what the typical predictive processing is oh but you'll you'll sometimes go for a short-term increase in surprise for long-term reduction and then what we're talking about is overall rate but the problem is you then need something that is selecting between different long longitudinal domains are processing because of course it's not always the case that you should prefer long-term over short-term right and there's right there isn't there aren't simple comparative algorithms for this you need dynamic trade-off relationships between short-term long-term between well I like some short-term I don't want all my short-term surprises to go away because if that was to go away then you know that would impact greatly on my long-term because I get locked in local minima and that's not the only explanation for periodicity but that's that's one kind of explanation you give and you go yeah right but you know that means you can't use a simple hill climbing you got to do other things what are you doing and you know there's people like Brett and I who think that you know there's things happening in the brain like self-organizing criticality at multiple levels of firing small world network organization multiple worlds multiple levels of wiring that are trying to constantly dynamically optimize these trade-off relationships and Chris Honey has talked about the fact that you can see the brain processing at multiple levels he actually argues and I don't know I don't know if I completely agree with this but you know he argues and I knew Chris was at U of T for a while he argues that there actually isn't memory there's just different speeds longitudinal rates at which information is being processed which is a really cool idea and so again what this does is as soon as you start to make those completely legitimate moves on behalf of predictive processing explaining things like curiosity like insight like wonder because wonder and curiosity aren't the same curiosity is there's something missing and you have a piece of information and you go looking to fill it in and get rid of your ignorance wondering as it was when you call yourself and your world into question in order to set yourself on to a developmental pathway it's an aspirational project and so predictive processing when it opens up its theoretical language in order to address these you can see how it starts to lean more into the kind of language that is afforded by recursive relevance realization theory great one closing note before we move to active inference is this question of does the fact that these models are reducing or bounding surprise mean I can't have any surprise and there's as you pointed out there's various ways to approach or balance that critique for example considering the long term or the rate of surprise and one of the most powerful responses is nested modeling which could be like the self belief I expect and prefer to hear from diverse and novel perspectives then yes one will be surprised in a little sense you know lower case surprise by every phoning but then in a deeper sense who we are can be connected with by hearing new information so it's absolutely a valid critique of the kernel of a single layer surprise minimizer just like you would have various trivial critiques of single-layer reward maximizers rather especially with nested architectures and considering embodied us we can see how higher level beliefs about the self which can include like the rhetorical narrative absolutely provide a context by which lower level surprises are accommodated if not surfed on top of just one thing I think we can make we can even make some significant progress and I did this in a series with Greg Enriquez and Christopher Master Pietro the elusive eye for integrating predictive processing and relevance realization models to give an account of the self you take you know the all the evidence that self reference which is really just self relevance is the glue of cognition and perception relevance realization and then you you can basically integrate that with howie and other people's I think it's howie and Michael their idea of a predictive processing model of the self and and they just go together very nicely and then you can talk about well there's relevance realization to the self and this is not just a homunculer thing and you can also point out the fact that cells are because we're mammalian primates and this is Greg's point is we learn to turn the arrow relevance the other way we are not just for our whole development how are things relevant for me we also develop how am I relevant to others and to the world and even to my culture and that again starts to connect up with meaning in life how am I connected to something bigger than myself this is the metaphor that people use see what I mean there's just so many things that can be brought into a convergent and powerful conversation once we get this integration going I'm not saying it's complete but I'm trying to at least make it plausible that is it has plausible promise to extend the scope and power of our scientific explanation into domains that it has hitherto been blocked from getting out that's heavy that's a massive cognitive entity that's groovy we're in the path of least action together let us move to active inference so in the paper you mentioned active inference one time though I think we can see it interwoven densely and the quotation in the paper was pointing to at all 2020's paper it has recently been argued that active inference and free energy minimization necessarily entail the minimization of redundancy and the maximization of degeneracy so first just by the way I just wanted to highlight redundancy and maximization just to give another example are drawn directly from cutting edge biology I keep wanting to show again and again that this is not a trivial claim of auto poesis and embodiment we are making use of fundamental biological principles this is again to give a basis for a reply to Evans I think everything that Evan does is brilliant right and so this is but this is again I just I'm sorry I take I have a deep respect for his critique and I just want to keep pointing out here's a possible response great so we have a few images and pieces of information about active inference which we will uncover through action we will reduce our surprise about what is under the gray box through action but first just wanted to hear your perspective what is active inference how did it come into play and when and what was it doing in your recent paper so for me and I know there's variation on this between people I see active inference within the predictive processing framework because I see the idea of the interplay between active inference and perception and that first of all Mark and Brett and many of the people that we cite are not bound to a strict propositional logical or sorry the logical manipulation of propositions when they use the word inference if they would do that then Evan really does have a purchase point because then they are being locked at the propositional level of processing they're not tapping into all the non propositional in a deep way but Mark for example very much inactive embodied he says we just use inference because that was the original term and it just stayed as a way of talking so the way I think about it is like this you have two ways of dealing with the error you can change your model or you can change the world and changing your model is broadly perception I just want to stop here and open that up when we say perception we don't mean what the empiricist gave us with the sense of perception we don't mean just a purely receptive bottom up taking of information because we also mean prediction right and so in a very real sense when I'm looking around this room I'm not detecting most of what I'm seeing I'm predicting it now that doesn't mean it's an illusion true predictions are true if you don't believe that you don't believe in science like you have to give up this sort of empiricist thing that unless I touched it somehow in detection it's not true no true predictions are true they are real relations they are true to the world right in an important way so I would put it to you that to get closer to what is meant by perception we have to take two terms that were separated in empiricism and then put a weird relationship through the enlightenment we have to actually integrate imagination and perception together you are constantly imago perceiving and percepto imagining your environment but that doesn't mean it's an illusion so first of all that point right and that opens so much up for explaining the imagination and distinguishing your imaginary from imaginal and all kinds of important and powerful things the other thing you can do is you can change the world right and what and I take it that active inference is a generic term for many different species of how we can change the world we can change the world by simply and we are constantly doing this by the way we can just slightly alter our perspective again the perspective of knowing and you're constantly doing this you're constantly just shifting your attention around so the world is disclosing itself to you in new ways right and you say well that's just a new way of seeing it's not just a new way of seeing you are you can't ever see all of one object you only see an aspect of it no matter what you're doing yet you think you're seeing the whole thing because what you're doing with this active inference is you are tracking and I really want to play on that sensory motor metaphor you are tracking the through line between all of these aspects that's one of the fundamental things that active inference means it means that I am look at how I'm doing it but notice how actually how unsurprised you are as I do this that is so active inference is not just I make stuff in the world it is that too I get the water and I drink it and so now I'm no longer thirsty it's even in how you are tracking which is an action a sensory motor action how you are tracking the through line in the multi-aspectuality of anything in the world because the amount of information that any object can present to you is actually combinatorially explosive for me and again I'm not denying the gross motor changes in the world of active inference I'm most interested in this cognitive navigation of the multi-aspectuality and multiple perspectives of the world such that we find the through line and here's the thing that's really intriguing Daniel the through line is not itself an aspect it's not itself a perspective it is something that runs through them and ties them together and that is the track I propose to you of active inference at its most primordial level and I think that golden thread is how the broader system changes through time which we model action as influencing how things change through time and the causal structure of the world but to pull a step back you pointed towards the two ways that a given agent can reduce the free energy of their generative model so this is not Gibbs free energy it's a chemical free energy that causes combustion this isn't Tesla tech free energy though I eagerly await the day when that synthesis is made this is an information theoretic quantity that bears resemblance to the free energy calculations used in chemistry and physics and other areas and we call that Bayesian mechanics and the process theory by which it occurs being active inference and so you pointed to two ways a given agent can reduce the free energy which bounds the surprise with attractable heuristic of their generative model and those are to change the model so change one's self and that can subsume an entire taxonomy of cognitive types and parameters as model they don't have to be a specific neuro anatomical region and those parameters that can be changing to reduce free energy of a generative model can include the continuum of perception learning as well as expectation preferences and also models of action and the other way in which a given entity can bound their free energy and seek to reduce it is through taking action in the world and so active inference is built around the action perception loop and it gives two meta affordances for reducing the free energy by way of minimizing surprise so I'm going to go one more level into detail with the active inference by revealing the first gray box so this is an image from a recent paper this year by Chris Thorne called an informal reconstruction of the free energy framework examining the conceptual problems that arise and the red text is an additional annotation that we've added into this excellent summary figure and we can read this figure as being a perhaps a kyros based historiography it's not a literal chronology of history but we can see it as part of the dialectic of research progression and it begins with the necessity of learning and developing which leads one to adopt a Bayesian inferential scheme my personal preference is that last names don't stay attached to terms but Bayesian inference broadly just means models that update as opposed to for example descriptive frequentist statistics right the necessity of heuristics for Bayesian inference comes very early because exact base is very hard to compute especially on large or unknown state spaces the necessity of heuristics takes a further developmental stage by the necessity of proxies that bound the exact Bayesian or even the heuristic or approximate Bayesian solution and so we can consider that like uncertainty reduction and then this is all pre-Fristain et al here Dion with the Bayesian brain work and also the work of Hinton and others in level 4 where we see free energy minimization free energy minimization is a computational heuristic that is already widely in use going back to Feynman and before to bound uncertainty with a tractable computational approach that includes reducing uncertainty and kind of being on this Pareto optimal frontier of not collapsing uncertainty too much or too little and in a way that's kind of like the energy and the enthalpy term in the Gibbs free energy world except we're in an informational world now so that's where we were at before the recent work in the last say 10 to 20 years of especially Carl Friston and collaborators and what Thornton points to as some of the key contributions that were made was this particularly a genetic interpretation of the lower four levels by suggesting that the figure could be separated from the ground that the generative model could be separated or partitioned from the generative process which is the niche and that entails a process theory which we call active inference which is suggesting a synthesis of inference broadly considered as you pointed to and action and finding all the contact points in the ways that inference and action come together just as the term active inference would suggest which means like action is used to improve inference you turn over the object inference is about action it's about selecting action it's about learning the consequences of action and how conditional they are on world states and other hidden state inferences and so the one level seven that we can bring in which in the active inference textbook is called the high road to active inference is the necessity of persistence and quite simply objects that aren't acting at least as if they're surviving will not be around to be observed later so whether it's a rock or whether it's a computer or whether it's a biological system it must at least be acting for the time that you're observing it or modeling it as if it is surviving in a potentially very complex statistical niche and so these six levels do an excellent job of describing how general statistical and learning frameworks like Bayesian statistics can become improved and elaborated upon through computational approaches which have only been developed for example the Bayesian graph approach and so on in the previous several decades in the agentic frame directly leads us to considerations of active inference and understanding how inference and self modeling including self modeling about action of the agent are a way that either we can either think of what agents are actually doing or at the very least we can model what agents are doing through active inference. Excellent. I'd like to I don't know this paper it looks like a paper I would like to have so I'll send you to send me a link later. Awesome. So then do you want to add any more or we can go one more level I mean I think I don't know if what I'm proposing is an additional level maybe it is or maybe it's a way of adding a dimension to the highest level or dimensions to all of at least maybe level seven six and five relevance realization and again we relevance realization predictive processing have common ancestors who of course was at U of T and deep learning was one of the explicit things we drew upon when we were trying to talk about relevance realization and remember the title of the paper relevance realization and the emerging framework in cognitive science we were trying to say look there are general strategies in different areas that look remarkably similar and they seem to be strategies for dealing with this central problem of relevance realization. We could see the organisms persistence or eco evolutionary fitness as being like the environment as the selector of relevance and then we can think the view from the inside the psychological or even experiential aspects of relevance to be a part of that broader system and we can imagine kind of two cases one is where the organisms experience and cognition around relevance is aligned with ecologically selective fitness at which case you're going to find that entity surviving and thriving you can also imagine where relevance leads us a skew and things become relevance whether it's repetitive behavior self-destructive behavior through information filtering all these cases in which our relevance realization is missing the mark it sins with respect to the ecological selection exactly and that's my point how the very process that makes you adaptive also makes you perennially susceptible to self-destructive self-destructive behavior you sin you miss the mark in a way that you can't realize you're doing and we calibrate and we learn as long as we are around right well what I meant is as long as you're in your sin that it itself encloses and the adaptivity the ability of an adaptive system to reduce perturbation means when you have these kinds of parasitic processing going on it hijacks that out activity and prevents your attempts to perturb it and break and help you break out of the inappropriate framing that's why you need very comprehensive ecologies of practices in order to cultivate wisdom it's why simply that you shouldn't do that same stupid thing you do in your romantic relationships is massively insufficient for not doing that same stupid thing you keep doing in your romantic relationships yes in the agentic framework with the ecological richness that is being brought in helps us contextualize and take a pluralistic approach towards the various kinds of interventions which might be suggested because interventions for different kinds of relevance you know mal alignments self-perceived other perceived in cases where there's conflict there they could include like neurochemical it could include an action oriented there's so many different ways in which this nexus of the entity engaging in inference and action can be modified and shaped that without integrative models to help us understand it can descend into kind of a disciplinary preference showoff yeah yeah the church of the level I call it the church of the level of analysis you believe everybody should be working at yes great so let's look at a second aspect of active inference which is its formal basis now there are many ongoing developments in the formalisms around active inference and the free energy principle and so this is barely even the tip of the iceberg but hopefully showing these formalisms which we explore in much greater detail in many of the other several hundred live streams that you'll find at the active inference institute just to show a few of the aspects of the formalism so people can triangulate and have a little bit of a sense for where is active inference between so this is from a paper of Dacosta at all from 2020 active inference on discrete state spaces a synthesis I'll make this image as large as I can there's two key aspects in this figure the top which we won't go into detail in is describing two types of inference which are on the bottom perceptual inference which is like a best fitting estimate for perception given the sensory data which are incoming which includes the blind spot and the low resolution outside of the fovea and the lack of color vision outside the fovea and all these other attributes of our generative model such that we even know that we're in a generative model and not for example within some sort of naive signal processing inward only view so I see this as just absolute slam dunk for predictive processing architectures broadly that we don't experience the blind spot we don't see it and that's normal because our salience has been tuned down to blurriness in that area and so on and in the top we have expected free energy minimization which in contrast with the variational free energy minimization expected free energy is prospective it's about selecting amongst policies sequences of actions, sequences of affordances that can be expected to result in different outcomes now the entity could be wrong could lead to the entity not existing but within the scope of the generative model expected free energy is how different policies about one's own world states about the Markov blanket here which are the set of perceptual and active nodes that mediate the interface between the entity and the niche the way that those are balanced in prospective inference is summarized on the top and then it's quite interesting to look at some of these special cases on the bottom and so we can think about these as like the poles in which the big tent of active inference is propped up on these are cases where when one parameter is minimized or maximized or doesn't exist there are various special cases that arise so just to give a few examples where there is no preference we see information gain maximization that's like a pure novelty search in contrast where we have no ambiguity or risk so we have complete information then we can engage in classical utility like calculations so that's kind of like when we don't have to learn anything or reduce our uncertainty then that's a special case in which pragmatic value alone can be pursued where pragmatic value has been taken off the table because all outcomes are equal because our preferences across them are flat then the inference can move into a more exploratory mode represented by like optimal surprise or information maximization and there are some deeper and more subtle links with maximum entropy principles but this is just to show a few of the kinds of formalisms that we explore in active inference which bring together many previous threads in perceptual inference as well as inference about action for example from cybernetics control theory and so on just a few notes and then one last piece that I think will hopefully connect us to thinking about communication culture and then also bridge to the questions that I've seen stack up in the live chat so these are from the active inference ontology which is a project at the active inference institute that we've been developing for well over a year now and what we have is a what started as a terms list but continues to develop to be much much more and we think of active inference in building competency and familiarity with learning and applying active inference in terms of quite literally skilled performance of a language the terms and how they're related constitute the active inference ontology and so two views that speak hopefully to the directions that this can help us pursue rigor as well as accessibility simple English definitions that link to other terms just like you'd see in a dictionary a dictionary doesn't really tell you what the word means it just has more words yet it still is considered a useful artifact by those who are learning and applying a language and so that's exactly what our terms and definitions provide which are natural language definitions for some of these terms which often straddle the line or confuse optimally between everyday usages and technical usages again surprise, attention all these different terms which you might hear day to day being used informally they sometimes are used both in an informal and quite technical sense in active inference and a second aspect that's been a great direction and really inclusive area of participation is with translating all of those terms into different natural languages so here we see English as well as equal partners with Spanish, Portuguese, French, Italian Czech, Russian, Hebrew, Persian and hopefully many more to come and so by treating this linguistic artifact as a collaborative and participatory project and being able to bring in natural languages that people have as well as bring in computer languages like our nomenclature and how we name computer variables we aim for something like a triple play where we can connect natural language descriptions of everyday settings or of contrived settings those everyday natural language descriptions can be visualized and they can also be rendered as executable models because we see that as where the promise of applying active inference can be most powerful when people are engaging in the skills and in the ways that feel natural and meaningful to them such as in community engaging in practices and that is supported and scaffolded and in feedback with computational systems that facilitate the communication and imagination around the discussion for example visualization and also provide in the same flow executable simulations so it's not just like there's people having a conversation at the chalkboard and then the modellers go take it into a different room and that's where the alchemy happens we think that by connecting the ontology with natural terms and computational representations that there will be new ways to be developing certain kinds of cognitive models and infrastructure together so you're actually trying to afford wisdom what does that mean or what does it look like? Well I mean insofar as you're helping people to apply Sophia into Phronesis these are the two words for wisdom in ancient greek Sophia is your knowledge of first principles and Phronesis is how you apply it within a particular context so that it is effective both giving you virtuosity and virtue you're trying to help people translate Sophia into Phronesis which is the sort of ability that is required for people to be genuinely wise with virtuosity and virtue I see a lot of what could be positive for research and technical areas which is it's not enough to have virtuosity alone someone could be very skilled at taking aim in a variety of different domains maybe one domain maybe across domains but without virtue then what you have skill and capacity without alignment maybe even the propositional belief that alignment doesn't exist or it's impossible it's useless or worthless or meaningless and then all that virtuosity to say it's for not would be an understatement because that person already made that claim but it could be worse than for not and so I want to say the opposite is also the case, virtue without virtuosity is something that undermines the respect that virtue is due because you have people sort of attempting virtue without having the capacity the virtuosity needed to genuinely empower it in the world and so that's also problematic you can get the reduction of virtue to following simplistic rules or merely signaling the presence of something without having the requisite virtuosity to bring it reliably into being how does what you've been working on points towards or help us come to terms with wise technologies so that's I mean for me I try to take a big part of my I wear three hats one is the scientist hat which I've been mostly wearing today in our conversation but I also wear a hat where I help to develop, teach curate, create, cultivate ecologies of practices as you said pluralism I wanted to strengthen the pluralism point you made for me to intervene at the time pluralism I mean one of the things that the framework I'm proposing with Brett and Mark is to note that a lot of relevance realization and hence predictive processing is taking place by making use of tradeoff relationships between different values, different properties of the environment I gave you the opponent processing with the detention you can think of the opponent processing within the autonomic nervous system for constantly calibrating so what you do is you get a system that is two systems that are biased in opposite directions one is trying to interpret everything as an opportunity another one is trying to interpret everything as safety and recuperation and they are pulling on each other like this constantly and that's how your level of arousal in a non homunculator fashion is constantly evolving its calibration so many examples of this often in multiple by dimensions where I mean tradeoff relationships that are also in tradeoff relationships with each other that's why I used the metaphor of an ecology if you think of ecology is all these checks and balances both at the same level and between levels of organization ecology there is no panacea practice no free lunch theorem etc etc there is no panacea practice you need to get a dynamic ecology of practices that optimizes for that multi-layered opponent processing and I try to use my knowledge as a scientist to look for places within traditional and emerging ecologies of practices where I can see those design principles being applied very well or being developed very well and then I help to develop those cultivate them help build those communities and then that takes me into my third hat which I didn't ever think I'd be wearing and I'm wearing which is what is it to build these communities so that the wisdom of individual cognition and the wisdom of distributed cognition are optimally aligned so that we have the best access within a community and then build those communities into communities of communities that's this project that I now find I'm actually engaged in a lot I do not claim to have any particular expertise for this it's just that I have fallen into it and I seem to be the person that is undertaking it so that's why I'm sort of hesitant I'm doing it with other people and so the answer to your question is I wear these three hats and I try to integrate them so that they mutually afford and constrain each other as much as I possibly can very interesting a lot to say there there's a few ways we can go I think let's bring up one topic and then we'll go in the end to the questions so if people have any questions please feel free to add them now and it's a term that's one of my personal favorites which is I like this term you introduced it I did not know this term until our previous conversation off camera where you introduced this term to me which I really liked it's a great term arising from work on insects and the work of Halegan Stygmergy as a mechanism of indirect coordination in which the trace left by an action in a medium stimulates subsequent actions Stygmergy enables complex coordinated activity without any need for planning control communication simultaneous presence or even mutual awareness I think it's a really fascinating and provocative idea because a lot of time is when we hear about collective intelligence or collective algorithms more broadly there are examples like flocks of birds which absolutely swarm and flock and they use visual coordination but they don't leave a trace on their aerial environment or you have the examples of traffic jams or of swarms or mobs of people in which case again there's jostling there's contact amongst entities but a trace is not left and so those one might think of as like a live conversation where no trace is recorded and that kind of synchronous coordination is absolutely an important component to consider but especially when we think about at least two incredibly important cases which are our digital environments and the various modifications that we make to them and our physical ecosystems and the kinds of modifications that are made to these niches neither of those cases are going to even be hinted at if we only consider non-stigmaergic collective behavior and collective wisdom and so in some recent work with colleagues we've continued exploration that has been existing since the earliest days of Wikipedia and before around digital coordination as digital stigmergy because although there are times for synchronous coordination for example in this conversation broadly through all kinds of modifications including this video as an artifact that could be rewatched or transcribed or analyzed in the future, broadly communication in the online setting can be said to result in complex coordinated activity without the need for planning control etc etc so just wanted to hear your thoughts on stigmergy yeah I've got three things I want to say about that because it's super cool for me where stigmergy really called out for me is how it affords cultural ratcheting a proposal that one of our primary capacities for adeptivity is cultural ratcheting which means we don't relearn from scratch we come in and culture has preserved previous learning and then we ratchet it up so we culturally ratchet in a very powerful way so that gives us a kind of adeptivity that is closed off most organisms there are a few species that you could plausibly say chimps, bonobos maybe some evidence from dolphins and elephants where there's something like a culture that fights this this is where human beings excel and stigmergy seems to fit right in there powerfully you want to note that what we're doing right now is so important because we're actually we are collapsing the boundary that used to be exist between live speech and writing because we have all the spontaneity of this but then it's preserved and it can enter into this meta dialogue with other videos where people are moving between the videos in dialogical fashion until you get this layering effect which is also working within the cultural ratcheting and that is very very interesting and needs to be explored so that's cool and then I wanted to connect what you just said with the issue that the neuroscientist Reed Montague talks about of mutual modeling and this is something that I think needs to be properly integrated into this overarching framework that's being built I've done some work especially with respect to aspects of consciousness, mutual modeling so it talks about that you face a certain kind of problem in communication it's an efficiency paradox so the idea is well one of the things I need to do to increase efficiency is to reduce acting at cross purposes so I reduce acting at cross purposes by having two or more systems talk more to each other they communicate more but another way of course by which I improve efficiency is I reduce the metabolic costs or whatever the costs of my problem solving and of course communication is very time and energy expensive and so he says notice how these and I'm doing this on purpose aren't a trade-off relationship the more we communicate the more we increase our costs and we lose efficiency the more we reduce communication we reduce our metabolic costs but we increase acting at cross purposes and our efficiency also goes down how do we solve this and then he gives the wonderful analogy before he moves to the brain and he says well consider an old married couple happily married some people stay married because they hate each other and that's the best way they can torture each other but I'm talking about people who stay married because they are they have almost like this telepathy with each other he didn't quite use this language but he uses the language of modeling but I think calling it generative modeling is no imposition on him and he says his example is straight like a straight relationship I'm just going to use that for ease of reference I'm not prioritizing that or making any cultural claim I'm just saying it's easier to talk because I can just quickly use common language right so the husband has a generative model of his wife and the wife has a generative model of the husband and they can actually coordinate their behavior without having to talk to each other very much so that's where he leaves it now what I argued is kinda that's right but you have to cycle because periodically they have to come back together and talk to each other to make sure the generative models aren't getting out of whack and then they can go off and do their coordinated uncommunicating behavior and they cycle and then I said you can actually see the brain doing this do the task focused where you're processing all the world all these different areas of the brain are coordinated without having to communicate much on the task and then you go to default mode what are they doing they're all centrally hubbed and checking in with each other because you just do all this sort of self-referential imagination making sure all the models are in sync and then you go back on the world sync up again all these trade-off relationships and how you resolve the paradox and then I thought mutual modeling and stigmergy they really belong together as underlying concepts now Montague's argument is you see the brain adopting this strategy you see the brain doing this where you've got areas are engaged in generative modeling of each other and think how easily this could fit into predictive processing framework because there's areas of the brain are predicting other areas but the idea and then you can point to evidence of synesthesia cross-modal enhancement of perception the fact that one supposedly specialized area like acoustics can actually take over sight in blind people who teach themselves echolocation synesthesia and a whole bunch of other things just pile up as evidence that the brain is adopting this strategy and so I thought this was an interesting thing because this notion in one sense points to cultural ratcheting and stigmergy and mutual modeling between people but it also points inward to mutual modeling within the brain and like I said I think this is so close already to a lot of what's going on in the framework we're talking about they're bringing this in and then again we could extend the explanation to things like synesthesia that you can get cross-modal enhancement that you can have one area of the brain supposedly dedicated to one sense modality picking up the ability for in rehabilitation etc etc etc well two layers of marital stigmergy the first would be the neuroanatomical stigmergy by which local synaptic changes and other sort of neuroarchitectural changes enable complex coordinated activity without the need for a central neuron so at the micro scale we have the physicality of learning and then there's the post-it note and there's the note on the door there's the modification of the environment which is stimulating subsequent activity and it doesn't need to be the recipe for dinner it can just be a note or an inside joke and so when we have a model that's integrative and allows us to situate advanced cognitive agents in their niche then we can actually bring in those kinds of modifications which is the root of stigmergy to mark stigma not in a taboo way but to mark the environment and the ways that that can elicit action whether it's again a post-it note on the door in the relationship or a comment on a video these are all ways that we can spur others to action or and so you can you can take the mutual modeling idea and then do George Herbert Mead you're in a baseball you're on a baseball team and you don't necessarily need to form a specific model of all the other players you might want to have that a bit but he said what you do is you create the generalized other you create a model will allow you to interact with all of them very effectively and so we can carry around we can have stackings we can have more specific mutual models for how I relate to my partner but then they are in right dynamic recursive relationship with a generalized other model and that also is a trace of others like in the guts of our cognitive modeling in a very very powerful way yes multi-player rituals as cultural ratcheting in other people to be involved in cumulative as well as re-experienced and reimagined cultural praxis and if there's a level to generalize out to perhaps it is one of them that is relevant such that we can have participatory ritual as one can say every single culture has but taking awareness and wisdom into the process of shaping our rituals so maybe they can be aligned with our preferences that's exactly right and there's a whole movement and I'm involved in it and I spoke to this a recent invited lecture I gave at Cambridge about ritual knowing and the connections between ritual the imaginal and being rational and so very much very much if you allow me I'll give you one instance of this and think about how this goes to updating and all the stuff we're talking about in predictive processing and reducing surprise you go into academics who are supposed to be the cream of the crop so a little bit of self-deprecation there and you give them convincing evidence locked salt rigorously tight arguments that they should start saving for their retirement now you allow them to raise whatever objections they have you meet them you all agree that you've convinced them that they should save for their retirement you go away you come back in six months none of them are saving for their retirement hyperbolic discounting etc etc hyperbolic discounting is an adaptive thing etc I won't go into all of that right but then what you do and there's like you know Hirschfield and other people all this what you do is you now do something else you get them and this is the imaginal not imaginary you get them to imagine their future self as a family member that they have a responsibility for and that they love and they want to give the best possible life to two things you find when you come back six months later the people that did the do the imagining start to save and the more vividly they can imagine the more they save so you overcome a kind of egocentric hyperbolic discounting through and in an imaginal it's basically a ritual this imaginal binding of yourself to your future self so that you get a more adaptive relationship it makes me think about variational free energy which is about the present and the past as religion religar a and then expected free energy as pre ligar a because we need to bind ourself to futures that haven't happened there's an aspect there's there's there's a you know there's a current agentic right way in which religio that's my preferred etymological orange for religion binding connectedness and there's also aspirational religion often does is try to try to find a sweet spot an optimal relationship between current and aspirational religio amazing let us look at some of these comments so any of them feel free to add comments or give a response so first Ali has written in deluses words relevance questions are how much and how in what cases and who questions as opposed to what is X questions from difference in repetition so what are the questions that bring us towards relevance as opposed to questions that make relevance obscured um that's a really interesting question I'm not I don't know much to lose I've read a little bit about delus and and that's I should read more delus because I'm a deep fan of Spinoza and he has a very interesting reading I have the book by delus on Spinoza but I I'm really hesitant to give much of an answer here because I don't know delus very well it doesn't have to be a delusian answer just what kinds of questions and and interrogate interrogative practices help us find relevance so um I think that I collapse a large argument built from spurber and Wilson and their work on a book entitled relevance by the way um at least within communication I think actually this is where I disagree with delus but this is where I hesitant because I don't know delus well so that's my hesitancy I think all of the questions are ways of trying to get a reformulation of what we find relevant they bring out they bring out different aspects bringing in different perspectives and point the arrow of relevance in different direction and what spurber and Wilson they took for they said okay you have the pragmatic aspect of all communication what's pragmatics pragmatics is what you convey beyond what you can say because I can't actually put into my words every all of the information I'm trying to convey to you because if I if I did that well if if and by the use of if I'm making use of not the logical if but the temporal and conditional if I in this case I'm referring to me and I'm referring to me in so far as I'm in this current situation um did that in that was referring to what of all the things I've spoken of I'm speaking of you know trying to give give the specific uh verbal form of everything I'm trying to convey and then for each one of those current what do I mean by the current you see what happens you get a number of ever expanding attempts to to translate the conveyance and you get a combinatorily explosive list of sentences that I can't say and you do this all the time when you drive up and you open your window and you say excuse me I'm out of gas you mean your car is out of gas you're not trying to be more flatulent you mean gas is actually standing for gasoline you mean you're not completely out of gas you have some for you can drive at least a few more minutes right and you know all of these things are at work because if you violate any one of those expectations you get pissed off if if they say to you there's a gas station at the corner and you drive to the corner and the gas station has been closed for 10 years you don't go well well he told me the truth because you right it was conveyed to you that you could make use of the gas station because you're out of gas I won't go on on that I'm just trying to give an intuitive example and so Grace gives these four maxims that we are trying to always follow in order to manage the conveyance so it doesn't become combinatorily explosive and it's the in its manner and formats and what is the quality so he gives three that are not relevance and the fourth one is relevance explicitly so and then Sperber and Wilson just basically argue all the other ones collapse into relevance like quality is tell the truth well not no you don't tell the truth you don't tell everything that's in your mind when you're speaking to somebody you select out of all of this in there a subset that's relevant so give the relevant truth the relevant manner etc etc so we always are doing this and I think questions actually operate right primarily at the pragmatic level of course they have semantic consequence but what we're constantly doing is asking which what are you conveying and what are the implicatures which are not the same thing as implications what are the relevant implicatures before I try and derive the implications of what you're saying that's what questions I think do now I want to also be clear about this that we need to be directing the questions in two directions sometimes we are probing are we as specializing framing the arena well in terms of is the world this so we check to see is the world disclosing itself to us such that we are solving our problems and flourishing but sometimes we also need to step back and look at the framing itself and ask is the framing actually misdirecting or misleading me so I think questions are always trying to reconfigure the fittedness and one of the things we and therefore they're trying to enhance relevance realization at the pragmatic level but we should always remember to direct the questions in at least these two directions excellent thank you alright here's a question from Brock can you give some examples of how depending on ignoring to realize relevance can prevent realization of meaning does this relate to reciprocal narrowing and opening yes yes totally and so Brock is familiar with my work and I'll just let me just quickly open those those final things up so other people are aware of what he's alluding to thank you Brock for doing that so then a notion of reciprocal narrowing is drawn from my friend and colleague Mark Lewis who has proposed a learning model of addiction rather than a disease model of emergencies like is it the disease model you have this thing it's infected you and you're trying to and it's debilitating you now I want to make something very clear Mark is not saying that there is not a neurochemical layer but he's trying to well he's trying to get an insight he's trying to get you to realize that the disease model makes all kinds of predictions that have been empirically disconfirmed I won't go through all of them but they have to do with the effect that we have that model we'll tend to think of the substance as in and of itself addictive you know cocaine is so addictive heroin is so addictive well you know there was lots of soldiers in Vietnam using opioids and when they came back the vast majority of them stopped without having to have any special treatment they stopped using opioids but opioids are so addictive well what's different presumably this chemical structure of heroin in Vietnam and the United States is the same what is different the Asian arena relationship is different you're a soldier in a traumatic circumstance in Vietnam the United States you're a citizen and you're not and of course that caused all kinds of other problems for these people because the agent arena relationships are so different here's what Mark proposes so I'll use alcohol because it's one people can easily relate to I have a very challenging arena and I'm stressed so I take some alcohol and it reduces the stress but it actually limits my cognitive flexibility so my problem-solving abilities go down so the options available to me in the world goes down so the world narrows as the world narrows it becomes more stressful to me so now I want to take more alcohol to keep the stress away but that's now impairing my cognitive flexibility you see what's happening the world as an arena is narrowing and my capacity as an agent is narrowing so you've got reciprocal narrowing until I can't be any other than I am and there is no future for me at all other than this moment of taking the drug and that's addiction it's reciprocal narrowing now I was literally I was literally at lunch with Mark Lewis and I said but Mark if there's reciprocal narrowing there has to be reciprocal opening and he went oh you're right and I talked about how Plato talks about this in the model of what's called anagage his model in the Republic of how you ascend out of the cave into seeing the real world reciprocal opening and so I think when we engage in ignoring relevant information we will often very readily fall prey to a very powerful kind of reciprocal narrowing so let's let's go back to the example I used you don't have the insight you think she's angry when she's actually afraid you don't have the insight and so you push on her and you only take evidence confirming confirmation bias that she's angry in to your model and then you only act towards her as if she's and she's going to start probably almost like a self-fulfilling prophecy respond more and more because your anger it right you start to act towards her in a way that gets increasingly aggressive and you reciprocally narrow and then it gets harder and harder for you to see that she's not angry and it gets harder for her not to communicate to you in a way that you have to interpret as anger because when she tries to give you something else you ignore it in zero and you have been in arguments and fights like that at some point in your life with somebody that you don't want to be severed from but that kind of thing can sever the connectedness and when you're at the end of your life you don't really care how many objects you've manipulated you don't care how many french fries you eat you care about the relationships you had and how deep they are and if you didn't pay attention at that time you lost a relationship and severed a connectedness that enhances your meaning in life you fell into reciprocal narrowing it severed a relationship and your connectedness mattering to another human being has been lost there's no more interpersonal gravity and the relationship does not matter alright I'm going to read just a few non-question comments then get to a question in these last minutes so Griswold wrote ants hold hands in the river of information to flow along together that's wonderful Ian wrote vision and fusion communities coupling and decoupling and imagination as allostatic remembering of the future wow nice alright here's one more question this is from Dave so Dave says have you found I don't know if this is the correct way to pronounce the name but guiles falconiers work on mental spaces and conceptual fusion useful in explaining and improving the evolution of concrete mental activities we can also see this work on generalized space navigation with chris fields and mike levin and many others so whether you're familiar with that researcher or not how does this idea of cognitive spaces help us work in physical and social spaces for me again and this how the connection to relevance realization and its connection to biology and evolutionary modeling is really helpful here so give me a moment so one of the ways in which evolution selects is it adapts it doesn't have to generate an entirely new mechanism so let me give you a concrete example it's one of my favorite ones what I'm doing with my tongue right now is speaking of course tongues did not evolve for speech because we have so many organisms that have tongues and they aren't capable of speech but what does the tongue do is it's an elasticate so it has to be highly flexible it has to be a poison and calorie detector so it has to have lots of nerve endings and it has to be in the air passageway because that's how we evolved and so it's almost pre-designed as some people say pre-adapted for speech and so it gets exacted into speech I think there's growing evidence look at Barbara Tversky's book Mind in Motion Anderson into Hand in Others the brain is an exactive process the brain is constantly exacting machinery that emerged for one arena set of actions for another specifically it looks like the set of abilities that we generate for navigating physical space is exacted into moving around in conceptual space the fact that we even calling it conceptual space and we talk about things being higher and lower and we move et cetera it's just totally impregnated with navigational language and we talk about orientation and finding our way or losing our way can you follow this argument that's tracking language can you follow this argument you know I lost the thread again and again and again and so it helps to explain really weird stuff it's otherwise not well explained okay cerebellum more neurons than your neocortex often not included in FMRIs by the way which is a mistake many FMRIs I think should be removed from their findings because the cerebellum is not included and we are now getting overwhelming evidence for the cerebellum cortex loop and by excluding the cerebellum from our FMRIs we are smuggling in a confound that has not been properly controlled for just a general point there when people are sitting what's the cerebellum for sensory motor balance when people are sitting absolutely still closed perfectly stable in meditation their cerebellum is firing like crazy why they're not doing anything they're not moving they're in a position where they can be deeply comfortable because they are aligned with gravity and like they're stable stable their eyes will presumably because they are navigating and they're trying to balance complex contingencies of attention and move through the experiential space in a powerful way and the cerebellum when I started talking about this it was controversial now it's approaching consensus position that the cerebellum has been exacted for these kinds of functions so I think the idea that I've only briefly looked at Fukunie's work when I was doing work on metaphor and I didn't find it particularly helpful for metaphor so I didn't go into it more deeply but the idea that we are doing something like mapping between spaces I think this is I think this just plugs into the exactive nature of our cognition that it is a form of rapid cognitive evolution awesome well in the final minutes do you have any words of wisdom any thoughts where do we go from here where do people go from here well it depends I mean you can go theoretically in which you can look up my publications and Mark is coming he got an amazing grant and other grants and so he's coming to U of T he and I are going to do work Brett and I are doing work the three of us are going to continue work so look for that Gary and I and I have one written on integrating relevant realization and attachment theory in terms of attachment is actually an optimal gripping practice between the child and the mother frequently the mother the primary caregiver I should say and so you can look for all of that for those of you who are interested more you can look at the stuff I published on wisdom some I don't think I wrote any wisdom papers on my own they're with other people if you're interested in mindfulness I've got two publications one on an Oxford book an anthology edited by Ymir Raz on hypnosis and meditation and more recent one which I recommend in the Routledge Handbook on the philosophy of meditation where I talk about all of this how this all works out in terms of attention and meditation and contemplation why they're not the same thing why they're an opponent processing relationship why that's conducive to insight and more systemic and systematic insight etc if you're interested in learning about the meeting crisis you can go to my YouTube series Awakening from the Meeting Crisis and follow that argument all the way through I have a new series coming out which is about trying to tap into how we can afford these kinds of flowing emergent conversations that take us beyond where we can go in an individual cognition and empower us and it's called After Socrates where I'm tracing out what's going on in dialectic into the logos that's coming out this year probably October, November Rebel Wisdom David Fuller they're making a documentary film about me and my work that's coming out also October November of this year so that's a bunch of things you can look for and look at amazing well thank you very much for joining you're always welcome to come back we really appreciate it thank you so much Daniel and thank you so much for I mean you there's a couple things maybe if you sent me your slides or something from this that would be great because there's a couple of papers that I want to read now and you is excellent it was genuinely back and forth and I appreciated your insight and your perspective a lot and also your graciousness thank you thank you very much thank you John till next time thank you Cara thank you everyone