 Hello, welcome to the Active Inference Livestream. This is Active Inference Livestream number 19.0 on April 1st, 2021. Welcome to the Active Inference Lab. We are a participatory online lab that is communicating, learning and practicing applied to Active Inference. You can find us at our links here. This is recorded in an archived livestream so please provide us with feedback so that we can improve on our work. All backgrounds and perspectives are welcome and we'll be following good video etiquette for livestreams as outlined in this checklist. This is the schedule that we've had so far for 2021 and the upcoming two discussions on April 6th and the 13th will be number 19.1 and 0.2 on this paper that we're gonna be discussing today. So right now in 19.0, the goal is to set the context and give a little background for 19.1 and 19.2 and people who are just learning about the paper which is deeply felt affect the emergence of valence in deep Active Inference by Hesp, Smith, Parr, Alan, Friston and Rampstead in 2021. And this video is definitely just an introduction to the ideas. It's meant to be a two-way on-off ramp from Active Inference. So exposing the Active Inference community to ideas outside of Active Inference and people who might be interested in some of the bigger ideas connecting them to the Active Inference implementations. And we have a bunch of sections planned for today in 19. So thanks a lot to Steven and to Blue for helping with the slides and for coming on today. So maybe we can just introduce ourselves briefly and then we can go on to the rest of the slides we have prepared. I'm Daniel. I'm a postdoc in California and I'll pass to Steven. Hello, I'm Steven. I'm in Toronto. I'm doing a practice-based PhD around social topographies and community-based development. And I'll pass it over to Blue. Hi, I'm Blue Knight. I am an independent research consultant based out of New Mexico. Cool. Well, I think this was a fun paper for us to read and we can just go right into what were some of the big points of the paper and the background ideas before we go through the figure and the model, which will take up a lot of the time. The paper is called Deeply Felt Affect, the Emergence of Valence in Deep Active Inference. And it was published in the Journal Neural Computation in 2021. And so I'll read the first one and then either of you can give a thought on that. In this letter, meaning paper, we demonstrate that hierarchical, deep Bayesian networks solved using active inference afford a principled formulation of emotional valence, building on both the work mentioned above as well as prior work on other emotional phenomena within the active inference framework. Yeah, so that's just giving us that insight into this idea of valence. So we'll be speaking about that more, but that's an important part of this paper. I'll say the next one. Our hypothesis is that emotional valence can be formalized as a state of self that is inferred on the basis of fluctuations in the estimated confidence or precision an agent has in her generative model of the world that informs her decisions. Cool. So we're going to be connecting some of the decision-making aspects of agents and inference aspects of agents to emotional valence and they're going to demonstrate it. So those are the goals and the claims that they set out in the paper. Any thoughts, Blue, or do you want to go to big questions and maybe introduce a big question? I'll introduce the big questions. All right, go for it. So the big questions of this paper were how can we make more active inference models or agents that are expressive, powerful, realistic and interpretable? What are the implications of this? Active inference formalizes our survival and procreation in terms of a single imperative to minimize the divergence between observed outcomes and phenotypically expected, i.e. preferred outcomes under a generative model that is fine-tuned over phylogeny and ontogeny. Thoughts on this? Well, the first one, this to me is about just developing the active inference framework to capture more of the patterns that we see in nature or in designed systems. So how do we make systems that plan for multiple time steps? That's deep active inference. How do we make agents that interface with the environment? So that's sort of a big move overall and then this is going to be zooming in on the valence and some of those affective aspects. That's on the first point. Then the second, this is a quote from the paper and it says something that active inference does. It formalizes our survival and procreation not in terms of evolutionary fitness. Again, you could agree or disagree with the authors or think it's consistent with fitness or not, but claiming that the imperative is to minimize a specific kind of a divergence. And so what are the implications of that? If you think, okay, well, if that's the imperative then where's the situation where we've been acting like there's a different imperative and is that consistent or inconsistent with this? And then the last quote, and then someone could give a thought is a quote where they say we conclude with the discussion of the implications of this work such as the relationship between implicit metacognition and affect connections to reinforcement learning and future empirical directions. So what does active inference or this model say about cognition or metacognition and reinforcement learning? Yeah, if I could add something, I think it really opens up and a lot of the philosophical questions and starts to give a modeling interpretation. So how the body comes in, so the emotion starts to tie in to a way that the body comes in and an active inference starts to become modelable and it starts to also show how affect is somehow connected to all cognition and all processes. So it's not some weird artifact of shame which might be more of a Freudian trap, but it's actually an integration of the cybernetic process all the way through in some way. Very nice. It's also captured a little bit on this slide which is recycled from some 2020 vintage actinth which is asking about how active inference is gonna be saying something about the relationship between agents in the world. And we're imagining an agent, not a one step tic-tac-toe player, but we're thinking about an agent that's imagining through deep time and they have imagination or the capacity to simulate and then they have affect and they can also predict their affect and have precision and so on. So here's sort of the drama masks for the positive and negative affect and then there's sort of different world trajectories that are being estimated through time that lead to positive or negative sentiment and that gets around some of the sort of basic considerations like in the paper they say, well, wouldn't the agent as they meet their demise wouldn't they have precision over their sensory states? And the answer is like, yes, but it's not part of a long-term trajectory that involves desired preferences. So by appealing to model nesting, some of the sort of single layer, not paradoxes, but just scenarios that models might fall into, like always going with what was known before never going what was known before these can be resolved just with empirical modeling. Interesting. One reason why it matters and we'll come back to this when we quote the authors later is that these kinds of richer models lead more closely to experimentally tractable and unique predictions of active inference because if the only thing the model predicts is the reaction time, that might fit the distribution better than some other model. Now, if it can also fit the reaction time and the EEG and also the self-report, it's starting to become a very rich multimodal and multi-measurement way of talking about behavior which is what a good instrument should do. So this is helping us come closer through these contact points with other big important areas of research like reinforcement learning and autonomous computer agents and just inference agents of all kinds, especially ones with action. But then maybe some of these other areas that we'll bring into the discussion because if we can have a scale for a framework that does apply to cognitive agents and then we frame it in terms of the input and the output and the actions and the policy, the generative model, all of these terms, maybe some of these other systems like online teams might be able to also be said to have at least computationally affective charge or precision or different things like that. So we kind of talked a little bit about that in our 2020 paper for online teams. Any thoughts on sort of the bigger topics that this paper is up against before we jump into the abstract and the figures and everything? If I could just say one thing I think that's very interesting is that it opens up the realities of trying to bring in the mind, the body and the environment. And there's been this question, well, how do you do that? And why didn't we do that before? It was partly because we wanted to be able to measure things and lock things down. And the question of how do you bring this whole dynamic in? How could you possibly get a handle on all of that? Well, emotion seems to actually make a lot of sense in terms of, or let's say valence, should say valence more broadly, makes sense in terms of how it could be calibrating the more conscious levels of that process. So I think that has implications for scaling psychology and bringing consciousness back into psychology and helping to really move into a ecological and ecosystem approach to psychology. Lou, any thoughts or do you wanna start with the first part of the abstract? I'll hold my thoughts. I have some but I don't wanna ruin the surprise of the paper. So let's start off on the abstract. Agreed, fun times. All right, does anyone wanna start with the first part of the abstract? I'll take it. Yep. The positive negative access of emotional valence has long been recognized as fundamental to adaptive behavior, but its origin and underlying function have largely eluded formal theorizing and computational modeling. Using deep active inference, a hierarchical inference scheme that rests on inverting the model of how sensory data are generated, we developed a principal Bayesian model of emotional valence. This formulation asserts that agents in further valence state based on the expected precision of their action model, an internal estimate of overall model fitness, which the authors term subjective fitness. This index of subjective fitness can be estimated within any environment and exploits the domain generality of, sorry, my screen is in the way. Second order beliefs, which are beliefs about beliefs there. Nice. So in this first part of the abstract, kind of priming what this whole research agenda is about, I'm seeing a few threads. One is about fitting good models, overall model fitness, especially with a subjective twist. So that's another kind of a bonus that it's actually not just objective model fitness, it's subjective. It's about good fitting models with this recognition that that's personalized. So that's cool thread. Then there's the first words, which are about the positive and negative aspect of emotional valence, which is despite the richness and the amount of experiences and all the diversity of altered states of consciousness and adjectives, there's also kind of this approach versus recede binary. And this is, as they're saying, it's fundamental. And we see it in living forms, like go, no, go signaling, and we also see it in computational agents. And so this is putting a lot of different kinds of systems that have very nuanced maybe behavior in their outputs, but at a top level, they act as if they have a sort of go, no, go that in a way helps them be adaptive in their own niche. And then the way that they're gonna go about that is actually with a second level or second order cybernetics model, which is by enabling belief about belief, maybe some of these gaps between systems can be bridged. So interesting research questions that are not always brought out because the abstract is so reduced and they have to say what they did do, but for those who are interested in those questions, that's good to know. All right, I'll read this one and then anyone can give a thought. We show how maintaining internal valence representations allow the ensuing effective agents to optimize confidence in active selection, action selection preemptively. Valence representations can in turn be optimized by leveraging the base optimal updating term for subjective fitness, which we label affective charge or AC. AC tracks changes in fitness estimates and lends a sign to otherwise unsigned divergences between predictions and outcomes. That's pretty interesting. We simulate the resulting affective inference by subjecting an in silico in the computer affective agent to a teammates paradigm requiring context learning followed by context reversal. This formulation of affective inference offers a principled account of the link between affect, mental action and implicit metacognition. It characterizes how a deep biological system can offer its affective state and reduce uncertainty about such inferences through internal action, i.e. top-down modulation of priors that underwrite confidence. So this for me was a big surprise of the paper. Like the valence and the relationship with valence and subjective fitness or precision accuracy. This was just a really different take compared to the last paper that we did just like on valence was very much positive or negative in terms of feeling or emotion where this is like really positive or negative just based on like surprise essentially like what if my model doesn't match my expectations like or the outcomes if my expectations don't match my outcomes that is negative valence. Which that was a kind of surprising take on this for me compared to the last valence paper that we did. And the other thing is really this the way that they've modeled this is totally like a perfect example as I talk about in the discussion of the Bayesian brain and why Bayesian statistics and updating like especially when they talk well I'll hold my thought for them for that time but really this was just an exemplary paper for that for me. Nice. Yeah. One thing that came up for me there is it talks about representations again and so this idea that internal valence representation so suddenly we've got a way to bring representations back into what had become quite a non-representational modeling approach and that's really interesting because it's like normally you think of representations previously like the most concrete is the thought then there's the body and there was this fuzzy thing called emotions which somehow was there and axiology and all this stuff but now it's like, well wait you've got an internal valence representation which could inform the body and the cybernetic thinking metacognition process so it gives a way in for representations but it's not representations as we normally maybe approach that term so that's quite cool. Cool. All right. Third abstract, who would like to read it? Okay, I'll read this one. Thus we demonstrate the potential of active inference to provide a formal and computationally tractable account of affect. Our demonstration of the face validity and potential utility of this formulation represents the first step within a larger research program. Next, this model can be leveraged to test the hypothesized role of valence by fitting the model to behavioral and neuronal responses. Nice. So they set up the problem by being about using higher order cognition, this second level modeling to approach some of the interesting dynamics of agents like their apparent affective charge influencing their behavioral decisions in the present moment based upon inference about the future and they're gonna demonstrate a simulation. There's gonna be kind of a model layer and then there's gonna be a simulation that goes down into the TMAs and a mouse and so there's sort of two levels of generality that we're gonna go through here. They start the paper with the qualitative but still rigorous level of generality which is just talking about action in different systems and multi-scale biology and the need to integrate Bayesian approaches into how we think about evolution and development, ecology and so on. And then they turn directly to the methods and perform this four step walkthrough that will be somewhat familiar to those who have seen model stream one with Ryan Smith and Christopher White where they also step through the construction of this model but it's something that as we'll probably see soon it's just always worth it to kind of build up this model incrementally. Then they with this implicit metacognitive model that step four of their models elaboration they discuss a few kinds of important key terms that are like bridges between previous computational psychological models and their active inference model and then they implement a simulation with a rat in a TMAs which we'll discuss as well and then they bring up all these very interesting questions towards the discussion section of the paper some of which we've excerpted and conclude with a discussion on the future empirical directions. So before we go into the affect and valence model simulation and implications any thoughts on the roadmap? All right, let's go to affect and valence. This is quoting from their paper so definitely read it to see everything that they say but they refer to classical constructs of valence. So maybe here just representing the classical valence construct on a kind of clock chart. What would either of you think or say about valence or what motivates the study of valence this way or overall? So I just had a comment and it's based on something that the authors had said about valence. So here we've got the arousal and like high and low arousal on one axis and positive and negative on the other axis but the authors had mentioned that some people see valence as like an array of positive like it's a positive and negative or two separate axes of valence and I thought that that was very interesting whereas like it's not a binary choice positive and negative but there's a range of positive and there's a range of negative and ambivalence to like contain some degree of both. I thought that that was an interesting take on valence. I agree. I've thought about before. Cool and they bring up good scholarship with bringing up a uni-dimensional view. So that would be like, yes, there's like a thermometer and it's kind of just like it goes up or down. It's a number that has to be strictly ordered and less than and greater than maybe even linearly correlated with a hormone or a chemical or a neural firing in some region or a pattern of activity then maybe there's multiple dimensions. Maybe there's like as here an arousal and it's a high to low arousal and high to low valence positive negative. Maybe it's multi-dimensional. So what's cool about the active inference framework is we could test the computational roles of these kinds of variables and ask, okay, this is fitting the data better, but is it more parameters? Is this worth the trade-off that we're getting here? So we're kind of taking this single-dimensional model and thinking about it with a family of other models where we could test the influence of other factors and see if such patterns are detectable in the data or ask what kind of behavioral settings or groups or questionnaires or whatever it might be would elicit a differentiation on another hypothesized axis. Yeah, and just one thing that goes with that is it sort of shows you the current ways to get at this is kind of through self-reported emotions and using this kind of graph and actually views this at all with our community with disabilities for reporting back on their experiences but you're essentially still getting this emotion as a self-report and trying to track back using sort of assumptions about how that plots with other self-reported emotions and whereas the, and this is also used a lot in market research for marketing and advertising this tool. So, but actually in this approach here, they've got a way to access this without just getting self-reported words. So, this I think shows how it has, you know, this is an area that's got a lot of application potentially. Cool. Here is a slide just upon differentiation in English between affect and effect. So, not to split hairs or to introduce words that they didn't use in the paper like effect but what would be the point on this Steven? Well, you've got that just to give the sense that it is often confusing those two terms. So, if we have the person, these two people on the left. So, you've got the person X as experiences and effect at some sort of, you know, relationship to person Y and it's created some affect. Now, that could be anger if it was reported later and they said what was their emotional state and this affect contributes to a policy selection where person X interacts with person Y and pushes them into the pool. So, you can see that, okay, so that the effect of this is that that person Y ends up in the pool. So, there's an act and there's a cause effect type of linear type of mechanical action that happens. And of course, then there can be additional affects on both the parties and effect resulting from this particularly if the person was to drown or something depends on the scenario. So, I think it just shows that there's a kind of one's more dynamical and relational and one is more objective, I suppose you could say and cause and effect related. Sounds good. And here is from a previous discussion, I think number Act M for 11 where we talked also about affect and about trajectories through affect space. So, whether or not it's like how things quote really are it might be a behavioral index or summary statistics of someone's online activity even if you're not directly measuring them or getting their self report and there can be trajectories through this space. Now, one topic that a sort of detail that's a little bit special for active inference that doesn't get emphasized a ton is that basically the certainty in the model is one in the same thing as having better affect better affective charge increasing certainty and high certainty in the model are taking the place in a reward centric of highest expected, for example, value. And it's one major difference between active inference approach to thinking about economics or decision-making versus a sort of just like classical maximum likelihood reward maximization. So in these active inference frameworks they're not modeling a happy to sad variable and calling that affective charge. They're actually modeling a precision variable and then saying that when things are more precise and are getting more precise there is low negative affect. So it's kind of like positive and then anxiety like negative affect arises as a function of uncertainty in the model. And here's just some words and some thoughts that we had put on anxiety from a previous one. But any thoughts on this about the sort of integration of valence and certainty rather than valence and expected average? What that means or how that's different than other models that others have seen? Yeah, well, I think that does have quite important implications because it's now prospective. It's kind of what was the contextual expectation on where things would go as opposed to just it being this kind of static expectation on the moment itself and it's all collapsed down into some sort of reductive. There's the idea that the person's valence is dynamic in relation to the expectation. But then there's also the precision at which that expectation then resolved or didn't resolve. So you may expect things to go badly and then things went worse than you expected or it went better than you expected but it still might have gone badly but it didn't go as bad. So it gives you that much more, well, I don't know, what is the word realistic? I don't know what term you'd use for that Daniel, but yeah, that question. Cool, blue. So I can't remember which paper it was exactly but it was talking about anxiety. It was one that we did on the live stream talking about anxiety as like a fixation on the future. And it talks about depression as a fixation on the past. Was it the big five? I feel like it was the big five paper. But that is just an interesting and kind of different perspective on anxiety as relative to the valence that's presented here. Cool. And the question of how to map model parameters to experiences that we have which are encultured and which are relating to our individual kind of qualia, that's a big jump too. So we shouldn't just instantly also paper over the computational psychiatric gap but that's what we're gonna unpack is what does the model really say and then what is that gonna help us do or rethink? One note is that they wrote that they are referring to their model distincting it from just sort of active inference more broadly as affective inference because it sort of has the active in there but it's also has affect in there. And they're saying there's what makes it different than just general active inference. And one of that one attribute that they apply here is deep. So that's meaning deep through time or potentially also using deep learning or deep neural network approaches. We had, I think Christopher White or someone it all blurs together but somebody gave a few senses of deep active inference. And then they're also modeling affect. And so that's why they call it affective inference. So it's deep active inference. So if you know what active inference kind of is about they're taking that skeletal framework and extending it through time and adding an affective parameter or a set of interacting variables. Way to think about and model the kinds of things that people study affect in that's what they want to study with this new development. So that's kind of what I think they're doing in this paper. And it's also a little more general than just affect because as they say it can generalize to potentially other higher level explanations for conditional dependencies between higher level parameters and how they reflect behavioral outcomes. So that's a pretty general statement that's very important though. It's saying that it could be possible to have other higher hidden states that cognitive ideas let's say. And those don't have to be just the positive or negative approach or recede that they model here. It could be something that's a lot more subtle or symbolic or categorical rather than this is more of like foraging go no go. So this is a initial foray into this kind of deep and hyper parameter reaching back down affective in that sense. And it's tied to the affect rather than to the, for example, cognitive work. But it's as they say implicitly metacognitive. And Stephen, as you said, it's not that affect is latent, latent, latent and then like strikes into the scene. It's actually that there's this sort of fundamental cybernetic role for affect regulation. So it's not a red flag mode that is explosive or something. It's something that's intrinsic like metabolism. Okay. They bring up a lot of previous work and some limitations of the model. So check out the paper more if you wanna learn about, for example, if you think, well, that's a very narrow model definition of valence. They might bring it up in their paper. So check that out, but let's just get right into what the model is. And there's gonna be sort of two acts coming up here. There's gonna be the model, which is just the mathematics of what they lay out and the formalisms they lay out. And then we're gonna go into the simulation with the mouse or the rat or the ant or whatever in the teammates. So the first is gonna be general. And the second is gonna go into a simulation that they're gonna get results from as well. So we're gonna walk through kind of four clicks in the elaboration incrementally as they write on the active inference model, okay? So fun stuff. Thanks everyone also for joining live. So here we are in figure one. Who would like to go first? Maybe Blue, you annotated this really well. So what did you see in figure one with model one and model two? Sure. So model one is the generative model of perception, which we've seen before, where S is hidden state. O stands for observations. D is prior belief. A is this likelihood mapping or like a probability distribution. And then tau is time, which we've seen before. So in M one, there's no time. This is the first step of a generative model and it's just perception. So it's like sensory input and the posterior over hidden states is all that's really taken into the model of perception. And then the M two, so as the models, as we go through these M one, M two, M three and M four in the next few slides, there's an increasing layer of complexity as we transition from M one to M two to M three and then M four with M four being the most complex. And so in M two, which is the generative model of anticipation, the idea of perception is then extended forward and backward into the past if you wanna go there using a transition matrix, which is this B sub T for, to represent the hidden state. So it's just an extension of perception essentially and it includes the anticipation, what you expect to happen, as opposed to the just the sensory input, which is just what's present in M one. So it's anticipating hidden states over time M two. Nice, awesome. Good explanation. And we can look at some of their, or anything else to add actually or ask your Stephen on figure one. No, it was really helpful. I mean, the only thing I might add is that, as you mentioned that, is it's almost like the predicted input in that funny sort of way. It's like, so we got what was, what's the predictions of what the sensory input is if I could sort of somehow infer that and that's that dynamical. So I think this is really helpful though to go through it this way. Yep, and to give a note on that, a lot of times when people think statistics, they think descriptive statistics. Like we're gonna take the big data set, the big image data set and we're gonna describe it with a summary statistic. And that does map onto this topology, like observations get mapped through likelihood onto the most optimal state of the world. But where generative Bayesian modeling comes into play is that that model that's inferred, that hidden state, what is the real temperature or what is the real average height of the classroom? It can also be used to generate and to combine with other models to generate testable distinctions can generate data. And then this duality, where you can like take a little bit or a ton of data and then update your prior already using the Bayesian terminology and then use those prior estimates to generate more data. That is like expectation maximization or EM optimization or the parametric empirical Bayes approach. So it's not just taking big data and crunching it into a regression coordinate and saying, all right, we found the most likely one, it's done. This is actually a framework that allows us to bounce back and forth between the summaries of big data and the actual large observations themselves. And then here, you can think of the prediction is on a single observation with a single hidden state. And here, it's like getting a vector through time. It's then you can imagine that it would be more dimensions, but this is just the skeleton. So let's keep on walking through it. Yep. Let me interject here really quick, sorry. So can you just go back? There we go. I love this what the authors how they describe perception. I thought it was very clear. They said perception corresponds to a process of inferring which hidden states or posterior expectations provide the best explanation for the observed outcomes. I thought that that was very clear. Cool. And the softmax is the sigma and that's just saying it's like a decision-making criteria on function. But we can also ask the authors more because that's sort of machine learning term. Yeah. Actually, just one very quick thing on that as well. I like what Blue's just said there as well is by bringing it on the outcomes as opposed to that outcomes brings in action. So you because the out it's like the perception it's not what the data is telling me. It's also the outcomes of what happened in the world with the body. And that is really quite, I don't know if that's been so easy to bring in before in the language. So I thought that's quite interesting. Yep. And we're going to bring policy in in a second. But for now, you're right. We have the difference between the outcomes and the hidden states. Is it raining or not? That's the hidden state. And then is my shirt wet or not? That's the sensory state. But we have... Or why is my shirt wet? Why is my shirt wet? Exactly. But we haven't gotten to this policy level which is should I go outside? So table one is just describing in a little bit more detail the P and the Q distributions that they use in their figure one for this like basic level. And that comports their math with just the broader modeling, Bayesian statistics, letters used, although the ABCD are sort of specific for active. So let's go from that M2 model and bring in policy. So this is M3 model. This is the next click that actually gets us to the first thing that looks more like a real active inference model beyond just a Bayesian time series approach. So maybe blue, go for it again with what you would say for M3. So for M3, this is the third step. And this is the generative model of action. So here it's inferring in phenotype congruent policies from all of the terms that we had talked about in M1 and M2. And then this brings in policies which that's Pi policies. And then E sub Pi is the baseline prior over the policies and G sub Pi is expected three energy. C is phenotype congruent preferences and then F sub Pi is the variational free energy conditioned on a policy. So the policies are here already here and we just have these M2 is the what's perceived over hidden state based on perception M1. So this is kind of just adding that extra layer including policy into the action model. So it says here posterior policy beliefs are informed by the fit between anticipated and preferred outcomes while at the same time minimizing their ambiguity. So to minimize that. So nice, thanks for this. A connection between variables in this kind of a graph represents them having like a statistical dependency. So we can look at Pi, which is policy and then ask, okay, so we had M2 which was this time series that looked kind of like I don't know, like a rake or something or a horse. And then policy Pi gets linked in not directly with the thumb on the scale influencing the hidden state but actually influencing B sub T which is the transition matrix through time of that hidden state. So one thing, because there's no Pi onto the hidden state you can't have policy at a snapshot like timeless policy. All policy is intrinsically related to the movement of time because it's hitting the B matrix not because it's not plugging into a hidden state directly. So you're always talking about a temporal model just how deep are you talking? And then it also conditions the estimate of how the hidden state evolves. That's what that vertical line like given, A given B. So this is, we're gonna do the time series how much Bitcoin will I have under the policy of not acquiring any zero easy estimate easy heuristic. So then those kinds of conditionalized estimates of different kinds of things can help agents act in the world. So that's why policy is interesting to think about. And so instead of thinking about how to optimize a massively complicated scheme for maximizing utility with an unknown variability this is allowing the agent to model how one is embedded in hidden states estimate or changing through time. So that's the Pi part. And then also this is kind of explore exploit. So minimizing the ambiguity is one imperative but also you want it to be fitting the whole time. Like I want my shoe to fit but I also wanna figure out where it doesn't fit but I need to be walking the whole time. And so by keeping a bunch of these features linked in the model, the dials at least exist to modulate in really creative ways like E is this baseline prior. And so you could modulate it so that the prior is relatively unperturbed by sensory outcomes or you could make it very sensitive to specific kinds of sensory outcomes. So this is again like a modeling skeleton that potentially could be used to evaluate how different sorts of behavioral phenomena like approach and receipt or something like that or freeze, whatever it happens to be might be linked by the organism seeking precision on an estimate of how things are gonna change in the niche given its policy. So of course there's a lot to say because it's kind of an interesting area but that's just the third level. So we're integrating policy decision into how states are changing through time. And then policy decision is the interaction of several variables, the baseline prior like the affordances, the habit as some have said it here. And then see the preferences. Phenotype congruent is like if I'm a shoe size 10 I want a shoe size 10. Then if you have a phenotypic preference that's different than what your actual niche or needs are then it will be ill fit. So you won't be fit for your niche. So then the niche will be filled by those that are fit to it. And then FAPI is similar like it's calculating variational free energy which we can definitely ask the authors more about and see how that is related to calculation on a specific policy, you know. Yeah, okay. Any thoughts either of you? Well just one thing I think is quite helpful as well is they call that perceptual evidence, the variational free energy conditioned on a policy. So it's like, and then that gives you the sort of perceptual evidence and then the actual action model which is kind of like the expected free energy. And that helps sort of show how free energy is coming in at this kind of larger time scale. And I suppose one thing just to mention to the viewers, there is a whole load of nested free energy processes going into all of this process. But at this scale that we're talking about, it's so that we can understand in terms of policies that we're trying to understand. Sounds good. And so yeah, go ahead. I'm going to add to your like shoe thing. Here, you know, anyone who has small children knows that their feet grow very fast, right? So when we're talking about the action model, expected free energy, it's the expected free energy because I don't like the actual free energy can't really be measured in this situation. So we're dealing with expectation. So there's this phenotypic risk component on the left side of the equation. Like how much money am I going to spend on these shoes and minus the perceptual ambiguity? Like I think my kid is a size 13 but they might be a one or two. How expensive are these shoes? Do I want to buy shoes that fit them right now or do I want to buy shoes that they can wear two pairs of socks with and wear for a little while? So you're always weighing out this risk and the ambiguity that occurs. Okay, in that specific example, the specific policies are like, okay, I could measure the feet every day and that's my sensory outcome. The hidden state is the true size of the foot. And then every day my policy could be if they're in a new shoe size, we change that day. Another policy with a different trade-off structure or different measurement requirements might be every two shoe sizes. Another one might be when the perceived ill fitness is too high and the squeaky wheel gets the grease, then we have an action-oriented response policy. And I'm going to be thinking about all the outcomes in this model and about the certainty on it. So nice. Yeah, I mean, it also relates to that. I mean, my daughter has a big toe starts to get sore. That's my perceptual evidence. It's like, okay, it's like that shoe is rubbing a bit now. So yeah, at some point I've got to make a choice. So that's a good example actually. And it's an error signal that it's ill fit and so the policy and that's a niche modification with a shoe, but it's a similar idea. So table three kind of goes through M3 in a little bit more detail. And that brings us to figure three, which is the fourth click of the active inference kind of chain here. And it adds just two more letters. It's mostly the same as figure two, like the C, G and the E, which connect at pi. That was my mistake in the 18, two. And I connected the E to the G instead of the E to the pi. They're connected at the pi as shown here. The C, G and E are now in M3 with a blue box. And then M4, one more dependency is being clicked in here, which is gamma, which is being clicked to G. So gamma is now an input to like a bigger G function. So the G is a function which you could input just a number or you could input a few, depends on how you write it. So this is a slightly more expressive G function because it includes this precision variable. And gamma and B are kind of like one over each other. The reason why it's kind of like when people take logs or do one over or negative this, it's just sometimes you wanna get the most of something, but it's easier to frame it as getting the least of something or vice versa. But what this gamma represents is the expected precision because this is kind of a certainty. So this one is uncertainty. So what would either of you say about this final click to M4 or Steve? So this is so fun. So the gamma is like how much did that shoe not fit? And the beta is like how fast are their feet growing? So I think that that's the, I mean, I guess beta doesn't take into consideration time, but it was described as a rate. It was described as a rate. So maybe it does include time. Not sure. I don't wanna go too far down the rabbit hole with the shoes, but it's almost like there's more precise shoes. Like you'd be more precise with a steel toe and then you could have the more stretchy. No, no, no one take this too. Seriously, it's about what it actually does say, which is basically the precision, which is the expected precision is in the gamma so that the uncertainty can be, yeah, in the B. And this figure three, the reason why they even went to this level of incrementing it out. First off is to show us level by level how we're building on, okay, it's about a time series that's being predicted through time of sensory outcomes that are like generated from an underlying time series of hidden states vis-a-vis a likelihood mapping and a transition matrix. And that's getting played into by priors at that S1 level, at the first state level. And then there's policy, that's the whole cybernetics control theory and then this is the machinery of the decision-making and now we're just adding one more level, which is the implicit metacognition. So now we're adding this implicit kind of always on, always being integrated into the model. Again, not a disaster mode, recovery mode for the otherwise non-affective agents, but it's an affective agent whose implicit metacognition, its kind of key regulator here is related to its certitude, not the thermometers tracking the average of its returns on investment or its average estimate, but its uncertainty in its generative model. So that's what we get to with their figure three with a fourth click. So wait, I think we need to talk about here because it's the first time that it comes up is their affective charge here. Oh, yes. So in this model, which also M4 really is the top level illustrating like the hierarchical nature or the nested nature of how this model is normally. And then in the affective charge, it's described here as phenotypic progress, which is the policy in the prior minus the policy in the posterior and then the free energy of the prior posterior minus the prior, wait, no, I said that wrong. Prior minus the posterior, there it is. Yep, here it is in table four. Perfect. So let's look at what they say. So expected precision is gamma, okay? Looks like a little dart, gamma, precision, you want more precision in an optimistic world model and policy pie, okay? And it turns out that gamma, the expected precision is one over a gamma distribution that's gonna be the beta. And that's just gonna be a statistical distribution that plays nicely with statistics. Just like you see like natural log, like the gamma, the normal, these are tools that statisticians use. And so they kind of work here, but we can ask the authors to what extent does it have to be this way? But anyways, gamma is the expected precision. So gamma and beta, we can kind of think about them as like linked because again, they're one over each other. So wherever you see a gamma or a beta, it could just be replaced with the other one. And affective charge is being defined as, yeah, the change in the policy multiplied by the free energy of that policy. So not just if it's good, but unchanging, then this is gonna cancel out. So the phenotypic progress is low. If you expect the 10 Bitcoin and you see them and you're not changing your policy, in this formulation, that's why we don't need to tie it too closely to a specific emotional experiences because those are gonna always be super subjective and require a ton more specification. But we're just kind of broadly calling this AC something that's related to both the way that the phenotype is changing, the policy is changing, and the way that the free energy estimating of that policy is changing. But- So something important to note in that equation is if the policy doesn't change, then the affective charge is zero. Yes, and let's again, let's have the authors give us the authoritative information on how to read some of these terms. But here is what they said about that step four. The fourth level, which is again, this gamma precision, is required to enable the agent to estimate its own success which could be thought of a minimal form of implicit non-reportable metacognition. So even to be able to semantically express metacognition, like I am unsure, that sentence is many levels or another level away from just this precision variable, which doesn't have a self-report circuit. This is just like a model that has an error term that it's updating at a slightly higher level than just the likelihood mapping. And the expected precision term, that's gamma, reflects prior confidence in the expected free energy over policies. The expected precision term modulates the influence of expected free energy on policy selection relative to the fixed form policy prior E sub pi, which is the habits or the prior. So it's kind of like, should I fall back to priors or should I go with what the G function lean into what my G is telling me? Then, yeah, formulated this way we can think of gamma the precision as an internal estimate of model fitness, subjective fitness. So higher precision, my internal model is doing better. Lower precision, my internal model is doing relatively worse than it could be. It represents an estimate of confidence in M4, that fourth click in the phenotype congruent model of actions given the inferred hidden states of the environment. So they kind of tessellate it back out there. They're saying it's about those outcomes and about what generates them and how those change and how our policies play into it. And it's about how certain we are in that generative model. And there's a lot of ways the environment can change. So if we're getting good data from our observation then we should be reasonably confident with staying with what works to some extent. But if that should change, there should be another mode that allows us to rebalance preferences, affordances and integrate that in a different way in a different allocation to our policies. And then to emphasize its relation to valence in our formulation, remember valence is like feeling good or bad, positive, negative. Going forward, we refer to gamma updates using this term affective charge. So here's AC, affective charge is change in beta, AKA change in gamma in just another way of saying it. It's change in precision. And then you can imagine if the precision goes up, it's gonna be good, precision going down bad and flip it for the one over or however it ends up being written. But that's what it's getting at. When the precision is going up as new data are coming in, like I'm thinking of a slot machine and then it's like, oh, you won something and now the second digit is gonna spin. So there's both an increase in charge and then that kind of re-stabilizes and now it's about whether that changes again or whether the second digit is better. And so that is how affective charge is being tracked through time with these gamma updates. Okay, any thoughts or continue 32? Well, I think the one thing that I find helpful hearing this here is the general overarching active inference parameter of uncertainty, it's like they're using the term confidence and precision more at these higher levels because it's like a meta, it's still at some level an interpretation of uncertainty but because it's on policies and that confidence and precision are at play because you're interpreting your ability to act not just the raw sensory sort of malure, so to speak. Right, there's a big difference between like visual uncertainty, it's just two shades of gray and they're so similar you just don't know what, you don't even see a difference versus two strategies, two chess moves that are very different and you're just not sure which one to do. That's a very different situation for inference because the way that your eye scans on the page is kind of reversible. You're getting to retract your choices but with strategy, it might be very irreversible, very nonlinear. Okay, here's another example they give on 32. So this carrying on right below equation 3.1 which was screen shot there. So AC can only be nonzero when the inferred policies are different from the expected policies. So blue, that's what you had said. So if it's sort of at the bottom of the bowl from a policy perspective, thermodynamically, it's not moving, the delta is zero. It's positive when perceptual agents, it is positive when perceptual evidence favors an agent's action model and negative otherwise. So it's like, are the data coming in locally good for my model or locally not congruent with my model? In other words, positive and negative AC corresponds respectively to increased and decreased confidence in one's action model. So as the data are increasingly good you actually like lean into your model and prioritize that prediction and then you kind of start maybe overfitting and then maybe the model has to pull back and rely a little bit on something different to re-equilibrate that to stay on this multi-dimensional front with model fit and optimal learning. So that's one of the ways that active inference is rethinking the explorer exploit is by relying on the generative model and then the trade-off with different ways the generative model can get combined with priors you can get explore and exploit behavior as well as a lot of other modes and a rapid switching that's informed by the instantaneous performance of a model. Then they give some examples of different a prey and a predator animal. Okay, and then here's five three. Yeah, any thoughts on that before five three? Okay, five three affective charge lies in the mind of the beholder. So that's cool because it shows that the authors in this research line have taken up the challenge of ecological psychology or of action-oriented philosophy or whatever it happens to be. It's like this relational insight that these variables have to be about the beholder and their relationship to a situation partially for computational tractability and partially just because it's kind of how these systems seem to be. We are modeling things that are in the mind of the beholders. We're always gonna be with a situation in the context and we're always gonna have to go into the specifics but we're also gonna find the patterns that are similar. For example, we all as they say tend to experience a rush of satisfaction not at five hours of satisfaction unless it was a long jigsaw when we solve a puzzle or understand the punchline of a joke or active inference in a moment. Our explanation is straightforward. In active inference, biologically plausible forms of cognition inevitably involve policy selection. Policy as fundamental. So policy is not the disaster mode. This is as fundamental as affect. Whether those policy internal selections or selections are internal, citation, citation, citation or external, period. Therefore, AC is also elicited by mental action typically in form of top-down modulation of lower level priors. I'm getting the lower level pain signal. I shan't not think of it. And then now I'm feeling better as that is top-down priors influencing the bottom-up signals just to give one example. So this is like bringing a lot of pretty interesting areas together, especially because again, it's not just about humans as cognitive agents. This is about a scale-free formulation for how affective charge works in active inference. But we can look at the specific example that they do provide with the simulation. So, Blue, any thoughts here or on figure five as far as what they do in the simulation? Maybe should we go through the simulation first? Do you have that figure up or no? Yes, I think it's, is it this one? Yeah, I figure four. Oh, okay, sorry, yeah. Yeah, I think that's probably better. So in this simulation, the way that they set up the T-Maze is it's a rat or it could be, I mean, it's a simulated agent, right? So it could be a rat or it could be an ant or it could be anything in a maze where on one side of the maze is food. Or so the mouse is placed in the middle of the maze and on the north end, say, you have on the left side food and on the right side a shock. And on the south end, there is an informational queue about whether, about which side the food is on as opposed to the shock. And so the food location changes over the trials and then the subject is allowed to move themselves. And so typically the subject checks the queue and then gets the reward. So there's a reward for, like you don't get shocked accidentally if you first go see which side that the food is on. So how the queue is associated, how the queue is associated with decision-making, well, if they know to epistemically forage, like that that's more valuable than accidentally getting shocked, then they go to do that. But I think in the trials they changed, they kept the location of the food consistent for many trials and then they changed it for many trials. And so the mouse had to relearn what to do. Yep. So let's just imagine the food was always on the left side, never changed. Well, as soon as you went to the left side 10 times, you'd be like, I don't even need that queue anymore. It's kind of useless info gain. It's gonna be not good time spent. So I should just go for the policy of going right for the food. Now, if it was switching every time, it would make sense every time if the shock was painful to get that queue every time. So the mouse is able to, in one way, make a binary decision which arm to follow for the food, but also there's this higher order decision about getting information. And actually in the active inference, the ant paper that I did with some of these co-authors, we also use a T-Maze, but there was no metacognition. There was just the stigma of the colony. So the T-Maze is really a generalizable framework. And once you have the code to make the T-Maze, you can just make basically anything. It's just like any other shape maze, it's just another matrix. So this is again, it's like an initial foray and the places where people see that it's locking, like that's where we wanna kind of build in the gaps. Let's get to a little more formal view of how you make that active infrared that is doing the active inference on this foraging decision. So there's a bunch of sections to this figure and blue, would you say it's better? Yeah, I think five is better before seven, right? Yeah, okay. So there is a generative model. Oh, first off, I mean, there's a lot of entry points. People should check out the model and just read it themselves because there's a lot of ways to enter into it. But one thing to notice is this C, the preference over outcomes. If it's not, if it's food, it has a plus four points, shock is negative six. These are just numbers, you can make it scale differently. And then if it's not a shock or food, it's zero. And then also, it doesn't have a preference for going left or right. So these you can already see are some of like the little atoms that are gonna get fed into a bigger model. So this bigger model is gonna be able to take kind of these variables. Like how much is the relative weighting of the food versus the lightning bolt? Or is there an implicit preference for one side versus another? And then here, the prior, D. The prior is that they're 50, 50, that it's on either side. But again, that's actually different than a preference for left or right branch. It's different than a belief that it's likely to be on the left or right branch. So we can separate out those two variables. I wish it were this temperature, that's my preference, but I'm very accurate that it's not that. And that's why I'm sad. So then you can talk about those kinds of combinations of different parts of the model. And B, so that's D and C, B is the state transitions. And so these are, it's kind of like a second level matrix. And this is kind of, it's like a cube. If you think about it, it's like a four by four by four because there's four positions it can be in. The weighting position, that's this front one, then it can go to left, to the right, or to the bottom. And then there are state transitions that are defined. Like if you go from the center to the left branch, that's gonna be a defined move in a matrix that you can make. Then A is the mapping between where you're at and whether you get the Q, left or right. So that's the info that you'd get on the bottom, the south branch, and then the reward or the punishment. And that's the likelihood mapping whether you go to the left or the right, regardless of whether you got the Q or not. And then E is that baseline prior over policies which is just, it's even here, they're all 2.3. We could ask, what is the effect of different values? You could work that through in your head a little bit, but it could be different. But here, all of the expressivity is in the other parts like the preference for the reward over the shock. So one doesn't need to specify like, yes, and I'd rather take a left turn at this exit while I'm going there. But that could be specified because again, it's just sort of a skeletal model of this scenario. Any thoughts on this before we go to seven? Nice, but isn't it, it's cool model. And it's nice how even with only four locations, there's so much richness in the behavior. So it shows you don't need the little scurrying animal to simulate the spatial dynamics of the maze because then you're gonna get trapped up. Oh, is it a narrow chamber or a broad one? But this helps you look more semantically perhaps at the way that the agents interact with their environment. Okay, so now on seven, Klu, do you want to say the two halves of seven? Yeah, so this part of seven, this is the generative model that they've created for affective inference. And the next slide is gonna talk about all of the different matrices that go into producing this model. So that's why, I don't know, maybe it's this one. Maybe this is better first. Yeah, go for it. So here, the different matrices that go into the larger model in figure seven include the prior expectations, D two for any initial states. So that's this top left. And then the second one, which is down here on the bottom is the affective prior, which is the likelihood matrix reflecting some degree of uncertainty in the predictions, which was also shown in figure five, multiplied by that beta, and sets the expectation on expected, sets the expected precision, so which varies between negative point five, which would be a negative valence, and then two, which is positive valence. And then this contextual prior is this matrix AC, A to the C, for the likelihood mapping from the context state to the lower level, reflecting the fact that the agent is always certain which context they observed after the trial is over, so that either got the food or the shock and they know that, right? And then this last one is the state transitions, B two, B to the two. And this is state transitions at the second level, reflecting a couple of assumptions that both the affective and contextual states vary, but have some stability across trials, point two to point three probability of change, and that the agent has a positivity bias in the sense that they're more likely to switch from a negative to a positive state than vice versa. Thank you, Blue. So what's happening in seven is we're adding another level. So here we see that beta and gamma being connected to the whole CGE policy and then that rake formation. And now we're gonna tie in this implicit metacognitive layer with another hidden state. So that introduces this parentheses notation where if there's a parentheses, it means which level of the nested Markov decision process it is. So the S that we knew and loved that was giving rise to the sensory outcomes is now S one. And that's the first level of the model. And then the S that is given rise to the precision is S two, the hidden state. So that is now what is being called the implicit metacognitive component, which is the second level of the model S upper score two, which has its own prior D upper two, just like this has a D upper one. It's actually not shown here, but it is there. There are the same Markov decision process structure is being recapitulated, but there's a different interpretation because the hidden states at one level are now the observations linked through an A at the next level. So here's the observations, like what square am I on? Sensory outcome. Am I then hidden state? What kind of square am I on? And then that's getting mapped to am I happy? Or that's the whole question of this paper is what does this actual second state? Yep, Steven. Oh, yep. You had now? I'm gonna have to head out. Sorry. Thank you. Peace. Hi. So then you get all these kind of interesting different kinds of matrices at the second level. Like, yeah, given that I went left and the food was left, then will I find out about that? And so you need a few matrices to kind of close the doors and make the model run, but that's a little bit on the implementation details and we'll probably learn from the authors. And in 10, we kind of just have a summary. This is, I think, a great figure because it shows how multiple timescales of nested processes are linked from the inheritance of prior phenotypes, like how big is that foot? And also development learning, like how fast does it change? Get linked in through multiple levels that are getting nested closer and closer to action and perception in the loop and then rippling back out. So pretty nice overarching figure. Any thoughts on this or continue on Blue? This just really, like you said, it's a nice figure, a good summary and just reflects the deep temporal nature of this model that they've built. Nice, let's go through the results of the simulation and then just deal with a few little further implications and then that will be a nice 19. So in figure six, they simulate out that scenario that we just talked through a few times with the maze for 32 trials, the food located on the left side. So this was just testing that it will converge to expecting the left side. This was kind of like a little sanity check for their model, but also it shows that we're doing this in the broader SPM toolkit. And so that means that a lot of these things, like the ability to visualize and easily run these models and check that they're complete and debug them and run them at scale, a lot of these things that tie up a lot of innovative modeling efforts, a lot of those are simplified somewhat. And again, we'll ask the authors about this. So they simulated in figure six just to show that they could do different policies from the exploratory policy in the first 12 and then switching over to just the go left policy. Okay, figure eight, anything to say, Blue? Okay, yep, so figure eight, they're kind of just giving a few trajectories of what can happen. So it's characterizing the model in a little bit of a playful way, but I think it's fair. And as we'll see, it's at least computationally warranted. So here we have initially anxious. The state is negative for the first eight time steps where it's getting the Q and the left side and then it develops confidence. And then the green line here in eight is where the situation changes. So then there's a few time steps where it still hasn't caught on before it gets shocked. And then it gets anxious again and it gets renewed confidence. So this is capturing a few of the, just like with the ants, how we showed how it could switch arms here. They're showing not just that it can switch arms from Q and left, left and stuck, then the shock unsticks it. It's like, whoa, I was very sure that going left was the right decision. I got shocked, now I need to go take the Q and not just one time, I'm going to get the Q multiple times. Okay, I think that I have renewed confidence in this direction. So it's really nice model behavior. Yeah, what do you think, Blue? So they did, actually there's one thing. And so it's not showing here, maybe it's going to show in figure nine, but maybe they, maybe they, I don't know if they showed it in the figure, or maybe we'll just let the authors talk about it. What about it? So about the model that lacked the last level, the metacognition about like, so in one of the models, they took that highest level of integration away to see if the mouse would similarly adapt. So here they changed the position of the food, right? In the top right figure. So they changed the position of food, it was always on the left, always on the left, always on the left, the mouse adapted, it didn't even go look for the Q. And then in the model without that metacognition, the model thought the metacognition didn't adapt to going to look for the Q, like they just like stayed stuck in the middle, like not sure do I, I don't know if I should go up or go down. I thought that that was really like funny. So this is what happened with the metacognition, initially anxious to develop confidence, anxious again, and then renewed confidence. But in the model that left that integration, that metacognition, it didn't, it just like stayed stuck in the middle. I thought that that was a very like cool anomalous effect and like a really neat control like trial for this test. Nice. And it actually relates again to this idea that affect our anxieties in the loop. And it's something that's a continuous variable. And in healthy settings, we could say, maybe anxiety is actually warranted here. There was a shock and a strategy that was, there's nothing wrong with a strategy that works and there's nothing wrong with switching strategies when it doesn't work. But there is some variable, whether we associate it with suffering. Again, that's the second question, but there is some feeling or some computational need for the system to actually pull back and re-wait how it uses its prior evidence versus how it deals with incoming evidence. So I think this is a really nice example of just the categorical, the discrete kinds of behavior that are very different from low precision and negative valence and then a kind of phase transition rapidly into a stable phase into negative and back again and integrating it with decision-making with, as they say early on, the imperative is that reduction of uncertainty with a generative model. Not maximize reward, not minimize suffering either, but actually to optimize its precision, aka reduce its uncertainty about a generative model of the states of the world, including its own action. So it's just really nice work. Some of the reasons why Active Inference is so cool actually. And then figure nine, they simulate 64 trials. Oh, I think this is what you were speaking to. And if an affective agent plotted in orange and an agent without higher level contextual and affective states plotted in gray. So one could look at figure nine a little more closely to see the difference, but yes, the expectations in, like just looking at the top one, the expected food location, they both converge to left and the orange goes a little bit higher. So it learns a little faster. It's like, yeah, I'm doing really well on this test. This is an easy course. Then the green line happens and the gray one slowly updates. It's like, hey, I've been right for 20 and I was only wrong twice. So I'm at 90%, what's the issue? And the orange one's like, whoa, something's wrong because I'm getting error signals when I wasn't. And the green line here represents the food changing, right? Like that's the context switch. I just wanted to contextualize it for people. Yes, the context reversal. So it was on the left and now it's on the right, aka I got a shock. And so then also look at this, the affective charge, the orange one, anxiety, drop, blissful ignorance, anxiety, drop. And then the gray one doesn't have affect. So we don't see it there. But then, yeah, or the gray ones affective charge is actually a little bit all over the place, but we can ask them what that means. And so yeah, pretty, it is super interesting because like learning, even organic chemistry is an emotional learning process. So how actually affect and policy and mental effort and all these things come together, it's kind of cool. I wonder if this is like, just like to take it to the like meta level, like is it like, you know, self-reflection, right? Like at what level of integration is that, right? Like, so you do the action, you select the policy and my doing teammates is don't self-reflect. But like when you integrate at that level, there's agents that do humans that do self-reflection and then humans that just kind of don't think about it, right? And so what is the better adaptation there? It just provides evidence for that, I think. Yep. And similarly, it's like, you know, I want to be doing the right thing, but then will that preclude you from having a thought that where that's actually being, that expectation is being violated, especially if there's sensory evidence that that's the case. Those are really interesting situations. Right. So like psychopathology, like are they just missing some higher level integration, like for lacking that affect, you know? Well, I see authors, nice. So then just the additional info section of the paper as we kind of close out in last little few slides here, there are four appendices and the code is available on Casper Hesps GitHub. And so you can see just from the top lines of the main MATLAB script, yeah, it's in MATLAB. It uses Markov decision processes and it involves active inference and learning using variational base, the factorized version in the SPM toolkit using MATLAB. So check out the model streams and let us know how it goes to work with the code. We kind of just had one or two more random areas to cover. A few directions that this takes us in, which we've discussed a little bit. One of them is more advanced simulations. So can we have two dimensions of affect or could we have another summary variable that represents some other feature of systems? Could we bring it more into alignment with computational psychiatry and translational research? And could we embody robots that implement these kinds of algorithms? How would that change some type of situational like physical robot or even a software robot if it had these kinds of estimators internally? And then we're always just thinking about where does active inference come into play? Which is easy for this paper because that's the whole model, but where are the next empirical steps which they mentioned in 5.7 a little bit? And how does this relate to this realism versus instrumentalism question? We're always getting at like, is this what the systems are? Or is this how we're gonna model the systems? Or are those one and the same thing? Any broad ideas and questions before this little Freudian interlude? Yeah, okay. So thanks, Dave, for pointing out a lot of the recent work of Mark Solmes, specifically this new project for a scientific psychology general scheme, which the link's given down below, but it's pretty interesting work that has to do with integrating recent developments in neuroscience with some Freudian ideas and beyond. And I guess I just like ideas that aren't named after people because then it gets pretty much like, oh, who's doing Darwinian evolution? What does it really mean to be Freudian? It gets a little bit historiographical instead of about the ideas, but the mappings that are made are clear and stand on their own and put the sort of history hopefully to the side when we see that there are commonalities across very disparate models. And here we see a familiar structure, which is kind of like that Markov blanket separating the internal and the external states, but the letters are a little bit different. And so there's an abbreviations table given in this paper. And also for many years, Friston and Carhar Harris and others have been writing papers that are about different Freudian concepts and relating it to free energy and drive and a lot of other points of contact with different psychological threads. It's just not an area I'm super familiar with, with the literature and the debates around, but it's an area that's obviously important to many and influences a lot of areas. And it's just interesting to see how these developments play into the absolutely still open for evolution field of psychology, psychiatry. So that was pretty much what we wanted to cover in 19. So it was pretty fun and little sprint to get the paper finished because it was somewhat of a long paper, but it's really worth it. So, Blue, any final thoughts or what are you taking moving forward or what do you hope that we discuss in the point one and point two? I don't know, I'm looking to really discussing the Bayesian brain integration or like co-relation here. I really think that the ability to update your beliefs based on what your previous experience is, is like this is like the perfect case study for this kind of philosophy. And I know like the Bayesian brain catches some slack sometimes and Bayesian statistics catch a lot of slack. But in reality, like we aren't gonna go to the shock side of the box if we know that the food is always in the left side of the box. Like so having that updates, I'm looking forward to just see what people have to say about the Bayesian brain aspect of this paper, if anything, we get that far. So we can make it all the way through the model, right? Nice, yeah, I agree. I was also curious about, okay, well, what if we could just associate a like, like click share or whatever on social media and then maybe you don't even need the shock or maybe you could add that or something. But with that matrix, could there be up vote, down vote systems where maybe the data already exists and then the positions that somebody could be in would be like, I have not seen it or I have seen it. Something like that and they're moving around pages, seeing or not seeing things and that's influencing their beliefs about the world and their beliefs about themselves and how accurate am I if this is what the info I'm getting is and then that's without even going into the instrumentalism realism, at least it's something that it feels like to be searching for information or a question that can be useful. So it's pretty cool stuff. Thanks a lot, Blue. And thanks also Stephen for joining for 19.0. We'll see everyone for 19.1 and two. Thank you. Bye.