 All right, hello and welcome everyone. It's Actinflab Livestream number 46.0 and it's June 10th, 2022. Welcome to the Actinflab, everyone. We're a participatory online lab that is communicating, learning, and practicing applied active inference. You can find us at links here on the slide. This is a recorded and an archived Livestream, so please provide us with feedback so we can improve our work. All backgrounds and perspectives are welcome and we'll be following video etiquette for Livestreams. Head over to activeinference.org if you wanna learn more about Livestreams at the lab or other projects that are ongoing. All right, we are here in the first discussion, the dot zero for background and context in stream number 46.0. We are learning and discussing the paper, Active Inference Models Do Not Contradict Folk Psychology by Ryan Smith, Maxwell Ramstead and Alex Kieffer. And the video, just like all the videos are, is an introduction and a contextualizer and an appetizer for some of these ideas. It's not a review or a final word. We're going to say hi and give introductions. Then we'll be covering the roadmap and abstract claims and aims and then we'll head through some of the core sections of the paper and just get some of the key arguments down. And during the dot one and the dot two in the coming two weeks, we'll have open space for asking a lot of questions for those who wanna participate live and ask questions as well. So, on we go. We'll start just by introducing ourself as much or as little as we'd like. And if we want to, mentioning something that we're excited about in the paper like what brought us to want to contribute to this dot zero and I'll start. I'm Daniel, I'm a researcher in California. And I think the title says it all, though there's still more to add and it speaks to many of the hottest and brightest debates in active inference, which is the rubber hitting the road with mind and brain and body and psychology and previous and different conceptions. So I'll pass it to Dean. Thanks for having me on, Dean. I'm up here in Calgary. I think, well, a couple of things. First of all, one of the first active inference that I was on was a paper that Ryan had written. And so I was kind of impressed with that paper. And I knew that if I sort of invested some time in this one, the stuff that I would become previous and the primary thing I came here was looking at active inference from the decisioning and the movement standpoint, seeing that's looked at from a higher order and the lower order type of processings. And that when I bought the Hama Gaia into the paper and started looking at the mind to world and world to mind piece and seeing where the potential connectors were between sort of coming into it or not, necessarily knowing that there would be different things of active inference, much like I didn't realize there was an extended active inference till I read Axel Constance paper, that was kind of helpful. And I'll pass it to Jacob. Hi, everyone. I'm Jacob. I'm a student from the Czech Republic. And I'm excited to reduce my uncertainty about the new kind of formulation that was introduced in this paper or at least it was the first time I came across it in the active inference literature. As you mentioned being about the different higher order and lower order descriptions and how they conceptually map to the different mathematical formulation and also to the psychological ontology. And overall just discuss what that means and what it might mean for other applications as well. And I'll pass it on to Ryan. Yeah, thanks. So I'm Ryan Smith. So I'm a research associate professor at the Laureate Institute for Brain Research. So I'm the first author of this paper. So obviously I was especially excited about the ideas to write about it or sufficiently motivated at least to try to clarify some things that I am, from my perspective, or sort of common misconceptions. Yeah, I mean, I'm probably more gonna sit in the background here. I'll just be here to answer questions or clarify anything. If something comes up where something in the paper wasn't sufficiently clear because I didn't do my job well enough. So yeah, so I appreciate you guys being willing to talk about it. Thanks. Well, an overall comment was we really appreciate the clarity of the argument and the writing and the weaving together of the math and the formalisms with the argument, the rhetoric and the ontology. That's prime time. So we'll just jump right in. And of course, anyone is welcome to share their comments. So Dean, would you like to help us contextualize with a big question? Yeah, so I think that there's some questions surrounding and I'm just gonna take some of the important quotes that I lifted from the paper. The concern that activists focus on folk psychology because they do not explicitly include terms for desires or other cognitive constructs at the mathematical level of description. So given that distinction, that there are active inference models of motor control which need not have desires that are folk psychology and active inference models do have desires within folk psychology. Then three is that it can explicitly include constructs. Dean, could you, I think when you're talking and you like turn away a little bit or move back it kind of cuts the audio. So maybe, yeah, we'll try that. Continue from the given. Thank you. Okay, so there's a distinction in active inference models of motor control which do not have desires under folk psychology and active inference models of decision processes which do have desires with psychology or so it's argued here. Then the worry is that if active inference models can explain cognition without appealing to constructs can be mapped onto the common sense notion of desires. And that's explain what that is. Then this could be seen as threat or intuitive folk psychology of ourselves as agents. Such a situation would also pressure the traditional belief desire intended model of folk psychology that is prominent in philosophy. The BDI model is a model of human agency that explains what it means to act intentionally. So for me, I was introduced to the BDI model Yes, great. And one way that I kind of saw this big question even pulling back a layer is like is active inference recontextualizing, reframing, augmenting, building on what is already familiar in a sense, though there are many folk psychologies and I think that could be something we go into or is this the displacement of some cherished framework for some other construct and the way that people speak like I want the cup of coffee or something like that, are we gonna need like a different word for an active compatible folk psychology and linguistics? So, Jacob, any thoughts on that? Well, I guess one question I perhaps had was like this, even this formulation of the initial statement, like given this distinction of motor control active inference and decision processes active inference, then we have this issue. Well, one thing that I was a bit uncertain about was whether this distinction was made to fit the distinction that's within folk psychology, like there is some kind of ontology for motor control and ontology for decision processes. So, therefore, we try to split active inference into these two parts or whether there is actually, whether this distinction follows directly from the same mathematical formulation of active. Great. Oh, I think one thing to declare, but I think the way that this kind of given the distinction followed by then here, I think could be leading to a little bit of a misunderstanding. So, the idea that there's a concern about a threat to folk psychology comes not from that distinction, it comes from the fact that there's at the mathematical level of description, it doesn't look like there's anything in there that's desire-ish, right? I mean, that's where the worry comes from. It doesn't really, the worry doesn't come from the distinction between motor control and decision processes, right? The distinction between motor control and decision process is actually one thing that by making that more explicit, it actually helps to resolve the concern or it shows why it's not really a concern or that part of the concern stems from a failure to make that distinction explicitly. So, just to kind of clarify the, so I think the order of the thought process here is a little bit different than the way it is in the paper. Thanks, helpful times, and we'll clarify and go through the argument in order two. So, briefly just the aims of the paper as they present it are to provide a brief review of the historical progression from predictive coding to current active inference models and show that despite a superficial tension when viewed at the mathematical level, the active inference formalism contains terms that are readily identifiable as desires and related cognitive constructs at the psychological level, which is downstream of that clarification of the distinction that Ryan just mentioned. And then they discuss the additional insights offered by active inference and the implications it has for current debates about active inference. Any other aims you'd wanna add, Ryan? I think this is fine at the moment. I mean, I'm sure things will come up. Great. And then claims. Jacob, could you read the claims? Yeah, so firstly that the apparent problem posed by purely doxastic looking constructs simply is not a problem. There do not appear to be cases where the phenotype consistent prior expectation in DAI often called prior preferences will ever conflict with or make distinct predictions than a traditional full psychological account in which beliefs and desires are integrated to point intentions. The second claim is that what we have referred to as DAI, the partially observable Markov decision process formulation of Acton is a corollary of the FEP and it can be implemented using prediction error minimization, but there are many other aspects of the FEP and many other theories that fall under the umbrella of predictive processing. And in some, there are beliefs and desires in the active inference framework. Thanks. And those are just a few claims that we pulled out, but many other declarative sentences will also be claims. I wouldn't necessarily call those the primary claims of the paper. I mean, there's certainly statements that we make, but I think the idea that there isn't, I think the idea that the apparent problem isn't a problem, I think that's a claim. And that it doesn't, that there won't be a conflict with an account where you're combining beliefs and desires to form intentions, that's true. This idea about what falls under the predictive processing umbrella, I would, I wouldn't necessarily say that's something we're arguing for in the paper. It's just something that matters when you're trying to correctly frame this kind of debate because predictive processing is just a very generic term, right? Like it doesn't refer to any particular mathematical formalism. It just refers to a really broad idea that in some way the brain's doing some sort of predicting either with respect to resolving problems in perception or problems in decision-making and motor control. So there really isn't, there's not enough, there's really just not enough mathematical specificity associated with predictive processing as a term to really even test any predictions that it would make, right? So I mean, I mean, what you need to do is pick, whatever specific mathematical formalism, what actual hypothesis you're talking about and you evaluate and test claims with respect to that, right? So there just aren't clear, like what even is just predictive processing is just too general in bank is the point. So, we can only really say, look under the current models used that are called active inference, right? So these partially observable aren't good decision process models, that minimize expected free energy as a way of making decisions, right? Those arguably count as one particular theory or class of models under the predictive processing umbrella. And we can evaluate claims with respect to that model, but what's true of that doesn't need to be true of the other 20 things out there that might also fall under a predictive processing umbrella. So I mean, the main point is just, we need to evaluate claims and predictions and things like that with respect to a specific model, not with respect to the vague general category of models. Thank you. Okay, so just rapidly through the abstract, active inference offers a unified theory of perception, learning and decision-making at computational and neural levels of description. In this article, we addressed the worry that active inference may be intention with the belief desire intention BDI model within folks psychology because it does not include terms for desires or other cognitive constructs at the mathematical level of description. To resolve this concern, we first provide a brief review of the historical progression from predictive coding to active inference, enabling us to distinguish between active inference formulations of motor control and AI, which need not have desires under folks psychology and active inference formulations of decision processes, DAI, which do have desires under, or sorry, within folks psychology. We then show that despite a superficial tension, when viewed at the mathematical level of description, the active inference formalism contains terms that are readily identifiable as encoding both the objects of desire and the strength of desire at the psychological level of description. We demonstrate this with simple simulations of an active inference agent motivated to leave a dark room for different reasons. Despite their consistency, we further show how active inference may increase the granularity of folk psychological descriptions by highlighting distinctions between drives to seek information versus reward and how it may also offer more precise, quantitative folks psychological predictions. Finally, we consider how the implicitly cognitive components of active inference may have partial analogs, i.e. as if desires in other systems describable by the broader free energy principle to which it conforms. Here's the roadmap. So the dot zeros, all of them in the world wouldn't be enough to hit every stop. So we will go through roughly in order of these sections and the section titles are listed here. But Ryan, what was the thought going into the ordering or the structuring and why there was such a comprehensive historical and preliminary consideration section? I mean, a lot of, the vast majority of these sections are just kind of building up background, right? I mean, sufficient background to kind of see, right? Where the, both, I think historically where potential misunderstandings might come from and also just enough of the, providing enough of the formalism, one part of the formalism that's kind of built on another, right? To see why the parent problem just isn't there, right? So, I mean, the vast majority of these sections are just preliminary, just kind of building up so that the reader has the information necessary to understand the argument. I mean, only the last couple of sections there really are the meat of the argument itself. I mean, so, for example, just section two that are what we just said from the predictive coding active inference is just kind of highlighting how these initial formalism, Asian brain as implemented through predictive coding was purely a model of perception, right? So then, you know, so then what people, but a lot of times you'll see especially in the older literature and some philosophy literature, predictive processing somehow both refers to predictive coding, but then also is talked about as though as though it's the same thing as active inference, you know, or there'll be these kinds of ideas where if you just extend predictive coding as kind of the way it's talked about as a kind of way of thinking about what the whole brain does, you know, then all of a sudden, you know, controlling the body ends up also just being something about predicting what the body will do. But, and especially when described that way, right? It sounds like something like desire and motivation and goals and things like that are just kind of completely out the window. It's very hard to make sense of what how a system like that would even work, right? And so it's, and part of that I think is because there's, you know, some, you know, some the way that certain things are written and some of the literature can be a little bit confusing or it's not too, it's not too hard to understand why, you know, these sorts of, you know, these sorts of misunderstandings could, you know, could happen, but, but the, you know, section two that was really, really is to show how, you know, initially, right, when you're trying to move from predictive coding again, which is purely a model of perception, it's not about decision making, it's not about action selection, you know, it's not even about motor control, right? It's just a, it's just a Bayesian, just a theory of how the brain can do approximate Bayesian inference and perception, you know, how that was initially just extended to say, hey, like if the brain works this way, then, you know, how do you, how do you get a system that actually controls the body using the same generic architecture, right? And so then, you know, the story that Carl Friston, you know, and, you know, some other people started trying to put together was the story about how you can use predictions, you know, descending predictions if weighted appropriately, right? As a way of essentially setting the target states of the body, right? Like, like it's just kind of, you know, the way you see it talked about is, you know, I predict that my arm will be here, really it's down here, but if I weight that correctly, then it'll kind of like, you know, you'll kind of control the set point and a reflex arc to, you know, so your arm kind of moves up with a preposition, right? But that's just a theory about how you can use a descending prediction signal to essentially act as a motor command signal, right? There's nothing in that at all that's deciding what that prediction ought to be, right? There's nothing deciding what that, which motor command should, should be the one, right? That's getting sent down in, in a, you know, in the, with the form of a prediction, right? So, so again, so even there, it's just a theory about how you can use predictions as motor commands, right? Like there's no, so the very big distinction between that and whatever the system is on top of that, right? That's deciding, okay, what motor command, what motor prediction, right, do I send? And so just making, you know, making that clear, right? Because around 2015, there was this big shift, right? From these, from these act, where from active inference being talked about as a, I just kind of a motor control process, right? Like a, like a story about how you can extend predictive coding to, you know, also do motor commands, right? Like from that to, you know, these larger, more comprehensive decision-making models, which are the current, right? Like on BPS, and those, you know, which was what we aim to show very explicitly, you know, have something that, you know, in the mathematics, it just does, right? It specifies the goals of the system. It's pretty hard to get around the fact that you need something, a system needs something like goals, right? Because it has to evaluate somehow why one decision is gonna be better than another, right? The only way it can really do that is with respect to how likely it is to get whatever the system wants. So it's just, I mean, in my view, it's very difficult to see how you could ever get a system to be able to evaluate one action as better than another without having some sort of target state. And so you can call, right? Like the target states in active inference, a type of prior belief. But at the end of the day, it's functional role is to specify what observations are better than what, right? So I mean, you can really think of active inferences as using this kind of trick in a certain sense, right? Of specifying desired outcomes in the form of a probability distribution because that helps keep everything fully, fully, fully Bayesian, right? But it's not playing the functional role of belief, right? It's encoding, you know, higher probability just means, right? Like the better, more rewarding outcome. So a lot of it is just, you know, kind of working step by step through both kind of the historical progression. So you can see where the misunderstanding could come from. And then also through the current formalism, so that you can see exactly where those, where the, you know, the goal states are encoded in terms of something with the form of a probability distribution. Thank you. The keywords are active inference, folk psychology, predictive processing, Bayesian beliefs and desires. So now we're going to jump in and some of the slides have a lot of text topically arranged. So we won't need to read all the text on many of the slides, but especially if somebody wants to pull out like one of the highlighted sections and like bring it to our attention, that would be awesome. So, and also Ryan, thanks a lot for that great historical overview. And I think there's so much more to be going into about what happened before 2015 and what happened in the last seven years. In fact, I think you covered many of the key points in what you just described here, which had to do with the development of predictive coding framework into understanding brain and body. Is there anything that anyone else wants to add about these points? Then all discussions and dot zeros are like a two-way street where some people with active inference familiarity are learning about new ideas and frameworks and also people who might have familiarity with a broadly used area outside of ACTIMF are learning about ACTIMF. So for both directions on that freeway, it's important to understand what is the target non ACTIMF framework that's being juxtaposed and found concordances with active inference. So the article is addressing the worry that ACTIMF models may be in tension with the belief, desire, intention model. And there are some consequences to that worry. In other words, given some priors, it's a founded worry. Would anyone like to summarize what the belief, desire, intention model is? Thank you, Sher Daniel, because you, I think you put the slide together. Ryan, what led you to select the BDI model? And I wondered about almost this tension or paradox with like, it's folk psychology. It's about what the people think, but we're gonna do a citation and one specific academic acronym for what people think. So is this like the main game in town for folk psychology from an academic perspective or what is the BDI and what led you to select that rather than like a small portfolio of alternate folk psychologies? I think that the belief, desire, intention model is just a very kind of generic button, but also widely known account of trying to capture folk psychology, it's a major sort of target of a lot of discussion within like philosophy of mind. But I mean, it's just very generic, right? I mean, it's just this idea of what are the necessary ingredients to make a decision. Well, I have to have some beliefs about how the world is and I have to have some desires about how the world, I would like it to be and my intention involves integrating those two things, right? And then if all goes well in terms of translating intentions into controlling the body, then I'll act out something according to my intention, right? I mean, simple, like the dumb example that you'll often hear is just something like, I desire some ice cream, I believe that there's an ice cream truck down the road, given those two things and necessarily those two things, right? I can inform and touch it and inform an intention to go walk over to the ice cream truck and buy some ice cream. So I mean, the question is how else are you gonna explain why I went over to the truck to buy the ice cream? That literally just is, that is the general sort of thing that when we're reasoning about other people's decisions, other people's behavior and we're trying to figure out, okay, why did this person do what they did? If we're assuming it's a voluntary action, right? If we're assuming it's not because like they're arms spasmed or because they have some like really highly ingrained habit or compulsion or something like that, right? And when we're talking about voluntary behavior, that is just the way that we tend to reason about how people do what they do. Either when somebody does something that we think is weird, right? Then usually we have to explain that in terms of, okay, well, they just believed something that was incorrect, right? Or they had some really funny desire, right? That I don't really, you know, it tends to be, you know, one of those sorts of things, right? And I mean, I mean, I don't know a lot of other, you know, separate folk psychological models that would entail anything different than this, you know, at a very generic level. And it's also exactly the sort of thing, right? That people try to contrast active inference with where instead of, you know, belief desires, beliefs and desires coming together to form intentions, it's said it'd be something like one kind of belief and another kind of belief coming together to form an intention, which again, for a lot of reasons, is, you know, confusing and rightfully so when it's presented that way. Thanks. Yes, Steve. That's a good question. So, Ryan, when we've had some conversations with other authors and other papers where we've talked about the scale three formalism and we've also talked a little bit about the sort of scale friendly times when you have to find that history and that context. What you just mentioned was sort of the generic generalized sense that the BDI umbrella provides. You're not saying that scale three quantification and a generic way of modeling something are the same thing. But what I think, without putting words in your mouth, are you saying that the scale three-ness as it moves to something more scale friendly sweeps up some of that generic big tent idea or model and now moves it along. It allows you to sort of parse and be specific and precise, but also be able to generalize as you move through those, you know, quantifications of the distribution and density and such. Because that's kind of what I was reading into it. I just want to make sure that I got it. I was kind of, maybe you can help me clean that up a bit. I mean, I really don't know that anything that we're saying really depends on or even gets into a lot of those sorts of specifics. I mean, really all we're saying is that, you know, if you want to take a model like the current active inference formalism, the current kind of like vanilla formalism, right? So in terms of interest to standard one level POMDP or, you know, if you want to scale it up hierarchically, it doesn't ultimately matter. It is if you want to use the vanilla active inference framework to actually model some sort of voluntary decision process in a human, right? So I mean, a lot of my work has to do with people, you know, modeling behavior on decision-making tasks, right? We often have to do with, you know, the person has the goal of maximizing how much money they went or maximizing some sort of social reward, something like that. The idea is just that when you're trying to, if you're going to take an active inference model and you're going to use it to model, successfully model actual human behavior when they're making decisions, then there's always going to be a mapping between the different elements in the active inference model of voluntary behavior that, whatever the prior, you know, prior preference distribution is, you just check that so that it just goes to the level of reward associated with different situations. You know, so there's really nothing about any scale or anything like that. This is much more generic. It's just if you're going to use active inference to model voluntary choice, then it will always have something analogous to a belief and a desire, because that's just what you need, right, like to model voluntary behavior. It's true that, you know, that's probably, if you actually were going to try to capture something kind of like more of the actual architecture in the brain as opposed to just this kind of like voluntary decision process aspect. I mean, I'm sure there's a lot of kind of hierarchy below that under the hood, right, that's translating more abstract policy selection processes into whatever the dynamic signals are that end up controlling, you know, moment-by-moment muscle movements and things like that. But the point is the below this kind of top level where you're doing policy selection, you know, at that point you don't really need the desires anymore, right, you just need the policy to specify whatever the motor commands are going down that can take the form of predictions. So I don't know if that helps, but it's just that there's a certain level in any kind of hierarchy of a model that's going to actually be applicable successfully to human behavior and at the level of policy selection, prior preferences will just include the desired outcomes. They'll just encode the goals that the agents trying to reach and the policy that's chosen will be evaluated as being the most likely policy because it's the thing that's predicted to have the highest probability of getting the desired, you know, generating the desired observation. If I may also comment on that, and I'm not sure if this is answering your question, Dean, but the way I thought about it when I was reading the paper was that part of the reason why the Believe Desire intention model kind of works well for mapping the active inference ontology to the folk psychology ontology is because it's a discrete model in that it separates these different parts. And I think that also might have motivated the choice to consider mainly discrete POMDPs and describe the decision active inference in that way as well because then this discrete model can be more easily mapped to, say, the discrete model of the POMDP that's described like a factor graph. I was wondering how this would map to continuous tasks as well? Well, I mean, I think part of the reason why you need something discrete is because action is discrete, right? I mean, there's either policy one or there's policy two or there's policy three, right? So those just are sort of necessarily different, right? I mean, you get continuous state spaces work well or are appropriate at lower levels in a hierarchy when you're talking about kind of dynamics and motor movement, dynamics in the set point of some reflex arc or when you're trying to estimate something in perception that just as a continuous quantity or like brightness or orientation or things like that, but at the level of decision-making, models that are necessarily right is free. I mean, there are kind of hybrid models that are out there in the literature, like Thomas Parker, for example, has published several papers using these sorts of models where the kind of policy at the higher level, which is in a discrete model, generates some observation that then sets the set point for some continuous, for some continuous lower level model that then can end up in a way that's kind of like this kind of reflex arc story, like move the eyes around the cod to different locations and things like that. So I kind of think that the discreteness is the reason why things move to a discrete state space architecture is just because of like the necessary, what's necessary with respect to a model of policy election because it is kind of all our none. Just to keep the dot zero zeroish, we're going to move a lot faster through the following slides, and there's many important questions in the chat and also arising. So we're just going to carry on more rapidly so people can pick up on these key points in the zero and we'll have a lot of time to explore soon. Section two, from predictive coding to active inference, traces the history and the development from various fields and just one sentence here that Dean and I both highlighted on and then Dean I'll let you describe the button was that crucially for the purposes of this paper, the first generation active inference was not a theory of decision making. It did not explain how we decide or plan where to move our body. It only explained how body movements can be executed using the predictive coding apparatus once a decision has been made. So how did you connect that to the bottom right? Yeah, so if you're on a some sort of a website and you have to pass through to something, my question was based on that statement. When the expectation is to know that you've actually crossed a threshold, you've moved beyond one page and there's an expectation that there's something that you validated or confirmed or whatever is the feedback that you received. Does that have, how does the person who's actually expecting that they're on the other side, what kind of feedback mechanism do they need for that confirmation? Is tactile feedback enough? Do they need some sort of a visual confirmation as well? Like what is the, what's the, is there, because we're talking about thresholds, is that different for every single person or is there sort of expectations built in depending upon the kind of situation that you're dealing with? I do know that with, but the idea that like changes are saved or links are copied, sometimes the click isn't enough to give people confidence that in fact that process is carried through. I mean, I would, I would say that I mean, a lot of those questions are really empirical, empirical questions, supposed modeling questions. You know, I mean, in terms of, in terms of how you would model that sort of thing, right, you just specify what observations count as tactileity, specify what observations count as visual, and, and the, you know, what ends up being thresholds for sufficient evidence and things like that, just, just not to deal with the way that processes just have the dynamics naturally, you know, unroll in the model under whatever the model parameterization is, you know, so, so you'll hit a threshold faster if the mapping between observation, whatever observations and states is more precise. For example, I mean, there's, you know, in relation, in relation to that, when we're doing policy selection, right, we also have to check and see whether things are actually going as we expected them to go, you know, under a choice of policy, right? So it's possible that you choose policy one and you start to get the subsequent observations and they don't actually match what you expected, you know, given that you had chosen policy one, you know, in which case, you'll update your beliefs through perceptual inference, and that might influence how you act going forward. So, but the specifics about, you know, what's enough and whether it's visual or tactile or anything like that, I mean, those are, those are really just empirical questions that would have to be answered in studies as opposed to something about model choices. Okay, so there was really not a good enough thing when you did the dark room aspect of it. Like the risk that you were prepared to take on, right, like somebody was prepared to take on the risk because they really urgently wanted the ice cream versus the person who was like, eh, I'm agnostic, right? So that's maybe, that's kind of what I was, because this was on the second reading of your paper, right? So I was kind of now backfilling from further down in the paper. So, let's keep it to that. Yeah, we'll go in the order of the paper because we're mentioning things that we haven't brought up yet, but let's continue on and we'll return to that. And Ryan, I also agree that in any specific case, it's going to be a model parameterization. Some of these are very difficult to answer in the abstract model selection setting. Here, the distinction between motor active inference and decision active inference is introduced and I'll allow Jacob to just convey one pass. What is the difference between motor active inference and decision active inference? So I'm not sure whether I can say much that it hasn't been said yet, but briefly, the motor active inference is does, well, as already mentioned, it does not consider desires in the active inference and for psychological sense as was then connected in the paper. But also it is my understanding that the motor control version of active inference is modeled continuously like the models of the movement of individual muscles, which does not entail decision making. So it's modeled with continuous time rather than in discrete time. And the decision making active inference, Ryan already said as well, is all about decision making. So it describes the discrete process of decision making with prior beliefs and preferences. And it can be modeled at different levels of cognition as well, which is one thing that I'm still a bit uncertain about how we can move through these different layers while still keeping the same mathematical formulation. But I think we'll probably get to that in the other slides. Yep, we're going to continue with this distinction of MAI for motor active inference and DAI for the decision active inference. Here we're showing some key sections from the paper and some citations. I think we will continue on without going into this in depth, but it describes some of the specific model quantities that are being described in active inference formalism and we're going to come to them when we look at some equations in the coming slides. In section three, preliminary considerations, two broad points are introduced and we might show a second one on a later slide. How can we think, Ryan, about what is said here that decision making AI models are largely taken to describe sub-personal, non-conscious processes? And also it was asked in the chat, not that we have to address it now, by Duvid, has active inference produced a single reasonable model of Aqualia? So when we're thinking about this first broad point that was made here, where is experience and what is the distinction between the personal and the sub-personal in DAI? Well, I mean, so there's a couple different questions you asked there and I don't necessarily think they're synonymous. So when you said I think the first question you asked was something about has active inference provided some sort of explanation or model of Aqualia that would be satisfactory in the strong sense. So in the sense of Aqualia as an explanation for like the hard problem of consciousness, then I think active inference is in the same place as anyone else and no one has a good explanation for being able to deal with the hard problem of consciousness, right? I mean, like massive literature out there, but I think everyone thinks it's just as mysterious as ever. So no, right? On the other hand, I mean, myself and others have published multiple papers showing how you can successfully use active inference models to model conscious access processes, right? So the distinction between when the brain is representing something unconsciously versus when it's representing something consciously in that it can self-report what it's representing can use the information to make voluntary choices. And we've shown that those sorts of models are able to reproduce like empirical results in like EEG studies and fMRI studies and things like that that also can make novel neurophysiological predictions, some of which seem like they are in some subsequent work have empirical support for those predictions. So in terms of providing what seems like a useful account of the processes associated with what people self-report that they consciously experience and the brain basis for that, I think active inference has at least the starting point for being useful and explaining those sorts of things, but why one little posterior inside a model versus another posterior in a model actually corresponds to like the experience of red versus blue or something? I mean, no, I think active inference is not any better than anyone else at any other model for that question. I guess the second thing that you asked had to do with this kind of like related question of why it's the case that one level has to do with a conscious process versus unconscious process and the best I've really got there and what falls out of the models that we've shown really just has to do with temporal scale. So there will be a certain level of temporal representation in a hierarchical model that integrates enough and represents regularities over a long enough timescale that it can contribute to in a sort of prospective it will be sufficiently compact to generate things like self-reports, right? So I mean, think about how complicated and temporarily extended reporting and choosing to report something like, I see something green in front of me, right? I mean, that's a very complicated and really deep you know, type of policy to select and it requires integrating a bunch of information from stuff that lower levels that's happening over faster timescales. So at a minimum, right? You just need to be at a level of representation where the regularities are sufficiently long and are especially temporarily deep and have access to all the relevant information you need to integrate to be able to generate reports like that. Whereas much of the other kind of little pieces in the model, right? Like, you know, degenerated, say like expected or like variational free energy gradients and things like that that, you know, people might call like surprise, right? I mean, no one's claiming that and often they will not relate in any way to people's conscious reports about feeling surprised because conscious reports about feeling surprised will have to do with representations of surprises in states, right? As opposed to, you know, some sort of prediction error like, you know, gradient minimization process. Thanks. Also in section three, there are several clarifications and some seeds that are planted. Namely, if computational and folks' psychological predictions converge and no other available theory can account for behavior equally well, this could entail the mathematical structure of DAI as more than a convenient tool instead that it corresponds to the true information processing structure underlying and enabling folks' psychology and related abilities. And this is really nicely put here that the crucial point remains that one should not conflate mathematical and psychological levels of description. However, DAI models might nonetheless offer more detailed information about the true form of folks' psychological categories and processes. And then they add that their aims are to demonstrate that there's a clear isomorphism between the elements of DAI models and those of the BDI model. And then show how provided one does not assume probability distributions in computational models must be identified with beliefs at the psychological level, there will be no tension between DAI and BDI. And that comparison or juxtaposition is clarified by separating MAI from DAI like has been mentioned. Anything else before we start to jump into some of the formalisms and keeping all of this in mind? I guess I had one question on which ontology we're working with when we say there's isomorphism between the elements of DAI models and those of the BDI model. Because I think there are multiple ways to interpret this. One thing that I'm wondering about is whether this means that we can basically construct a mathematical formulation of the BDI model that is like a subset of active inference or maybe there are certain elements of the BDI model that aren't necessarily encapsulated with an actin that weren't discussed in this paper. And that would mean in terms of formulating this mathematical formulation of the BDI model. So I mean what the isomorphism just means is tell me the description at one level and I'll be able to translate it for you very quickly and directly into whatever the description is at the other level in both directions. I mean so it just means tell me what it is that you want. If I say do you want ice cream or do you want pizza? Tell me that you like if you tell me that you like you want pizza twice as much as you like ice cream then very easy to just put a four in the distribution in the preference distribution over the observation of pizza and the two for ice cream and then softmax it and log it. That's the description. So just in a very direct way just encodes relative desires for one thing versus another. Tell me what your beliefs are. Do you believe that pizza is to the left or to the right? I can put that in as the different categories of hidden states that you're going for a distribution over. So like it's just the point is that in either direction give me the full psychological description I can translate it very straightforwardly using the same elements every time as being desires or beliefs. Give it to me at the mathematical level. I can translate that into description in terms of beliefs and desires. It's just one to one. I can't think of an example offhand where there would be any kind of like mismatch or an ability for one to account for the other but it's kind of hard to prove a negative. So I mean maybe you can give me an example but I can't think of one. Awesome. Jacob. Yeah I was just going to say that I can't really think of an example right now. I just found it really interesting with the usage of of just the word isomorphism because that like as you said it's just a bi-directional and surjective map. So it's basically equivalent. It's just set in a different way. So I can imagine that meaning that we can basically instead of the Wikipedia page for the BDI model using all of these terms like belief desire intention it could just be a set of active inference equations and that's the BDI model because it's an isomorphism. Right. Literally your beliefs are just your cues and your desires are just your P of O's. I mean in the way that we and others have tended to write it now is to literally just explicitly to say like that desires are or the preference distribution is P of O given C where C is just a matrix that defines your preferences. Right. So I mean it's a lot clearer to do that than to just just leave it just P of O. Right. I think that's part of the confusion is that it's not clear where the preferences come from but they're just it's just because they're conditioned on this other thing C. And C can be fixed or learnt but that's the basis of the calculation there. Yeah. Yeah. Here we get into section four, variational and expected free energy considerations and in large script we see the variational free energy equation. Dean, what is the skydiver doing? And during the one and the two we'll unpack more about the exact terms and what is coming into and coming out of this equation but just as an appetizer here what's the skydiver doing? Well now I've got the answer here. I just put through that in there because I like the I don't think it's a metaphor but the analogy of how so how do you respond when you jump out of the airplane and you know your parachute's not going to work. Do you go ahead first and really attack it or like what now what are you doing in terms of what your expectations are. So maybe that was a good analogy Ryan. Maybe like so where are we going with that because yeah. Well I mean I think the point of those sorts of examples are just to sorry to put this to show why it's not really correct to think about the like prior you know this prior preference distribution even though it's you know it's again it's cast as a prior belief right. It's not really appropriate to think about that as a belief per se right. I mean the or an expectation at a psychological level of description. So you can in a way to contrast that right as you can say look like the person in a model where they don't want right to fall to the ground and die right. That would be your that would be your PFO right. It's your preference not to have the observation of hitting the ground and dying but what you act right is is your Q of O given pi right. That's what do I expect to observe given that I choose this policy versus that policy right. So that's the that's the you know the belief part right. It's what I expect given that I do this or that and you know how close is that to how much does that diverge from what I what I want right which would be your PFO. So it's just a it's just a clear example of how your belief the expectations and your desire the expectations come apart right and it just doesn't work conceptual I think one is think of one as like the same kind of kind of thing as the other. Yeah I just thought is if I'm not going to get a soft landing how big of a crater can I can I create. Well that's that then is going to be packed into your preference distribution right right. I would have some desire for you know to make a big you know conditional on your going to die right like you know you'd prefer right to make a big crater than a small crater and that would still just go in your PFO right. So yeah yeah. I was just wondering and this might be me thinking thinking too deep or not too deeply about the equation rather but in this in this formulation where it's a variational free energy for each policy f of pi does that does that imply when in the last sentence on this slide that free energy is a functional of beliefs and a function of observations that doesn't mean that the free energy is essentially a function of three variables but in this case we are only writing it f of pi because it's always it's it's clear that it's a functional of beliefs and observations or should we think of it more as just a function of each policy. I mean the way it's written here I mean I think that the most great forward way to think about it is I mean it literally just means that using this equation you will calculate a different f value for each policy right I mean that's really it like you just there will be you're going to be computing some some variational free energy value for each policy and that's going to make up a distribution over policies and and I mean the the tricky thing I guess here is this is that you know this isn't about this isn't you know f of f of pi here isn't about making decisions right this is just about you know something like evaluating evidence for you know four various policies one because you you can't you calculate f when you already have the observation right so both you can't really do this perspective decision-making thing with f so this is just basically a way of describing how how under each policy your beliefs will change or wouldn't change right under when you know when you get conditional on each new observation that you got and and so it's not uh yeah so all it denotes really is it's just you know you have a value of f for each policy and that makes a distribution and that takes us to g expected free energy the letter after f and as shown here not sure if there's another reason for why it's g this is equation 2.6 from the active inference textbook just for reference with some other versions of how g is framed um g is a prospective value and they write decision-making does not only require beliefs about past and present states it also requires making predictions about future states and future observations this requires taking the average i.e expected free energy g of pi given anticipated outcomes under each policy that one might choose and there's um a lot to say on expected free energy what is just one short note that somebody could add or ryan why was it placed here um i'm not sure i understand the question like why was why is why is expected free energy required why is variational free energy not enough i mean expected free energy is just is just what the system's trying to minimize when it's selecting a policy right so i mean it's just trying to i mean in this i mean this decomposition right here is often called like the risk risk close ambiguity decomposition but i guess uh i like i think that i think that's showing the um you know the risk term the risk term here provides a nice kind of intuition for for again this separation between beliefs and desires and active inference because um you know you're you're uh so the the the term underlined in in red then right so the kale divergence between q o given pi and p of o right i mean that's just saying that um you know how different are the observations i expect given that i choose a given policy and you know how different how how different is that uh how different do i expect that to be from my uh my preferences or my desires right p of o um so the the closer those are together right the more uh the more similar i expect the observations to be under a policy to my preferred observations the smaller that's going to be and so if i'm trying to select the policy that minimizes uh g right then that means i'm going to select the policy that's going to get me as close that's going to get me as close to my preferred observations as possible right so literally just is that kale divergence literally just is trying to choose the policy that brings beliefs closest to desires very nice and then what does the blue term mean uh that's just an entropy term right so i mean it's just basically saying that the is what drives information seeking or part of what drives information seeking it's just uh it's just yeah basically the agents choose you know driven to choose policies that also are going to generate more precise expected observations observations that are going to disambiguate states more more easily great um here we'll explore it more in the dot one but there's a description of the posterior probability distribution over policies p of pi and an expression involving the softmax symbol with the sigma we'll return to that later but this is just so that we can cover like the key experiments and the simulation results today um this is also one other uh area of formalism we can explore in the coming discussions about precision and the relationship of um beta and gamma and what the convergence towards means but we won't go there there is precision parameter in this model let's go to the dark room problem and many lines of pdf screen and ink has been written on the dark room in the active world since um perhaps first in et al 2012 and also outside of active and um they write in a nutshell the concern is that if agents only act to minimize prediction error as opposed to acting under the impetus of a cognitive desire like state then they ought to simply seek out very stable predictable environments such as a dark room and stay there would anybody like to add anything about this dark room problem or what is the relationship between doxastic and cognitive ontology and active i mean are you just asking what like what those words mean or what like how do you how do you use these words i mean i mean doxastic is just a term that refers to you know beliefy you know like epistemic sorts of things right so so a doxastic ontology would be an ontology that includes beliefy things right whereas whereas uh you know cognitive just refers to yeah being like targeted towards something right like something about you know desires or um um you know something things you want versus don't want things you like versus don't like right so um it's just uh you know when we say it's apparently purely doxastic ontology it's just saying it might look as though superficially like the um the ontology proposed inactive inference like the things that exist in active inference are are purely beliefy and that's all that means great and it's applied in the setting of this dark room for the following two slides dean help us understand what concerns are addressed and what the child on the top right of the slide is doing yeah well so i'm not going to go through all all of the text but essentially this was my introduction to the dark room problem because it was kind of the opposite of all my experience which is that i tend to find people that were curious and were trying to move away from information um leveling off or being static to information gain so first of all i had to learn what what the what the argument was about why people would actually in it know who are normally social why they would move away from that social realm and into this sort of dark space under the under the idea that active inference it actually implies that's what people would do um and then secondly it was i was kind of trying to tie it into the idea of the refrigerator and where it is and and until you actually and dany dany and i talked about this a bit the pragmatic piece of till you actually open the refrigerator door which is on the kind of the next slide you don't really i mean you can have a belief that there's something of value in there but it can turn out that when you open the door or or monti raises the curtain the thing that you were expecting uh turns out to be a value but not necessarily of that thing that you were you were setting yourself up to believe and so that that young individual spitting the coin out um i'm maybe i guess i was just trying to be a little bit facetious there that it holds value but it doesn't hold the same kind of um tasty value of say having a lick of a nice cream cone so again it was it was trying to move away from the idea that the belief he stopped alone is satisfactory or necessary i i always get those two mixed up but i think i was just trying to reinforce the idea that you kind of need to have both so that you can compare and contrast so that you get a sense of what a person or what an agent is doing as they're getting past some of those shrouds or some of those um non observables into the observable state thanks we'll keep going um this is unpacking some of the concerns about the apparent pure doxastic belief oriented ontology of actin so ryan described earlier like is it about beliefs colliding and if so if it's purely doxastic then where is this um desire and um is there anything else that anybody wants to add here i would maybe um comment that i think this also relates to your to the earlier thing that was discussed how this confusion might stem from people misinterpreting or confusing the terms predictive processing and active active inference and i feel like this addresses it as well where if we just if it's only beliefs then there is obviously a concern that once we're in a dark room we can't because uh if we're only in the regime of predictive processing there is no way for us to move to a different state or there isn't the the formalism that this that describes planning in action um but as i guess we'll get to in the other slides when we do include this planning we suddenly have desire like terms appearing within the mathematical formulation as well yeah and i want to be i want to be clear it's not like they're thrown in at hawk right they emerge naturally as like necessary components of any kind of you know like using this sort of framework or or building it out to involve planning so it's not uh like that they emerge naturally and necessarily it's not uh it's not because someone's just kind of tacking them on great um and this is referring to that minimization of g uh the minimization of g drives the agent to seek out observations that will reduce uncertainty about the best way to subsequently bring about phenotype congruent or preferred outcomes so that's the information seeking another way to put this is that under dai an agent doesn't simply seek to minimize prediction error with respect to its current sensory input it seeks instead to minimize prediction error with respect to its global beliefs about the environment which entails seeking out observations that are expected to generate prediction errors such that uncertainty is minimized for the generative model as a whole and um another aspect of this type of global prediction error minimization process is that it pertains not only to beliefs about states in the present but also to beliefs about the past and in the future and so this is leading to one of the claims which is that in addition to desires dai captures folk psychological experiences associated with the drive to both know about one's current states and learn what will happen when choosing to move to other states among other parameters in a generative model let's go into this generative model and come to the darkroom simulation itself um and we'll unpack it more uh in future times but the prior over policies e of pi they write that um when the agent repeatedly chooses a policy this term increases the probability that the agent will continue to select that policy in the future at the level of the formalism this corresponds to an agent coming to expect that it will choose a policy simply because it has chosen that policy many times in the past this can be thought of as a type of habitization process but it doesn't have any direct connection to preferred outcomes because e of pi is not informed by other beliefs in the agent's model so here's the partially observable markov decision process just one representation of it and e is sort of floating there influencing but not being influenced by and we had some interesting discussions which we can return to another time about how we might be able to assess whether somebody is engaging in a behavior because they simply have habitually engaged in that behavior but they understand the actual mapping of how policies um relate to outcomes or in another case where somebody might be engaging in a behavior not because of habitualization but rather because they have um some sort of deviation from the appropriate mapping between policies and outcomes and maybe there's other ways so I thought that was very interesting how even within the same sort of sparse and first principles model similar behavioral outcomes might be observed on like a behavioral manifold and then different kinds of perturbations or evaluations might reduce uncertainty about what is giving rise to those conversion behaviors in in that case um okay dean with the ice cream truck and the eye no i'm gonna i'm gonna wait because i think that's i think this one in the and the next one are better to take up in a one two part because because if if ryan can come back there's some good questions i want to ask them about those two sections because this slide in the next one are kind of anti uh dark room and so I want to kind of pick his brain about what you know how we can sort of convince people that that that dark room problem is for such a small subset of the population that majority of the stuff that we're talking about here is done in in sort of social settings and social context and there's a third party sometimes it can observe an observer and what role that plays in trying to figure out what's what's going on here as we move from the from the idea of decisioning and and so forth so I'll wait great so here's part one of that and here's part two looking at uh social learning theory a little bit but let's go to the figure and to the simulation uh this is figure one simulation of an active inference agent deciding whether to eat some ice cream and um there's various ways to go about describing it but one of the key pieces if anyone can give a remark about what is occurring here is that there's three cases that are being compared there's a no desire case a weak desire reflected by this p of o and a strong desire case where the ratio between the two alternatives between um wanting the ice cream and or between the ice cream state and the no ice cream state is a sharper distinction in this second row so there's a case in which ice cream is in the fridge but where the kitchen is currently dark and so the agent doesn't know whether the fridge is to the left or right so it's like a t maze set up with three possible cases one where there's no preference for observing ice cream or not a weak or a strong preference for that um and then under these situations we can ask about what happens in terms of the action as well as the um beliefs and valence updates what else would you add about that ryan well i mean i think the point you know the point of this is just to show uh well i guess makes a couple you know i mean so dean mentioned the um you know the issue with the dark room right i mean as he said i mean as you said i mean this is essentially just a particular very similar to to like a t maze right i mean just putting a putting a different sort of semantics on top of it but the you know but the idea is just that um it's just set up to show that you know even without any kind of desire right just just resolving uncertainty um the agent will be driven to leave a dark room or do something to get rid of the dark room um so you don't need a desire right but but in fact any realistic agent is also going to have desires so um so in the case where there are desires then so there's two different reasons why um no active inference agent will ever um stay in a dark room or be motivated to stay in a dark room like in all cases it will have a motivation to to leave so i mean the point is just to show right in some explicit simulations that um generically right like the this kind of dark room problem um just will it will just never apply um to to active inference um again the dark room thing only applies if you assume the thing is just doing something like predictive coding but then somehow also assume that it's making decisions um but that's not active inference um because it's not a theory of decision making awesome um here is some description about the strength of the desire could have been shown before the figure but we wanted to introduce the the simulation setting and like as described there is um a uh desire as well as an information seeking and i believe that's the two reasons why the agent will escape the dark room and there's there's other escape patches depending on how one constructs their model um potentially like a hierarchical model might have a slightly different um explanation for why an agent does or doesn't stay in the room but even just within this single level model there's an information seeking as well as a preference realizing reason um any other comments here okay dean just a quick thing maybe right in the in the island i'll put you in the island go for it yeah uh so maybe ryan just just a just a quick this last last couple of senses however preference distributions are set to zero oh i'm sorry i might still on the previous slide dan no sorry can you go back one yeah go for it okay so however a preference distributions are set to zero as in our no desire simulation such that no one comes desired over any other net and active infronzations will nonetheless be driven to choose behaviors that will maximize information gain okay that'll make sense that's just the background this is the part i was curious about while this might reasonably be considered a motivational influence it is prima facie less plausible that it should be considered cognitive i may therefore be better seen as a type of doxastic drive here the formalism may therefore help us to recover and potentially nuance the folk psychological distinction between desire and curiosity these types of drop drives seem to differ fundamentally i i agree with you but i was wondering if you could maybe explain why that should be something that we should keep separate the idea of desire and curiosity i know you go into explaining it a little more into the paper but why why did you want to point that out to readers well i mean i think there's a couple things i mean i mean one i think that you know these things are both driving you know decisions about what to do right so in some way they motivate they both motivate behavior right but but at the same time you know and i think uh this is something you know that alex keeper and you know pointed out when we were writing this as opposed to just you know so it's not just me um but you know pointed out that um you know there is this kind of um fairly clear difference um between the um this uh you know curiosity or epistemically driven um sort of behavior um from the kind of desire driven behavior and the the the kind of clear differences is that the um this epistemically driven motivation isn't um it isn't driven toward anything right it doesn't have a target it's not trying to get one thing versus another um all it's trying to do is just um you know just become clearer about what's out there um essentially right so so the fact that it lacks a target um is really i think what makes the what makes the distinction clear um but they i mean beyond that i think the motivation was just a was just to show that um you know like just to you know generically right like the belief desire intention principle doesn't really um doesn't really say anything about uh you know types of types of desires there's nothing really explicitly just in that very very simple um um way of way of um describing the framework that that separates out uh information seeking from from reward seeking right um so you know so you could right if all you're doing is trying to you know compare active inference to um to you know the bdi model very simply stated then um then this is able to kind of nuance a little more right like different different things that that um might look like you know cognitive motivation and all kinds of things but that are separated those um um on you know why why decisions are made for one thing versus another that are more kind of tied to beliefs in a different way um you know that being said i mean at the same time i think that um you know like full psychology more um more broadly i think i think uh very intuitively and naturally um does you know recognize the difference between things like curiosity right and things like and things like reward driven behavior right those are concepts that we have just in our natural kind of full psychology um you know so active inference captures those right i mean i um yeah talk about this um just as an example of you know how how intuitive i think in like natural um the sort of aspect is um you know and i just like think of like most of the behaviors that say like my little dog does um you know if i look at if i look at what my dog does 90 percent of the time it's way more information seeking than it is reward seeking you know she searches out for some food a couple times a day but i take her in the car she's pulling left and right to see what's out the window and how that changed all the time you know any little sound perks up looks right none of those things are reward driven they're all just information seeking um so i mean very very clearly that's a big part of um you know what drives us and other animals to do um so so the point was just uh just to kind of show that active inference captures that and in a certain sense it adds some granularity to um to at least what the bdi model says but but um in that case the bdi model is kind of too uh too coarse grained and i think normal full psychology does include those things already great um also to be returned to dean yeah because ryan just basically answered it all these two images are basically doing is separating out the the sort of goal directed from the curiosity which is you know the the image on the right is basically a documentary about the franklin expedition where they were going out to the event horizon but they never returned so did they have a goal probably was that their curiosity being carried out i think we i think we've got to be able to make sure that we appreciate both so that's all i wanted to know this makes me think like um are you driven to watch the sunset over the horizon or are you driven to find out what's at the end of the rainbow and what's over the horizon and that's the sort of like infinite open ended curiosity drive beyond the horizon versus the preferred specific state that one can desire in terms of their um observations and reduce the divergence there but cool um wishful thinking we'll come back to we're just leaving notes so that we can um have more to discuss later because these are all like there's such vital threads um ryan and maxwell and alex because they um touch to our day to day experience in a way that few other frameworks and and even papers within act imp do so affect and the role of affect and precision curiosity these are all just like terms that touch humans um you know one thing that i you know that i think is probably worth just touching on i mean again something maybe you guys would want to talk about in the future you know maybe when i'm not around it and the other and the other section's not sure but um you know uh so there is a probably a distinction that's worth keeping in mind between curiosity per se and um and goal directed information seeking um i think some of the you know examples you guys have mentioned have been more kind of along the lines of one versus the other the thing it might be good to just keep that distinction clear that um you know and and vanilla active inference um it is really something more just kind of like curiosity you're just kind of like independently driven just kind of you know look where you're going to gain the most information um whereas you know the construct of directed exploration um you know and reinforcement learning and um and would also you know often something that can emerge i think a little more clearly and like sophisticated active inference um is um you know is information seeking specifically or you know in the service of knowing you know how to get your goal um you know the the the curiosity version that we're showing here it has the effect of helping the agent get what it wants but the but the drive to seek out information isn't actually itself um due to the fact that the agent thinks it will help it get its goal um you know it's just driven to seek the information independently um whereas um whereas uh in in other yeah like in sophisticated inference um the the agent is actually doing something more like i'm going to look over here because i think looking over here is actually going to help me get to what i want better right so this is kind of strategic information seeking and that's that's a little different than just like intrinsic curiosity so um it's just something to keep clear that's quite interesting it kind of ties a braid back but let's explore that later here we have a nice uh clean representation of the summary of the main argument their proposed solution is somewhat deflationary in the sense that it simply argues that the functional role of desire not the experience of desire is straightforward straightforwardly present in the dai formalism and then in more detail they argue that and they provide four points which it'll be great to go over with with the authors based on those considerations the apparent problem posed by purely doxastic looking constructs is simply not a problem there are beliefs and desires in the active inference framework um perhaps we could have uh explored this more with the formalism so we'll we'll bring it up to talking more about f but won't go into it now we'll return to the letters later and then the appendix is very informative there's a description of all the states and the factors that are used in the simulation and in the model stream one which is by the way ryan it's um our our most popular series was model stream one it was it was um it was a fan favorite but um it's some similar concepts and matlab script so the appendix describes how the figure one results were generated and maybe we'll see if anybody can run that and execute it we can play with a few different things and take some of these qualitative linguistics and even mutate the simulation a little bit see where that takes us we'll close out with our usual closing slide um so who would like to take the first last word yes dean then yeah first of all thanks for uh shepherding us these three cats through this paper because there's an awful lot of stuff to try to cover in a pretty short period of time and it was and it's easy i think to sort of go i don't want to say tangentially but go deeper into some of the parts of the paper because um uh for me it was one of those ones where i would go back and and have to reflect on something after i read a section and then try to fit it into the larger picture that was ryan i think the paper did a very good job of sort of ordering through what you were trying to say in terms of the reassurance of how the the quantitative uh and the and the modeling of the psychosocial could could be seen as working with each other as opposed to though this doesn't answer what this other thing questions but it does cause this was a paper that for me i won't speak for the others but i had to read the section and then try to plug it back into the overall narrative that was being told and again i think part of that is because the the translation from the stuff that's typically seen as the quantitative part is hard to move into the qualitative part i think that's that's a challenge for anybody that tries to move back and forth between those two things but uh yeah i mean in terms of in terms of making a case and and providing what i think people need to sort of see the two in the same light um thank you appreciate it sure i mean happy happy if it's helpful rocket yeah i think um in it in it uh i guess there are two points that i like to have touch on firstly i think i thought it was very helpful to map these folks these concepts that everyone even though everyone uh i guess that's the nature of psychology that everyone will have a slightly slightly different um probably distribution on of what they these terms actually mean but i think it's in just learning about active inference it's helpful to link the mathematical formulation and even just the active inference ontology which itself can be very cumbersome at times to these very um intuitive concepts and i think as we get we're starting to explore modeling within the active flap as well i think it's it will be really helpful to use this isomorphism between active inference and folk psychology to explain the the behavior of agents within these models and not beyond at our group problem when there is some unexpected behavior of an agent we can directly look at okay how how did um this tensor change its values and describe it with this uh folk cycle psychological ontology so i think personally that's uh the most exciting thing about this paper for me um yeah well here i mean if that's i mean if that's the kind of thing that you're interested in then i mean that's kind of the whole um it's kind of the whole motivation of you know for instance like computational psychiatry right which is like the area that i currently work in right so you you take for instance clinical populations where people may behave in unexpected or ways or ways that don't necessarily seem like they're all that adaptive right then you can just fit these models to their behavior and you can figure it okay well what is making their behavior abnormal right is it is it something about an overly precise preference distribution or is it something about um you know the belief that states uh states transition to uh to um in too volatile or uncertain away or you know things like that so i mean it is the yeah i mean the kind of uh you know the major point is you can use these models and empirical studies to figure out what the mechanisms are that are leading to healthy and unhealthy behavior um and you know that can give you kind of guiding information toward you know designing good treatments or you know trying to to measure these things in a more quantitative way things like that and um and the uh you know i'm just as i've said i mean that this is all um using using tasks that uh you know that involve some kind of goal right like seeking some kind of reward or social approval or you know whatever it is that you know humans seek out you could do that right unless you have a straightforward way to map um the formalism each element of the formalism to you know the reason that we think people behave in the way that they do right so so um so i guess i'm just saying if if that's an interest of yours then i would think that you would um you'd probably be a fan of a lot of the a lot of the computational psychiatry literature more broadly both you know both the active inference part which is a lot of what my lab does but but also just the broader um computational psychiatry community that uses reinforcement learning and diffusion models and all the other you know all the other classics that are out there cool and it opens it up to what organizations want and what the cells want and all these other um transpositions well ryan really appreciate that last minute um belief or desire combination thereof to join us it certainly helped resolve our uncertainty a lot and it's a great conversation in the coming two weeks we're going to be with hopefully some more authors and more lab participants and just i'm looking forward to taking some notes thank you see you all soon and bye