 Okay, hello, it's September 16th. We're in cohort one of the active textbook group. It's meeting 15, or I guess, yes, no, 16 on, let me just update the title. Meeting 16, we're in our second discussion on chapter six. So, are there any overall comments that people would like to address about chapter six or any other questions or points to add or just areas of chapter six that they found to be interesting? Distinction between the interceptive, exerceptive modalities, which have been mentioned in box 6.1. And I think, Kristin, with Michael Allen, if I'm not mistaken, has published a recent paper just today, exactly on that topic in which they've somehow tried to provide a computational model for the first time for the unification of the interceptive and exerceptive modalities. So, since this chapter is also about the modeling of those agents, I think this paper in the body's eye, the computational anatomy of interceptive inference might be relevant for anyone who wants to model these modalities for the aid of active inference framework. Thank you, very nice. Thanks for sharing it. Let's look at box 6.1. As people see it, what distinguishes exterreceptive, proprioceptive and interreceptive modalities? For example, why are proprioceptive and interreceptive separated? Not because I couldn't hear you. I don't know if it's on my side or connection was unstable from your side. Sorry. Let me try something. I'll be gone for a second. Am I back? Okay, so yeah, what? Yeah, Brock, I do need the star link. Why are proprioception and interreception separated? And why are these three categories modeled as different? Why have they been framed as different? Biological and neurological mechanisms, neural mechanisms that somehow control those different kind of perceptions, because for the exterseptive perception, the organism would necessarily need a kind of sensory, I mean, the sensation it gets from the environment is just on the level of its, of the imminent stimuli that it gets from the external environment, but for the interseptive modality, the sensation is just the internal sensations and so on, and for the proprioceptive, it's related to motor control feedback loop. So I guess because of their different mechanisms, it might be helpful to talk in different terms about each of them, but I'm not totally sure about that. I agree. I think one aspect, even in a pre-blanket or non-blanket modality is like, they're about external things versus about internal things. And then between proprioception and interception, you're absolutely right that they have different neural mechanisms and physiological mechanisms with some blurry space perhaps. For example, like interception about blood pressure is somewhere in between like a muscular reflex and an organ reflex, but the tissues being addressed are different. Let's look at this paper. Again, we'll return to the book, but we're gonna be thinking today, and then especially going forward with like, how do you identify the... Let's go to this question, which we just, just to look at it. Which system are we modeling? What is the appropriate form for the generative model? How do we set up a generative model? How do we set up a generative process? So let's see if we can determine some of those aspects in this paper. Hashtag figure four, three, it is the very same. Okay, this is just a general pointing to a partially observable Markov decision process with a slight plus computational biology graphical veneer to match the color scheme. Probably not, but it kind of looks like it. Here is where we see the schematic of the form of the generative model. Variables are used to reflect that those variables can be fit from empirical data or just a priori set and simulated. But this is the form of the model. These are the shapes of the matrices and tensors that go from, that are going to address what their model is about. Does anybody wanna like describe what they see in this? What do you think they were trying to explain? And what do you think they were trying to model? What is their system of interest based upon what you see here? Actually, one interesting term that I came up with, I mean, used in this paper is cardiac active inference, the term that they've used in this paper to describe the kind of interrelationship between the heart rhythm and the, I mean, the interception. So this generative model, I believe, is just about modeling that relationship between the heart rhythm and the interception. So we can see, as usual, the B matrices C and A matrices that we've seen before. And, but in this case, they've been applied to the variation in the heart rhythm and how it relates to the interception. And so, yes. So, again, using the variables that we know from figure four, three in the book and their figure one here, the hidden states that are not directly observed are on the interceptive side relating to the state of the heart. On the exteroceptive side, whether there is a flower or a spider, the outcomes that are observed are the visual exteroception of a flower or a spider and the heart rate, probably a low and a high heart rate. The piece that links these two is the sensory attenuation between the high heart rate state and the mapping of exteroception. In the B matrix, you can just see that even the allocation of the symbols is different. Like when you're aroused, when you're in the, this is like heart open and then heart contract. So when you're in the aroused situation, B is how the hidden states change their time. You're always going to go from here to here, whereas in the relaxed situation, sometimes you go from this state to this one, other times you're relaxed and so you actually, you dwell within that state. So here it's like pump, rest, pump, rest, pump, rest. Here, it's 50% slower, just it depends on the parameters, but it's 50% slower because sometimes you're going to go from one, but it looks like as it's here that it's actually like, or it's twice as slow. Rest, rest, beat, rest, rest, beat. So these are the two inter-acceptive states. One has a preference for having relaxed hearts when flowers are around and having contracted heart when spiders are around. The A matrix maps on these two sensory modalities between hidden states and outcome states, just like any A matrix would do in any POMDP. And then there's also a transition matrix of the generative process. So it's a very simple generative process where either flowers are flowers or spiders are spiders. As they would say, you can use standard routines to simulate these techniques. I hope that the code are available. You can plot parameter sweeps as you explore the beta and the alpha, yeah. So here's like the beta parameter and alpha parameter. So you can do parameter sweeps with this grid search or other approaches and then do statistics on those outputs. Is this quote active inference? Well, it's the output of a parameter sweep on an active inference model. There's various other kinds of methods. So not to spend too, too much time on the paper, but does anyone have any thoughts on like that case? Or how they see it applying to chapter six's content? I think it speaks to a fulfillment cycle in active inference research. This work example helps a lot. I mean, if anyone wants to, especially because describing this kind of cardiac active inference as a kind of system that has a definite mark of blanket, it was a really interesting question at the first place. And I think they've done a really great job obviously in that regard and thinking really creatively and out of the box about how we can describe that whole system with the aid of using markup blankets to describe its internal and external environment and so on. It also relates to this earlier question. So how does the figure four three paradigm discreet and continuous state spaces with all the attending complexities? Oh, do you hear me, Ollie? Oh, okay. Okay, okay. How does this representation connect to? Can you hear Daniel? Because I can't. It's a story for another day. Some of the incredible, the truly incredible internet. I'm gonna try one thing. So I think that, or Ollie, can you hear me now or no? I'll try something. Here's what's interesting with the Wi-Fi and then we'll continue because it's not a Wi-Fi learning group, but with the hotspot, there are a set of people who I cannot interact with but with Wi-Fi, I am able to interact with those people just to know. So we see the POMDP representation. Now, what is not in this paper? Here is a graphical layout of the POMDP, okay? What is not in this paper? A figure six one or indeed a figure one one representation? So why is that? I'm not sure I've understood your question quite clearly, but could you please elaborate on that a little bit? Yes, yes, sure. So when we look at papers like vassal et al, communication is active inference, social and active, active inference, loops, loops, loops, you know, Vatican et al, remote teams, loops, loops, loops, Markov blanket, interface, loops, entities, things, yet in the application, what is specified is not the loopy situation. What needs to be specified is chapter six, not chapter one. So I find that very interesting because so much of the focus, attention, discourse has been around abstractions around Markov blankets and their interpretations from various stances and about boundaries, physical boundaries and conceptual boundaries, but it's a non-controversy when it's written this way. And there are Markov blankets between many of these variables because it's a Bayesian graph that's not fully connected. So every Bayesian graph that's not fully connected has Markov blankets trivially. But this paper doesn't need to describe what is, this is enough to say what's inside and outside. The organismal Markov blanket, if we want to even think about it, is like here in this line kind of, not on the Bayes graph, but it's between the spider and the flower outside and the vision of the spider and the flower. But that's like taken care of with the A matrix of the exteroceptive modality, which is I think why this recipe does not focus on which thing are we modeling, but rather which system are we modeling? And yes, we're going to pursue the generative model before the generative process, even though they may have symmetry or they may be very similar. But I just find it very interesting open to hearing what people think, like why the cybernetic loop, action perception loop is used so much in contrast with the form of the model that's actually implemented, which is a different topology in a different discussion. I know that they're integrated and compatible. Just one comment on that is, you see, at least to my understanding, Marka Planckett is a kind of a kind of think modeling tool or a kind of a way of thinking about the, basically the ontology of, I mean, ontology in the sense of philosophical use, the ontology of the system that we're about to model. So it's not necessarily the relationships among the different components of the Marka Planckett. Is there something not necessarily physical or tangible relationships in that system? So it might be kind of just informational as we've just, we saw in I think chapter five, if I'm not mistaken, the information passing among the different components and so on. So at least I think the reason for not using the schematic model of Marka, I mean the cybernetic model of Marka Planckett instead of using just the schematic form, probably it's because of these kind of none, let's say, non-physical or non-tangible relationship among the different components. By non-tangible and non-physical, I mean something that can't be explained in terms of its ontological existence. I don't know if I could express myself in that regard or not, but. Thank you, Ali. Brock and anyone else? I was just going to, I don't know. I feel like personally, I'm just kind of at my, the affordances that I have to get pragmatic and epistemic value here without like doing some modeling of some kind. Yeah, I don't know, let's just. Good, thanks. I mean, this is the second discussion on chapter six and then this is setting us up to actually pursue a model. So we should be providing these things. Yeah, I'm just, yeah, I'm just saying it's definitely ripe for that. I mean, yes, Rohan? Yeah, so actually just to get some clarification, you're asking why the form of the model is as mentioned in chapter six of topology given in chapter six which is chapter one? Yes, I mean, they're compatible topologies that actually are describing an overlapping set of notes. I was just calling attention to the amount of discourse and focus on the potentially more familiar, more schematic action perception loop and the way that that draws so much attention to the organism environment interface whereas when it comes to the empirical modeling, that interface never had to be specified as like a first class type. It just was included by the by. We just said there are spiders and flowers out there, hidden states, S. And then there's observations of, and there's no error. There's no error in this. Right, but there would be no error in the generative process itself that model might have error, right? So. So there would be the... I mean, that comes from some measurement issue, right? So when you measure it and you confuse, say, spider for flower or flower for spider, then you would get some error and there may be, that's where I guess you can talk about the Markov blanket mis-absorbing something. So I'm not sure what exactly it's thinking. I mean, they're compatible, right? So in this case, there's no error, but if there was error, obviously, there would be something wrong. Let me actually just correct. This, I believe, BE, I didn't mean to say that this is the visual error. This is saying that they're unchanging. So they're either one or the other. Here, parametrically, it's possible to introduce visual error in the A matrix. But I'm just pointing to the architecture of the model doesn't require even engaging with this. It's totally compatible. And the observations are the sense states and the actions are the active states and so on. But as presented and as constructed, if we're gonna use a POMDP architecture, we don't necessarily even need a representation like this. Yeah. Jessica? Okay, that's interesting. Hi, yes. For me, the reason why, like it's like two-part, like the reason why the vision craft is so popular, I think maybe it's also the reason why it was used. This is the simplicity to show the concept and the relationship. So like the new one, it's great for, I guess, for the paper that it was created, but it doesn't show the relationship as clear as the other ones, it's very simple. So I think that's why it was used, maybe like remove a lot of extra information that the second people illustrate complexity in it. But the simplicity of that graph is why, like it really encapsulates the concept. And it's also the reason why it's so popular because it's so simple. So I think that's, you know, the relationship with it. From how I see it. Yep, totally makes sense. Thank you. And once we see that there is a mapping from the schematic cybernetic view, then we can probably fill in a lot of the parts of the PMDP, but just to look one more level here. I mean, the code, I posted the code below. So it's in MATLAB, which people may or may not have access to. Oh, sorry, okay. Ali and then Rohan. No, I just wanted to point out one thing that I forgot to mention before. One thing that might be helpful to keep in mind is that actually in this generative model, there is a kind of unification or a kind of integration of interceptive and exterceptive modalities because as they pointed out in the paper, this agent, these agents that they're modeling adopt two modes of engagement with the world. One of which is the relaxed and the other one is the aroused mode of engagement. And because of that, there are two sorts of hidden states related to each interceptive and exterceptive one. So I guess if they wanted to put that kind of that kind of interrelationships between these kind of modalities in that simplified schematic model, it wouldn't necessarily clarify these kind of integration between the modalities. So one reason might be related to that. I mean, peculiar case of, peculiar situation of this particular case here. Thanks. It also speaks to the step in the model of the form of the generative model. And then I think when people get married to the cybernetic scheme, you can imagine, well, then there's two nodes inside of the internal states. There's two modes you can go between. But now you're like mixing and matching what arrows mean where some of the arrows are reflecting causal influence and some of them are reflecting transitions between mutually exclusive states. And so then we end up back into the wire frame nightmare where it's like a schematic where some edges are tactile connections and some are causal links and some are mutually exclusive transfers of information and all these other ways that people visualize models. And this representation reminds us that we're dealing with variables and states and their connections. Jessica. So I think that was from before. Okay, okay. But yeah, but I guess I can add a little bit. I think it's like the visual, it has to do with what he's trying to convey in that moment and how much depth of information do you mean? I think like I already said, like for this one, you need an extra information so that all the model wouldn't work. Well, just share like with the idea of what the model is maybe this version with the extra information might be too complex for the other ones there's a better purpose. So it's like, what is the purpose he's trying to accomplish and what do you want to communicate? And depending on that, then models with more information and less information are more successful. Yep. Thank you. Totally agree. I mean, they didn't do any of the loop at all. They just went straight to the POMDP representation. And also this isn't the only form of an interoceptive, extraceptive integration model. One could brainstorm a whole host. And again, that's what the recipe is for. Well, we could have a mutually exclusive transfer between these two states defined by these two B matrices which entails a policy variable switching between the two or we could have a continuous variable called arousal and arousal is engaged in some sort of attenuation relationship here. So this isn't even the only form, but it shows us a form. Okay, soon in 20 minutes, we're gonna be heading into chapter seven. So let's preview it so that when we're working on this in the next week, preparing our questions, starting to think about what we want to be modeling, we'll kind of know what's gonna happen. All right, chapter seven is active inference in discrete time. People can still hear me? Okay, I think so. All right. So here is where we cross the Rubicon with modeling. This is abstract up till now. You cut out there for a second. Oh, okay. Okay. It's all good now. Yeah, just, okay, start from the preview. Thank you. Chapter seven, we're crossing the Rubicon together. And it's been abstract up till now. First, what is addressed is perceptual processing. You'll find a lot of resonance between this chapter seven and step by step, which is model stream one, maybe it's step hyphen. Yeah. This paper, step by step tutorial, is going to do a certain walkthrough of adding incremental complexity into the model in terms of model parameter depth, okay? It's model stream one. There's four streams that Ryan and Chris White did. Static perception, dynamic perception, dynamic perception with preferences and free energy minimization, dynamic perception with free energy minimization and flexible policy selection through uncertainty modulation. We're gonna do something similar in chapter seven. They don't do that upper left corner of step by step. They go straight to hidden Markov models for dynamic perception. This is analogous to a Coleman filter or any other number of general Bayesian filtering schemes. We have a prior D, hidden states S and a sequence of observables through time, T that are mapped to hidden states A. B describes how hidden states change through time. So we have the temperature is a hidden variable and then we're getting observations from the thermometer that are mapped to the real temperature by A. Maybe it's a good thermometer, maybe it's biased or whatever and then there's the temperature changing through time. Then there's a very Ali-centric discussion of listening to music. This has to be one of the most glaring errors in the textbook. It's very Warhol-like though. So maybe it has a certain aesthetic to it but the example that's gonna be provided is like a musical listening example. So the A matrix is a very similar to the A matrix is emitting notes based upon the hidden state which is the score of the music. The B matrix and they have 70% accuracy. So this one-tenth distributes through the matrix. So it's just to use like integers here but really this is like 0.7, 0.1, 0.1, 0.1 because it has to add up to one but when the hidden state of the score is in the first note, they play it 70% of the time and then the other three notes, they make an error 10% of the time each time. 70% of the time the musician hits their note. The B matrix is how things change through time, the hidden state. So this is the transitions in the score. And here there's a 97% probability against distributed throughout with this one over 100 just to represent it with integers. And there's a prior with no uncertainty or ambiguity of just it begins in the first position. And so this we can even see like doot, doot, doot, doot. You know, I don't wanna use ABCD but it would be like BCDA in the musical ontology. So here's BCDA like that. But then there is an error in this one simulation run. It'd be probabilistic but they're showing for didactic reasons a run where the third point in the third observation, there is an error. However, it's heard as the correct value. That's a pure inference example. This is not active inference. This is just Bayesian models of perception. This kind of sessile creature is rather uninteresting. Okay, now we move from a hidden Markov model, crouching tiger hidden Markov into a partially observable Markov decision process. A hidden Markov model is a partially observable Markov model because some parts of it are hidden, some are not. But the key piece here is the introduction of the decision. We know that we model that with policy that influences how states change their time and that policy selection is governed by G, figure 4-3. People can still hear because I see all these like frozen for me. Okay, all right. Okay, thank you. Yeah, it's just strange stuff. Here's 4-3 again. Here, when we're talking about policy selection we're talking about states that haven't happened yet. Observations and hidden states that haven't happened yet. We see a restatement of equation 2.6 with expected free energy here broken into epistemic and pragmatic value. Now to flesh out this POMDP, we turn to a classic PAR and Friston example which is the teammates. We're not gonna go through every single part of this teammates right now but it'll be a key area for us to unpack. Why do the matrices have the dimensions that they have? Where are the four locations that the mouse can be? Here, down at the bottom, to the left or to the right? So this is not like a mouse moving in continuous space. It's like there are four locations it can be in. Those are the columns. There's five hidden states of location which we can see are like based upon whether when it goes down to the bottom to get a Q, whether the hidden state is gonna reveal the Q or not. And then figure seven four and seven five, you'll see differ, first off, one of them is bolded. This one is bolded, this one's unbolded. But here we see a one in the L column because the black is on the L. And here we see a one in the R column because the black is on the R. So understanding why that is is going to be the crux of understanding how to construct an epistemic informative Q as a hidden environmental variable in a generative model. Here are the B matrices, transition probabilities. Here's the preference vector. We'll talk about what C1 and C2 are. Here are the priors. We'll talk about what D1 and D2 are. Now there's more unpacking of the epistemic value itself. So in the equation above, there was a restatement of equation 2.6 which partitions the expected free energy into the negative epistemic value and the pragmatic value. Now we're gonna follow up on the epistemic value, I of pi, the epistemic value of a policy and break that down into these representations. Here are simulations. Simulated epistemic and pragmatic behavior of a rat foraging in a teammate. So we'll try to model what these runs are. I think it might be one small note about the book that especially if they knew that it was gonna be black and white, which it seems like that was known, this is not ultra helpful. Perhaps people disagree. But another one. What? I said another whirlhole. Yeah, this one, this one. You can even see the Campbell soup. But this, I think if someone was familiar with reading these traces, they would still want to understand what these different lines were and like what the dashed one is. But if people are unfamiliar reading these kinds of error residual traces, I don't know what is being shown here. Figure 7.7. That was the simulation of the rat getting the epistemic cue and going to the left. So instead of just taking a 50-50 shot on where the food was, the rat navigates this Explore-Exploit dilemma. It engages in an epistemically oriented action or an action driven by epistemic value to reduce uncertainty, get the cue, and then it secures the food. In this box 7.1, we see a discussion about parameterization of uncertainty and how precision is the inverse of variance. And this omega is the precision variable here, just like Mark Solmes uses, but others use different ones. The lower this value is, the more even things are. And so this has come up in the ice cream example with the folk psychology live stream. Like if your C vector is 0,6, negative 6, versus 0,500, negative 500, that's like a sharper preference distribution. So it's the quantitative values in the C for example, that specified not just the direction, but also the intensity of preference. And then analogously, precision can be specified in a continuous form. Here is a reference to earlier Par and Friston work on vision that has to do with ocular motor foraging, which is heavily epistemically driven, but then they are using this example to explore how precision, the viewer shows a version to the upper left square when it is specified with a less precise, more ambiguous likelihood mapping. The lower left square is epistemically attractive when the transition probabilities are specified as more uncertain. We'll try to understand these examples. Learning a novelty 7.5. Here we take that POMDP and like we push it back in the genealogy. Now we have a priors on the hidden states, which are themselves equipped with prior beliefs. These are Dirichlet conjugate priors, but they can be of other forms, but they're just shown as Dirichlet here because it's like a convenient statistical distribution family for variational inference, which we explored in several chapters ago. These can be considered as hyper priors but in the end, just variables on a graph and it's not like they're tagged with being a hyper prior and you can imagine priors that nest on them. Exploring these potentially open-ended Bayesian graph structures is why it's so important to have structure learning like Carl has pointed out for years and why we want to have parameter sweeps across cognitive model structures in active blockference because why stop? Why not have another prior on this one? So we should have the notation and the capacity to sweep that. Here is a discussion on conjugate priors. A conjugate prior means that when used to perform Bayesian inference, the posterior belief will be the same type of distribution. So previously we talked about how like the hidden state could be a continuous variable and then the internal state could be like a discrete variable. The temperatures continuous outside and then inside is it hot or is it cold? Is like what I'm caring about. So the families don't have to match but when we use conjugate priors, it makes it so that where you can design it to play nice, it does play nice. But we'll work through that also with anybody who wants to go into more detail on the math and because these are just like Bayesian statistics points. Here, theta is used to just describe parameters. It's a very common notation but not 100% pervasive to use theta for parameters just generally. Here, there's more unpacking of the update rules. This is probably of secondary consideration for just constructing a model. Here we return to expected free energy and several more representations beyond what was seen above and beyond what was seen in equation 2.6. Here the learning model that was presented above the POMDP with another generation of genealogy is described which is drawn from these two papers. Then structure learning is introduced. We'll see how deep this one goes but these papers are quite interesting and connected to Bayesian model reduction. Here, a hierarchical model is presented. Here's figure 4-3 in the dashed box. Now there's a figure 4-3, you know. Yo dawg, I heard you liked figure 4-3. Then there's another hierarchical model simulation from the above showing belief updating again with these trace diagrams and then there's a summary. So it's a somewhat long chapter. It contains like the music example, the rat example, the maze example, the eye example. So we're seeing like data points in model space. In the last three minutes, does anyone have any thoughts or questions? How are they going to tackle chapter 7 and as they start to construct their model if they want to? I am just attacking every direction that I can because it is quite dense still and kind of unapproachable I think. For me anyways, it's just quite difficult to get a grasp on it from the textbook specifically. At this point, the generative model, like what does that mean in code and where does that correlate to this particular function or that particular time step or whatever. It's, I don't know, it's just a lot. And so I'm just trying to, from yeah, I don't know, from just generally building up simulations which are just things that you can put together in a browser like with simple JavaScript to like CAD CAD and active block prints and these sorts of things and trying to grasp at some real world situation. So maybe that rat, you know, teammates is a really good place to focus on for me, like a good intersection between the book and something that isn't like the frog jumping or is it not jumping is, you know, so contrived it's really difficult to think about modeling that and in a way that would help explain what's going, it's like, yes, I knew the frog was jumping or not jumping. So that doesn't really help, right? Whereas the rat may be coming up with some left or right hidden state based on the queue that's partially, it's, there's some explanatory power there that may be realized, I think. So I'm just, yeah, I don't know, it's, I'm trying to bridge jump over the Grand Canyon of the philosophical conceptual, okay, got it, but the math, there's a, okay, but there's a gap there for me, so I'm trying to pull them together. Yeah, I have a question related to calibration of those priors and things. Is that ever explained as to how to go about doing that for any of these models? Yes, so model based data analysis is going to describe. Okay, so it's, okay, it's in the future, all right. I'll be for that. Chapter seven describes, just in our last minute, chapter seven describes the discrete state models. Chapter eight describes continuous time models. Chapter nine describes using empirical data for model driven data analysis. Chapter 10 is a summary. The textbook is, no. Yeah, yeah. We're accelerating towards the Grand Canyon, but I think, again, we have several months to unpack this together, so I hope that we can go quite far with this. So please stay engaged and add your questions and start building from both sides, play with the code examples provided, break them, and then start from the other side with a recipe on a situation that you care about, and then they're going to meet in the middle. Okay, thank you, everyone. Very well. Bye, everyone. Bye.