 Welcome, Blue. I'm about to play the theme song. Oh, everyone, welcome to Act In Flab, live stream number 35.0. It is January 4th, 2022. So also, happy new year to everyone. Welcome to the Act In Flab. We are a participatory online lab that is communicating, learning, and practicing applied active inference. You can find us at the links here on this slide. This is a recorded and an archived live stream, so please provide us with feedback so that we can improve our work. All backgrounds and perspectives are welcome on these live streams and we'll be following good video etiquette. If you go to activeinference.org, you can find out more about how to contribute or get involved because anyone who is curious and wants to learn more will have a niche for you and you will surely learn and create all kinds of awesome affordances. So if you're curious or you wanna learn by doing, please come get involved with Act In Flab. Today in Act In Live Stream number 35.0, we are going to be learning and discussing this paper, A Tale of Two Architectures, Free Energy, Its Models and Modularity by Majeed Benny in December, 2021, but kind of published in 22 and just like all the other dot zeros, the video we're about to do is just an introduction and a discussion. It's not a review or a final word and in fact, we're very much looking forward to the two upcoming weeks when Majeed may join us and hopefully many of you will join us and we're just gonna have a good time, talk about the paper, what have we learned from it and what are the implications. Today we're gonna be more focused on the aims and claims, the abstract, the roadmap, the background, the keywords and there aren't any figures or formalisms but we'll go through some of the key arguments which are very provocative and very salient. So if you wanna get involved, just get in touch and we will let you know how to participate. Let's begin with a little bit of an introduction and a warm up. So feel free to say hello and maybe just one thing that made you excited to discuss this paper. So I'm Daniel, I'm a researcher in California and I was excited to read philosophical work by Benny who always writes very interesting things and just understand how architectures of physical systems and their functional descriptions are related to the FEP. Dean? Thanks Daniel, I'm Dean, I'm here in Calgary, it's really, really cold and snowy but that's not why we're here. My real question was, so what do we do about this seeming failure or attraction to the ability to categorize? And so the question is, do we model up? And yeah, so reading the paper, I kind of tried to figure out the answer to that. We'll pass it down to Blue. So I like this paper from the perspective of like multi-scale dimensionality and also modularity like modules at different dimensions, the scale free aspect of the Markov Lincoln or the not scale free aspect of the Markov Lincoln. And anyway, I thought it was just really neat and a cool way to fit the pieces together. Excellent, so we have some slide with things that we wanna discuss in the dot one and the dot two and beyond. So at any point, feel free to add information there and we'll be writing things there. So we're not gonna get to it all, we're not gonna make this a 11 hour video. We'll get things down to discuss in 35.1 and two, but we can lead in with the big question, a big question, which is how do we think about modularity of different kinds like functional, effective and anatomical, which we'll be discussing later in active inference and free energy principle and why does that matter? Why does it matter how we think about modularity of different kinds? What does it mean? Here on this slide are some images related to just different uses of modularity, just modules like Lego bricks. Other times modularity can refer to like part of a network that's more densely connected, like a click. Other times modularity doesn't refer to hot swap ability but rather to a larger structure that's composed of smaller modules, whether they're the same or different. It's often applied to mathematical systems or biological systems. So there's so many aspects of modularity and today we're gonna be exploring, what are those aspects of modularity and why does it matter for the free energy principle? Blue or Dean, like any thoughts on the general modularity question? I got a quick question. So for me, when we see these sort of images that we externalize in artifacts and that sort of thing, when I started getting into the paper, the real problem for the modularity argument is what does localized mean when we're talking about FMRI representations as the tool? Does a bump in electrochemical impulse mean a clearly defined border? I don't think so. Of course not. Where one bump ends and another begins all inside the same energy field might be captured through FMRI imaging and is that misleading? Cause there's no moment of stasis because the magnets inside our heads keep pouring constantly. So that was again, another question that I sort of tried to carry on through as I was reading what the author wrote out. Yeah. Where are we talking about modules? Where are we modularizing? And does that map onto potentially realism and instrumentalism? Any big setting ideas blue or we can continue to aims? All right. So the paper that we're discussing is by Majid Beni, a tale of two architectures, free energy, its models and modularity, and it's in the paper, Consciousness and Cognition. Just to go through some of the key aims and claims and then feel free to add a comment. So the paper presents a model-based defense of the partial functional informational segregation of cognition in the context of predictive architecture. More specifically, to defend the modularity thesis, the paper confutes two counterarguments that lie at the center of Hippolito and Kirchhoff's 2019 recent computation of the modularity thesis. To confute is like to refute. The main insight of the paper is that Hippolito and Kirchhoff's counterarguments miss the mark because they dismiss a few rudimentary facts about the model-based nature of dynamical causal models and Markov blankets. So that's why we kind of let in with the big question about modularity. Why does it matter? Because we're kind of down the road a little bit, like, okay, modularity matters. It matters how we think about and analyze and design modular systems. And then how does it apply to FEP? And Hippolito and Kirchhoff make a computation of the modularity thesis, which we can look at soon. And then Benny is firing back with this current paper. So that situates us like in the biggest scope of thinking about modularity, H&K 2019. And here we are with MDB 2021. So can I add to that, Daniel? Yeah, so we didn't put encapsulation in here, but I know we're gonna talk about that a bit. But in order to be able to make sure that we don't mix things up, we have to understand that encapsulation is a separation of what matters or what we consider to be a signal from the noise. And I always think back to Nate Silver when I try to compare those two things. So it's a momentary on-off or it's a momentary in-out. Not that there's a walled off section per se in the brain like we can diagram out or represent as an artifact. FMRI images, however, might coerce us into thinking that that's what's happening, that there's actually a walled off section when that isn't actually the case. To avoid the cognitive overload, there are breakstop mechanisms in our mind. We don't have to tell our pancreas or our base of ganglia what to do, which then assists us in our ability to take the external states like these slides in. And I think that's what Benny was trying to point to, although he didn't probably go into that explanation the way that I just did, but I'm kind of in defense of this author. So I'm starting off right now. I'm in his corner. I'm ready to tag up and climb in the ring. So just to kind of piggyback onto that, like when you're talking about encapsulating, it's really talking about drawing a boundary around something and like isn't the Markov Blanket, like isn't that the ability to draw a boundary around something, right? So I don't know, like this encapsulating Markov Blanketing boundary drawing, like all of those are conflated in my mind and I don't know if there's like a separate way to tease them all apart or if I have correctly grouped them. Great, onwards. So I'll read the first two and then blue for the second two. The paper presents a model-based defense of the partial functional informational segregation of cognition in the context of the predictive architecture. The paper argues that the model relativness of modularity does not need to undermine its tenability. In fact, it holds that using models is indispensable to scientific practice and it builds its argument about the indispensability of modularity to predictive architecture on the indispensability of scientific models. More specifically, to defend the modularity thesis, the paper confutes two counterarguments that lie at the center of Hippolito and Kirchhoff's 2019 recent confutation of the modularity thesis. The main insight of the paper is that Hippolito and Kirchhoff's counterarguments missed the mark because they dismiss a few rudimentary facts about the model-based nature of dynamical causal models and Markov Blankets. Great, so nice philosophy abstract. Here is the roadmap where we are going and so this has a fun roadmap. There are nine sections. There's the introduction and then there are eight sections with or dare I say eight modules that have kind of interesting names that we'll be looking into and there are not any figures or formalisms but the sections are here. Maybe Dean will rejoin. And welcome back, Dean. C'est la vie. I need an internet cable from my heart to yours. The keywords are free energy principle, model-based science, modularity and then there's DAG directed acyclic graphs and then DCM which is gonna have two different unpackings as directed cyclic models and as dynamical causal models. And we're going to go through these keywords kind of as they appear in the paper. We're not going to have a separate section. So let's go right into the introduction section of the paper where we find somewhat of an epistemic preamble. So not to read the whole thing but Benny belongs to the clan of philosophers with Carnap and Kuhn who concede that the choice of theoretical frameworks is not completely arbitrary because some pragmatics affect the choice of theories. So okay, we have dispensed with the naive post-modernist perspective and more generally, Benny identifies as a naturalized metaphysician in the sense of Ladyman and Ross in that metaphysics is worthwhile when it is informed by our best scientific theories of the field. So it's kind of just like a classy co-informing process between science and metaphysics and it is drawing on this work by Ladyman and Ross. Everything must go, metaphysics naturalized. And then just to like look at the chapter titles because it's not a book that I was very familiar with. They're defending science. They're defending a scientific realism and a structuralism. So maybe we're not going to be in touch with the ultimate reality but at least we can be in touch with something that's constructively empirical and scientifically real. They talk about the unity of science, conciliance like E.O. Wilson and about causation in a world that's structural but also complex. And so that's sort of the preamble where Benny characterizes his own stance and kind of gives this background that is quite personal in some ways but then will lead into our understanding about the paper and the contribution. So anything to add on that? All I would try to quantify here is that he's not starting from a position of, did you see that? He's basically starting from a position of, did you see what I see? He holds that as being valuable. Great. That takes us to model-based science. So as we saw in the chapters of Ladyman and Ross scientists use models. It's kind of like saying we use maps on territories. And there's so many kinds of models. A few that we've seen are, for example, a graphical visualization of a Bayesian statistical model. And one example of that is like this partially observable Markov decision process that we often look at. And what do models do? Well, that's quite a big topic but they represent hypotheses as well as useful models of conceptual or empirical systems. And so there's many, many kinds of models and some of them can be reflected graphically. And in that sentence, graphical is referring to visual, like a computer graphic but also graphical like a graphical network. And so we're gonna be talking about the visual aspects of graphs but this isn't a study in aesthetics. This is gonna be talking about the topology of graphs in their network sense. But model-based science is just the kind of pragmatic perspective that scientists use models implicitly and explicitly. And there are certain things that we can say about those models. Anything that either of you would add on model-based science? I don't wanna know who disagrees. Who doesn't think that scientists are using models? Like is this just a total meaningless claim? Or are there controversies related to this? Okay, so let's look at a few different types of models specifically these graphical models that can be represented by networks. So one kind of special graphical model is called a directed acyclic graph or a DAG. And so DAGs as their name would have it do not have cycles amongst variables. So nodes are variables and edges are directed influences and there are no cycles in DAGs. And so it turns out that computing DAGs it's relatively efficient and it has known algorithms. So here is a image of a DAG that I got from a 2012 paper in ecology. And so the parameters are defined here and you can look for more details, but just visually these are different modules of the model. There's like this underlying process model. There's the individual covariate model. There's an observation model, whatever those things are but one can imagine that you can like start at the parts that don't have any parents and then kind of drip coffee your way to N and Y which are like the ultimate grandchildren nodes. And then N and Y don't feed back to change the other upstream parameters. So DAGs are not all of this exact shape or type but what makes them in a category is that they don't have cycles in the way that the variables are influencing each other. That makes them easy to compute and relatively straightforward. We can contrast that with directed cyclic models. So we're still talking about graphical models here. Directed cyclic models or DCM have causal loops in the graphical models. And so this can make certain kinds of computation more difficult. I'm sure people can think about different ways that that might happen. The direction of the edges determines the influence of one random variable on another. So if it doesn't have cycles, it's a DAG. And inference on those DAGs may be performed exactly using algorithms such as belief propagation or variable elimination. Like you can kind of collapse a subtree into a simpler system but with cycles it's a little bit more complex. So that's the difference between the DAG and the DCM directed cyclic model but then there's gonna be another DCM coming up later. Any comments there? Do you think that the looping can be kind of a spiraling kind of an accumulation effect that the DAG model which with it's narrowing and narrowing and sort of elimination focus. Well, he, Benny talks about the fact that they're actually complementary but do you actually think they're complementary as well? I think that they exhaustively describe the landscape of graphical models because either it has a cycle or it doesn't but it's quite literally the difference between like dropping a coin in a little machine that just dribbles down and then playing pinball where you could have all kinds of feedback. So I think they're mutually exhaustive but that may be sort of a deflationary stance because they're defined as contrary to each other. We're not gonna go too into detail here but sometimes you want to use a model that is acyclic even for a cyclic system. For example, within one time step it might be a heuristic approximation to make it a DAG model directed acyclic model rather than a cyclic model. So we're talking about our statistical models here not the topology of how things are connected in the real world but I just thought it was very interesting to talk about Bayesian networks and often Bayesian networks are framed as only DAGs. So the standard definition according to these authors the standard definition of Bayesian networks is usually one directional acyclic directed graphs. In higher dimension systems further probabilistic relations can be recovered from network cyclic properties and the joint probability of all variables. And this paper shows like for a two system where there's just a kind of back and forth between these two X and Y variables. Here's X, Y and Z in a three-way loop and then here's a four-way loop and then they also look at n-order loops. So it can generalize. So what was once only imagined is now proved. In other words, let's not get too confident about like what can or can't be done with one type of model family or the other because it's just one clever paper away for there to be an approximation or to be an exact solution. Now we turn to the other sense of DCM and this is dynamical causal models. And so these DCM can have graphical structure that also is directed acyclic graph but it's a different usage of the term. And so DCM, here we could point to many resources like Friston et al. the 2007 Statistical Parametric Mapping textbook or a later paper of Friston et al. on network discovery with DCM. And so I thought that 2011 paper was a little bit more relevant because it's about specifically inferring or discovering the functional architecture of distributed systems using dynamic causal modeling. What does this technique do? It uses Bayesian model selection. So comparison of the relative adequacy and complexity of different Bayesian models to identify the sparsity structure which is where there are edges or not or where there aren't which is the same thing as where they are in a graph that is best explaining in observed time series. The implicit adjacency matrix that's that sparsity structure specifies the form of the network e.g. cyclic or acyclic. And then later on we see unlike conventional approaches to network discovery DCM permits the analysis of directed and cyclic graphs. So DCM is an approach for looking at a time series that's the dynamic part through a time and then looking at how variables statistically cause each other. And so a statistical cause is like when one thing happens and then at some later time point there's another thing happening that's a statistical causal model like Granger causality. We're talking about our model. So if Bitcoin price goes up and then later oil price goes up across the whole time series so there would be a directed causal edge there might be a bi-directional causal edge or one directional causal edge it doesn't mean that in the real world one going up caused the other it's just talking about our time series data. So that's dynamic causal modeling and it can be applied to neuro-imaging data. Blue? I have a quick question here. So I get the time series that that makes it dynamic but I also thought like you could have functional studies that also fall into the category of dynamic causal modeling like gene knockouts, right? So like you knock out a gene and not necessarily a time variation but like a variation in the graph itself and so you knock out a gene and then isn't that also dynamical causal modeling or that's something different? One could have like the nodes be gene expression through time and then test how perturbations or counterfactual network architectures would result in different expressions through time. But can you take the time out of it and have it still be a dynamic model? I guess it's just snap-bots, right? So it's not time series but like gene series I don't know how the genes in the series to make it a dynamic model. If it were just a single time slot slice then it's a little bit unclear what the edges would reflect because cause is like effect through time in the statistical sense. So if it were just like I have like, you know there's 50 people here and 20 people here and 10 people here it's hard to say where's there a causal edge whereas if all three were changing then one could draw these causal edges. Okay, now we turn under the scope with a big question about modularity to specifically modularity and the free energy principle, FEP. Okay, so this is the big topic and kind of the entree that Benny is bringing to us to discuss and it's highlighting a really important issue which is the conceptual compatibility of modularity with the free energy principle. And it's a speculation that this conceptual point may well become significant in the course of scientific development of the FEP. So this is like a philosopher engaging in a scientific discussion because there actually is a lot on the table if we get modularity right or wrong. So what is the relationship between FEP and modularity? Is it compatible? Is it necessary, sufficient, incompatible? Like if we have modularity, do we get FEP? Right, that's sufficient. Do we need modularity for the FEP necessity? If we have it, can we not get the FEP incompatibility? Are we talking about the modularity of our world? Carving up nature of the joints or of our scientific statistical models, instrumentalism carving up our models at the joints that we designed. And then how does our stance or our perspective on this topic of modularity and the FEP influence our generative model? How we think and act, whether we're the ones collaborating and writing scientific papers or not. And then just to add one more note. Despite the great unifying scope, FEP has been criticized on account of not discussing the functional mechanisms of the brain. So the unifying scope is kind of like top down. That's the Ramsted et al 2018, all of space and time under the FEP envelope. That's the top down description. And then the functional mechanisms are like those minute particulars. That's the bottom up components that build the system. And that sort of meets in the middle with the real world systems that people are always discussing. So we're talking about how modularity works with the FEP or not. Any thoughts, Blue or Dean? Well, only thing I would add is that you can be strictly top down or you can actually, as Benny would point out, keep an open mind, which means that you've got some movement for both top down and bottom up. So we'll probably get into that more in the point one than point two. Yes, exactly. We're now going to go to some classic citations. So Benny writes, to address this problem of modularity and FEP, FEP theorists may want to provide finer details about the functional mechanisms that enable FEP. So that's how we're gonna make a comeback to Colombo et al 2021. Colombo and Wright 2016 is by providing finer detail. In addition to reinvigorating its unificatory scope, unifying scope maybe, that would be in line with first and preceding remarks that despite its roughness, FEP may be developed to provide a guide to brain functions. So we're going to do both. We're gonna keep the big scope and we're gonna provide details. So let's look at some of those details. This is also going to provide a really good introduction to the difference between the functional and effective connectivity. And connectivity here is referring to statistical measurements coming from brain regions because anatomical connectivity is just are the pieces physically connected? You know, the knee bones connected to the leg bone, that kind of descriptive anatomical connectedness. First in 1994, functional and effective connectivity in neuroimaging, a synthesis. So back when we were younger, weren't we? The paper reviews the basic distinction between functional and effective connectivity in neuroimaging. Going over to the origins and definitions. Functional connectivity is defined as the temporal correlations between spatially remote neurophysiological events. So that's not the only way to say it in English or any other language, but we're not talking about a philosophical sense of function, like what does it mean? Or what is its teleology? We're talking about statistical variation in measurements. This definition is operational. Very cool. And provides a simple characterization of functional interactions. The alternative is to explicitly refer to effective connectivity, i.e. the influence that one neuronal system exerts over another. So temporal correlation is things that are, like they could be uncorrelated in the micro, but then they're both going up together. That would have functional connectivity, whereas effective connectivity would be looking at how changes in one at the next time step influence changes in the other. And so those could be like going in different directions at the macro, but they could have effective connectivity because they influence each other statistically. Let's stay with this Friston 94 theme, but we're not gonna go through the whole thing. This is just to show like what was being discussed and explored and characterized about 30 years ago. So keep that in mind. We have a long pheromone trail to catch up with Carl on. This is a- They had a model back in 1994. This is 94. Yeah, they had models back then. Well, they were made of wood, but yes. This is a statistical parametric map, hashtag SPM projected onto a brain. And on this map of the images, take it their time of a brain that were fMRI in this case. One can take that data object and find the eigenvalues. And there are mathematical relationships between functional and effective connectivity. Not gonna go into it, but it's an awesome topic explored here and in several earlier papers and in the 2007 SPM textbook and in later papers. What was able to be said even way back then? Well, this is one of the figures. These are looking at the loadings on different vectors. So it's a spatial mode that reflects an eigenvector. So it's kind of like a vector through space and time activity that explains a large proportion of variance in the data. And the first vector, that's this top one, is clearly related to the difference between word generation, even numbered conditions and word shadowing, odd number scans. So this was a experimental paradigm where the person was being asked to do two different things alternating. Generate a word and then shadow a word, like I guess repeat a word that was already said. And then this vector, which had this activation pattern on the right in the brain was negatively correlated with the odds and positively correlated with the even. So it's a plausible mechanistic statement about the differences in the activated neuroanatomy. Doesn't mean it's simple to interpret, but it's a vector whose expression is correlated with something in the experimental design. Like if you did a double-blind study and these people got the drug and these people got a different drug, the correlation between them would be ascribed to the difference that you as the experimenter imposed. Whereas here's an example of another vector that declines throughout the entire experiments. And so that might reflect, for example, attention dropping off through time. And so that's a lot of neuroimaging then and now looking into these summarized statistical representations of activity patterns through time. And also for later, just it's cool to look at some of these citations like the Edelman citations, Tonini and Sporns 92. These are really classic citations. So there we were in 94. And then this is a dream of the future from 94. So here's Zargami and Friston in 2020. Again, talking about effective connectivity, biologically grounded models like dynamic causal modeling and dynamical systems theory. So improving and iterating on those basic ideas. And then also what does it look like to apply it and what kinds of conclusions and statements do scientists make when just applying these methods? So for example, this paper was in 2019 and it is called dynamic causal modeling of effective connectivity, should sound familiar, during anger experience in healthy young men. Seven Tesla MRI imaging study. And so you can read the abstract but they find that viewing certain kinds of films leads to activation of certain brain regions. Dot, dot, dot, dot, dot. We propose a model of effective connectivity associated with the anger experience based upon dynamic causal modeling. The findings have implications for psychiatric disorders. So that's what people do. They put people into scanners. They take measurements of the hemodynamics of the brain and then make a statistical model of it and the edges that exist across all subjects or only in the ones who have this diagnosis or only in these experimental conditions, those edges are interpreted in a certain way. That's what's on the table when we're discussing all these techniques. Any comments on Friston 94 or we'll go to the other classic? Just that effective connectivity is what I was thinking of like in the gene network, like how are they effectively connected? So it's like makes sense now. Yes, another classic but a more recent one is 2009. Friston's paper, again, single author paper. The free energy principle, a rough guide to the brain. So now that's how long ago, 13 years ago. The free energy principle might provide a comprehensive account of how we represent the world and come to sample it adaptively. Through action by moving our little eyes around. The FEP provides a mathematical specification of what the brain is doing. Now that starts to sound like realism, right? What the brain is doing, not a functional description of the models we make about the brain. It is suppressing free energy if it uses gradient descent. And so the box with the questions for future research are really still key. What optimization schemes does the brain use? These are like still important questions. And then also the domains of application. It's just always good to see like what was the FEP's description and scope 13 years ago. Some of the images we've even seen again and again with sometimes some variance but these are like images that we see commonly. We talk about action to minimize surprise and perception to optimize the bound. So this is like the predictive processing element and this is like the control theory element. We talk about the topology of perception, cognition and action. So suffice to say that there are equations, figures and it's another great paper on the long road towards the FEP. Any comments on 2009 paper? All right, Blue would you like to describe modularity? Sure. So I just wanted to explicitly define it because we talked about it a little bit in the abstract and we're gonna talk about it a lot more but in the paper that the author defined modularity as information encapsulation which Dean mentioned earlier. And he says informational encapsulation holds that when engaged in information processing the subsystems do not have access to information processing in other compartments and information processing within each subsystem is constrained within the computational mechanisms of that subsystems, of that subsystem. And so the modules are then defined by information segregation. And it's interesting to compare modularity like as encapsulating information in a module versus defining individuality which both like Mike Levin in the computational boundary of a self paper that we did last summer and then David Crackauer in his 2020 theory about the information theory of individuality both of those authors talk about information sharing like information sharing like with cells to form a tissue for example in Mike Levin's work and then in the information theory of individuality paper like up and down across different levels so that you can define individuality at any given level but it talked about how information is shared with this bi-directional causation in that paper. And so I think it's interesting to compare this idea of informational encapsulation in a module with these definitions of biological individuals. Very cool. Like is modularity the same thing as individuality? How about continue describing maybe in this discussion on intransitivity? Well, so I had to like look this up. So I just went into the paper like I just wanted somebody to just define what does this intransitivity mean? And so the author says the general insight is based on the intransitivity of the passage of prediction errors between levels whose information can affect one another only in cases of uncertainty and noise. So I feel like that this is like intransitivity of information between the modules but it was just not clear enough for me. So then I tried to look up like what is intransitivity? What is it's intransitivity theory? And I came across this really cool paper which is 2015 in entropy, intransitivity in theory and the real world. And I think it's really different than like what we're talking about here but it talked about like intransitivity in terms of preferences. And it said the choice between transitive which are transitive which are absolute and intransitive which are relativistic models depends on the nature of the processes that these models are expected to reproduce. Many people however have a psychological difficulties in accepting a relativistic approach expecting an absolute scale of judgment from bad to good which can be suitable in some cases but excessively simplistic in others. And they gave the idea of intransitivity in game theory and so like an intransitive game is just rock, paper, scissors which obviously like your choice there's no it's not rock is always good, paper is always good, tails never fails, it's always relative to what the other people are choosing right in this game. And then they really go into like multi-dimensionality and then transitivity. So he says in the paper, coarsening which is like coarse-graining in multi-dimensional cases become strictly intransitive and the cases without strict intransitivity are degenerate, either dimensions are redundant or coarsening is performed after merging the fractional variables into the overall utility instead of independent coarsening for all or some of the criteria. From a philosophical perspective this statement can be presented as a continuum argument for intransitivity. Small alterations are commonly overlooked for secondary parameters that can be accumulated into critical differences. And so I just thought that this was cool because it talks about like philosophy and some things that we're talking about and relative or like, you know, realist versus instrumentalist but it's completely unrelated. And I just wonder like, is it completely unrelated or can we go back and then like link in transitivity full circle into what we're discussing here today? So it was just, I put this in here because I thought it was a cool paper and in my search for intransitivity, that's what I ran into. Awesome, putting a number on something often makes it seem like whatever you're putting the number on, they are transitive. You know, if everybody gets a score out of 100 then there's a natural order or if we're ranking policies according to their expected free energy then there might be some ordering of policies according to which ones are best minimizing free energy. And I think this idea of intransitivity within the graphical model, what kinds of information are being passed and what does it mean? And then just more broadly like you brought up with the game theory, it's very interesting. All right, let us continue almost finishing the first section and get to modularity, FEP and the target of the paper which is the Hippolito and Kirchhoff paper. So without intending to map the course of the evolution of the FEP, like we will blue, I will just remark that the assumption of modularity has been fruitful in the study of brain functions. And so keeping an open mind about that assumption could be fruitful for the FEP too. Some significant critiques such as Hippolito and Kirchhoff 2019 notwithstanding. If you want to read more, definitely check out this paper from late 2019 by Ines and Michael. And they are also, as well as any of you, welcome to come on and discuss this paper. That could be really cool to like kind of embody and enact the paper and are we on the same page or what are our perspectives even if we are on the same page? And this paper is engaging in the discussion of the relationship between modules and modularity. The main goal of the paper is not to defy Hippolito and Kirchhoff's attack completely. The notion of modularity that Benny will defend is weaker than the notion that they attack. So what senses or types of modularity are we dealing with? What are the stronger and weaker claims? And we'll look at their definition on the next slide. The main point of the discussion is that following Hippolito and Kirchhoff, the modest definition of modularity is neither insignificant nor indispensable. So that made me ask, are we seeing a trend towards instrumentalism, bottom-up, weak claims like the entailment exploration of 34 and deflationary models and moves away from naive or realist strong claims? And so that just totally points to the role of philosophy in an ongoing scientific domain because things like strong and weak emergence or entailment relationships and these discussions on modularity, they really matter for how science is done and how it's interpreted. So it's just really cool to be seeing that happen in a new way. Again, we're not going to go into the whole argument of Hippolito and Kirchhoff, check out the paper, but they are going to consider the three most well articulated which is funny because that means well said but also well-jointed arguments for the view that modularity and predictive processing work well together. So predictive processing here is also referring to kind of that component of active inference. And they're gonna argue that all three of these arguments for modularity come up short, albeit for different reasons. So they kind of do like a steel person where they give the strongest argument for that tack and then they refute it. And the three arguments they discuss are the epistemic Bayesian courtroom argument, which is just the idea that courtrooms are modular and cognitions kind of like a courtroom like the eye is like a witness and then there's like another witness thereof. And so if that's modular, then shouldn't predictive cognitive architecture be modular? B, the intransitivity argument, which is that the causal influences across hierarchical levels are instantiating mechanisms that implement causal Bayesian networks. And they basically say that is pitching predictive processing in terms of the DAG as opposed to cyclic models. And C, the Markov-Blankett argument. The notion that a Markov-Blankett grounds the idea that predictive processing exhibits modularity. And so they are going to explore those and compute them as they say. And one of the key points is when cognitive neuroscience works with an acyclic Markov decision scheme, so that's like the partially observable Markov decision process that we were talking about earlier. But it may very well be methodologically misguided. Why? Because there's an increasing tendency within neuroscience to emphasize recurrent and reciprocal neuronal processing within the newly emerging dynamical causal modeling frameworks. So how are we going to think about the fact that the brain does have these feedback loops and square the circle with the DAGs and the DCMs and that's kind of the introduction. That's where we're set up, which is that Hippolito and Kirchhoff have framed and then attacked three pro arguments for modularity and cognitive architectures related to predictive processing. Benny is going to kind of take some of that energy and with a weaker version, or a little bit of a different version, not simply stronger or weaker because that would actually imply transitivity of strength. But with a slightly different tack is going to recall us to life through a discussion of modularity. So this section is the only gonna have one slide on it. And then either of you please feel free to add any comments. There's been a long history of debate on modularity in neuroscience. And Fodor, who's a very influential cognitive scientist, defined modular cognitive systems as systems that are domain specific, innately specified, hardwired, autonomous, and not assembled. That's not exactly the definition of modular that we've explored in some other slides, but the words are what they are. And that is the Benny quote about that functional isolation between modules is what defines them. So it's kind of like when there's more connection within than between, then that's modular. And so Benny writes, the present paper does not intend to defend Fodor's version of the modularity thesis fully, probably because some of these pieces about like innate specification, hardwiring, autonomous, unassembled, take it or leave it, it was the 80s. But Benny is simply aiming to show that some critiques of modularity do not need to undermine the possibility of informational encapsulation. And so, although a falsification of informational encapsulation leads to the renunciation of modularity, a defense of partial encapsulation would not establish a full-blooded form of modularity. So if you refute encapsulation, you have to renounce modularity. However, partial or transient encapsulation doesn't give you strong modularity claims. However, it can be compatible with the instrumental usage of models that have modularity. So those are some of the main arguments that set Benny up to defend a version of modularity that's quite modest. And it's almost like after that kind of a modest deflationary take, the question is really, is there any value in that definition of modularity at all? So the debate isn't over whether we can discuss things modularly. If we're going to take this different tack, then really the question is like, does it matter? Any comments on section two? Some, I don't know. I don't want to get too deep into it. Maybe you better to hold off for the dot one, but I just wonder about like if encapsulation, so if something's fully informationally encapsulated, like, my heart is fully informationally encapsulated from my brain, like how does a living system function, right? So this partial encapsulation, if that's not strong enough for modularity, I don't know how full encapsulation is possible. I just, I don't see it, so. Yes. Section three is entitled A Golden Thread, re-energy principle. And so we just had to go there with a blank reference because on his gravestone and one of his most famous lines is, I give you the end of a golden string, only wind it into a ball. It will lead you in at Heaven's Gate, built in Jerusalem's wall. And then here's a drawing that he made with a mythological character walking with a golden string. So the general idea, according to Benny, behind predictive coding is that the brain uses approximate Bayesian inference to decrease discrepancy between predictions and inputs. That is one of the key contentions of the FEP. And the organism's interaction with its environment can be represented by Markov blankets. Here I am using representation and modeling when speaking of Markov blankets and consider them to be Markovian models. So check out Benny's gas stream appearance. I think it's number one on Markovian monism. And this sentence makes it clear, we're talking about models. We're in the realm of instrumentalism. We're talking about Markovian models, not about whether we're making an ontologically real claim about Markov blankets like being the structure of reality. So in these models, Markov blankets are describing the conditional independence of internal and external states. And then it's that Friston innovation beyond Pearl and Markovs to describe the incoming dependencies as sensory and the outgoing dependencies as active states. So what is the golden thread? The FEP is an approach for embroidering or what, creating Markov blankets, finding them? What is the golden thread? What is the silver thread and golden needle? So what do you think the illusion means and what does the FEP have to do with it? Well, I'll speak a little bit on this. I might get it wrong, but that's okay, we're in still point zero. I think the predictive coding as an entailment question fits inside of the, what did you call it? BCC brain, brain, brain, brain, brain, brain. I can't find it now in here. Anyway, I think that, I think there's a subtle and a nuanced difference between what FEP is as a process and as a principle and what FEP architecture is. And I think that is what he is trying to bring to the table. There is a difference between the architecture or the product that results from the use of an FEP process and the FEP process itself. And once I figure out my own notes here, I can speak to what that BCC is, but I'll shut up for a second. Very well, interesting to describe the difference between the principle and the realized architecture. Section four, the track of a storm. So here's another single slider. So there's some apparent tension between the modularity thesis and the predictive architecture. So that's what Kerchoff and Hippolito described. While the thesis of modularity points in the direction of segregation and encapsulation, the predictive architecture grounds the integration of information in a hierarchical top-down architecture that unifies the mechanisms of cognition, perception and action under the FEP. So it's like what Blue just said, wait, if it's encapsulated and defined by its separation, then how is it part of this integrated whole? Well, Howie in 2013, the book The Predictive Mind and other people in other places have suggested that this conditional independence between the various levels of the hierarchical structure of predictive coding under FEP could be construed along the lines of functional segregation and informational encapsulation, which is why Fodor was brought up earlier. Dresen argued that the predictive architecture can be modeled in terms of this causal probabilistic dependency where different nodes could influence the adjacent levels, but there isn't global transitivity. So like the lowermost nodes are not influencing all, just the ones directly above them. However, the Hippolito and Kirchhoff paper challenge that vertical and horizontal account. So the vertical account is like the multi-scale nested systems, the horizontal account is like the collective behavior interactions across within a level. And so they aim to refute this account of functional informational modularity. According to Hippolito and Kirchhoff, and let's hear from them if it's not the case, the intransitivity argument for modularity is based on DAGs. However, they're going to argue that DAGs are not suitable for that kind of modeling because the brain has these cyclic mechanisms. And so DAGs have to be replaced by cyclic models, which don't have the limitations of DAGs. And then with respect to the Markov Blanket point, Benny asserts that Hippolito and Kirchhoff argued that the argument from Markov Blankets is of little avail to establishing modularity. He will get to both of those arguments. He kind of doesn't really worry about the courtroom as much, but the intransitivity and the Markovian Blanket arguments are what Benny's going to focus on. But first, remark on the importance of scientific models. So kind of a fun tack, but that's section four. Yes, please, Dean. Yeah, so I found out BCC is brain's cognitive capacities. That's how he described it in the paper. So I don't think it's a rock, paper, scissors issue either. I believe that predictive coding is entailed by the brain's cognitive capacities. I believe that the brain's cognitive capacities are entailed by the free energy principle. So it sounds like you've got a rock, paper, scissors, but that's not the same as the free energy principles architecture being entailed by predictive coding. I would have to diagram that out for you, but that's where he's saying it's not an intransitivity argument. And that's just notes to me. And that's where I kind of ran into his corner, even though I wanted to actually do a paper later this year, or a review of one of Hippolytus papers on an action that I think is fantastic in instrumentalism. But this was the moment where it was clear to me that it's not an intransitivity argument because back at what Axel pointed out to, we have to know what entails what. Very cool. Section five, a knock at the door, models and modeling. We really do need to do that joke stream, knock, knock jokes. Like, you know, who's there? I don't know, who are you expecting? Who do you prefer? In this section, Benny is going to describe some comments on modeling. He writes, I do not think we can find a reliable model independent handle on the architecture of the brain. So it's kind of like saying, if we're going to science, we're going to model. So let's not think that we could science without modeling. The general insight is that the interface of scientific theories and their target systems takes place through scientific modeling. So theories about the brain, the brain itself, and then the edge is the scientific modeling. Now the brain is like a special and fascinating system because it can be reflexively modeling itself. But if you think about theories of pendulums, theory of pendulum, and then the target system of the actual pendulum, it's pretty clear to see where the modeling relation comes into play. However, again, it gets kind of strange loopy with the brain. And Benny is drawing upon various works. Some of it is the Weisberg work on like simulation and similarity using models to understand the world. And also some of Benny's earlier work, for example, the ways in which Markov blankets are construed as scientific models. And similar points have been discussed by Mel Andrews, 2021. That was Actin 14. So scientists are making and applying models. And so we're not gonna break out of that VIP room in Plato's cave. So we have different kinds of models and maybe some of them are preferable. But like, if your argument is predicated on breaking free of a given scientific model, it is something to hear if you think you have it. Dean? So this is, it got interesting for me here too because if you look, for example, at a slice of time in an fMRI image, you could be focused on the concentration of heat in different areas of the brain. That would be one bit of information that you could take away. Or you could look at the proportion of blood in different areas of the brain, which is gonna tell you a little bit of different information than the actual concentration of heat, that heat bump per se. And so I was kind of curious who is a modeler here because he talks about, we can't take the observer out of the observation. So again, what relative bit of information do we tend to hone in on or do we think is the more priority information to take away from one of these slices of time representations? That was really important to me right here because if you think of encapsulation as that sort of frenetic image, a few slides back where all the brain is chopped up like here's the bacon and here's the ham and right, the different parts of the pig, you're choosing what it is that you think is the way of encapsulating, but I don't think that's what he's talking about. Well, explore it, but yes, taking this kind of a really strong principle stance on scientific modeling means that we can always ask who is doing the modeling, why? And so actually it's not just this sterile philosophical point, it really brings us to consider the social and the operational aspects of science. So it's a very important like sociotechnical point. Section six, the game made, the intransitivity argument. Okay, we're not gonna go through all the details here. We might have time in the dot one and two, but Benny is going to focus on Hippolito and Kirchoff's specific reply to the intransitivity argument that is centering the distinction between the directed acyclic graphs and directed cyclic graphs. Benny is going to argue that these two kinds of statistical models do different things. It turns out they actually also map on to the difference between functional and effective connectivity. And so applying one sort of the model DAG or the cyclic model in a specific situation is a matter of practical exigency and methodological consideration as well as the goals and the interest of the modeler. So it's like we're a hundred miles down choosing a model freeway and then whether you take this slight difference between one topology of graphical model that's quantitative versus a different topology, maybe that is not going to be super philosophically impactful because we already have taken like the big pill, which was making a scientific model or making a graphical quantitative model. There's a citation to the Friston Visa Hobson 2020 paper that also describes like where the cyclic versus acyclic are preferable and Benny recaps. And just basically to summarize, there is no universal reason for extolling DAGs over DCMs because the focus on functional connectivity or effective connectivity is a matter of explanatory predictive goals and interests. So scientists are using different kinds of models. And so maybe we should pull back to scientists use models and think about the implications of that rather than try to finesse out some philosophical implications from the very specific kind of last mile modeling choices. DCMs can do certain things, whereas DAGs can do other things. Which one of these approaches can be used to analyze data in the context of the FEP? We're going to discuss it in 35.1. Benny says, I do not think Hippolito and Kirchoff 2019 are bound to concede a non-realist reading of DCM. Rather, the critique of how H and K confute the intransitivity argument is pointed at their unjustified partiality towards DCMs. So it's like, if you could just scapegoat the DAG, then the DCM is the savior. But if both of them are just types of scientific models that are chosen situationally, operationally, as Friston described, then there is no savior model coming in to save the day. DCMs are formal devices. And then there's a citation to the emperor's new Markov blankets with Yella at all, live stream number 20. And so drawing attention to the difference between DAGs and DCMs, which are complimentary modeling tools, as we discussed earlier, they're defined as complimentary, could not confute the intransitivity argument. So if we have already committed to the instrumental perspective, so we're talking about our model, not about the world, then the difference between different modeling approaches is purely situational or operational. And so all scientific models have roughly similar philosophical grounding or they're part of a broad category with respect to this intransitivity argument. Nice philosophy and interesting points. Okay, call them in the storm. The indispensability of modularity and the indispensability of scientific models. Oh, I wonder if there's gonna be like a parallelism between models and modularity. And here's like a person in their Markov bubble in the storm. This section focuses on a realist interpretation of DCM in terms of dynamical causal models. So even instrumentalists can bat both ways. You know, we're not obliged to never talk about stances that in one certain situation, we didn't prefer. It's just really wide ranging intellectual exploration that's being characterized here. And here, Benny does play on a little bit of a acronym play, which is that DCM applies to dynamic causal modeling as well as directed cyclic modeling. So cyclic graphs or the dynamic causal graph. And he'll argue through an example that we're gonna discuss not today with Alison Bob. He argues that the modularity in the context of DCM, it's not as strong as the Fodorian isolationist concept, but in interesting and important ways, it's going to be enough. And so a moderate form of modularity is retained. And Hippolito and Kirchhoff argue that this weakened notion of modularity is explanatorily vacuous, it's empty, and there would be no reason for keeping it. And that's the bone of contention. He's going to dissent and say that the benefit of this modest notion of modularity is remarkable. So that's the claim. Any comments on section seven? What would happen if we couldn't categorize? Then it would be impossible to give a categorical response to that question. And a lot more, right? Like that's basically what this comes down to. Doesn't mean that the things aren't arbitrary and not necessarily put into the correct silos, but there is a great benefit. I don't think it's dispensable our capacity to categorize. Doesn't mean we always are obliged to do it. But when we do do it, is there a benefit? Yes, is there a cost? Maybe. But if we took it away, what would the world look like then? A triangle is a triangle is a triangle, right? There would be no specificity then. Interesting. We'll talk more about categorization models and the parable of Alice and Bob. Section eight, still knitting, Markov blankets. So here they return to this idea of the vertical insulation and the horizontal insulation. Like there's encapsulation, laterally, collective behavior, and then encapsulation like an onion, like a multi-scale system. So that's very interesting. And this is also where we get to this point about the scale-free or scale-friendly nature of Markov blankets. The main reason that H and K resist this argument about the Markov blankets is based on the fact that the regimentation of states into Markov blankets is scale-free. So it can range across scales. Maybe even that is already scale-friendly. Because Markov blankets are scale-free, they do not imply insulation, and thus the modularity thesis does not imply insulation. And so, Benny can see it's, yeah, it's true. Markov blankets can be applied to basically a wide range of phenomena. However, this is overlooking a subtle point. Markov blankets in the abstraction, in principle, are scale-free. There's no inherent constraint on their size and scope. Linear models are inherently a priori scale-free. It could be centimeters versus the general scale-free. So Markov blankets can be applied to basically a wide range of phenomena. It could be centimeters versus the GDP. It could be astronomical units versus temperature. Like it doesn't have a tie-in to a certain spatial-temporal scale or any feature for a linear model or for a Markov blanket. However, Q Dean from stage left or running in from the audience, their application or imputation to various target systems will impose a definite scale on Markov blankets. To make a long story short, in the context of their application under FEP, not in the context of FEP, Markov blankets are not necessarily scale-free mathematical models because they're being used in models. So even though the formal tools could be scale-free, application mandates calibrating to a certain scale. If that's the case, and it seems like it is, then counter-arguments based upon the scale-free nature of Markov blankets could not undermine arguments from Markovian formalisms soundly. So this is really nicely framed and placed in space and time. Dean, what do you think about that? Well, I'm still, as I said, I didn't, first of all, I wanna make sure to disclaim this. I never saw this paper until three days ago, four days ago. So my guess, my WAG about scale-friendly and sort of whether that was something we should consider, it was nice to read this and have somebody else go, maybe it's something we should consider. I'm gonna leave it at that because I don't know if somebody's gonna come along and disprove that and, you know, because if that's, if it's falsifiable that calibrating doesn't matter and categorizing doesn't matter, I'm open to hearing that too. But in the meantime, I think it's something that if we don't recognize the difference between the abstraction and that thing that we're focusing on right now, we can get lost in the idea that a triangle is a triangle is a triangle and nothing else matters. Lou, anything you wanna add on this section? Yeah, I mean, I really think that, I mean, just hit the nail on the head. Like we're the ones putting the Markov blanket on something. And so like just because something can be, you know, modeled using a Markov blanket doesn't necessarily mean it doesn't scale. And like things don't Markov blanket themselves, right? Like, so I mean, even like my layer of skin, like cells are popping off and new cells are adding on all the time. And so like there's, it's not a hard boundary. It's not a solid boundary. It's very fluid, like what goes in and out of a Markov blanket. So I think like our imputation of a Markov blanket on a system, I mean, yeah, they're scale free. Means you can impute them on any system at any scale. So maybe scale friendly is the way to go, but it's not like the blanket itself is scale free. I mean, all blankets are composed of stitches, right? Are you using big stitches or little stitches or like different layers of cloth or what? So I don't know, it's, I don't know. I think it really drives at home. Yeah. And the paper was, I mean, we can double check, but it's from December 21. So we have proof of live stream Dean that, you know, you were there, great. Can I just say one last, because I completely agree with what Pluja said. I was never up an opinion that a Markov blanket is the scale free, of course it is. And it's kind of like saying, there are subject matter experts. Of course there are, but as we had in the conversation with Dr. Friestin, you could also say that there's a prediction matter expert. Like all I'm asking is for people to consider that there could be both and they could be running in parallel. I wasn't trying to push back in the sense that, you know, people are wrong. What I was saying is what else could we include to fill in the wholeness of what is happening here? That's all I was trying to point out. And again, we could take active inference and focus it completely on the minimizing of free energy and getting what we're attending to, or we can talk about the availability piece and what, for example, when we're having new people on board to our lab, how we make that entry point so relationship-based. We could do that. We could make, for them, we could make them prediction matter experts before they even realize it in themselves. Now, whether we do that or not, it's not up to me, but we could. So that's what I was trying to bring up and that's what I like about this paper. Last section, last on-tent slide. Isn't it something they say, like, if the newspaper article ends with a question mark, I forget which one. Is it the, it's always yes or it's always no, but the footsteps die out forever. So this is a summary in the ninth section. Benny challenges the main points of Hippolito and Kirchoff. Hippolito and Kirchoff confuted the modularity thesis. So Benny is salvaging the modularity thesis from the pincer attack of Hippolito and Kirchoff. To support the argumentation of the paper, Benny draws upon the indispensability of a model-based conception of modularity. That's where that kind of science and society angle and philosophy angle came into play. Benny argues that DAGs and DCMs are complimentary models and the choice between them is operational. It's pragmatic. It's not metaphysical, per se. So modeling is situational. We can take a deflationary approach even to how we think about science. Benny argues that when we think about dynamic causal modeling, there is modularity because of how the statistical model pulls out differential coupling between brain areas. And that's not a statement about the brains, a statement about the model. Benny also challenges Hippolito and Kirchoff's answer to the argument from the Markovian formalism by showing that although these models are in principle scale-free, in order to represent any specific target system, they have to be associated with interpretations that do give definite scales. So the Markov-Blankett formalism is conceptually scale-free, but any realization application is scale-specific, thus scale-friendly, tickles 21 at best. Personal communication. And basically- I'll be saving the livestream soon. Yeah. Benny closes with just saying it doesn't undermine the claims. And this isn't like a radical pro-modular thesis. It's not the isolation of brain regions and they're each like virtual machines that are totally separated. The only point that comes to light by our enterprise in this paper is that the debate over modularity needs to take the role of scientific models into account and a model relative conception of modularity cannot be waived away easily. So it is a great service and an interesting paper in form and function. We have an empty slide for the implications and questions. We have the early slide where with what we're gonna discuss in 35.1 and 35.2 and it was an awesome discussion. So if you have any final comments, now would be a good time. Do you think that when you get people who have worked together as this author and the two people that he was kind of pushing back against you, do you think when they take it into this kind of a formal setting of paper writing that that changes their relationship fundamentally? That's one, I mean, I suppose we could only ask the authors themselves, but does it have to be a debate? Necessarily, does it have to be drama necessarily or is there kind of a moving forward hand in hand part to this I'm not seeing? It's a really interesting question. I'm sure scientists and all kinds of different people are gonna have different answers, but I think about the papers in their exact phrasing as like pheromones, digital stichmergy in the literature corpus, and then the person is like the nestmate who is weaving on. So any artifact is gonna be made by that person in the past. So I could be like talking to Blue and it's like, oh yeah, you know, Knight 2018 said this or you just said this, but didn't Knight 2009 say this? It's not a contradiction. It's just like the trajectory and the pheromone deposition of active and complex entities. And so I think that that I'd hope is the best of both worlds, which is we can humanize Kirchoff and Hippolito and Benny and all the humans involved and take like a compassionate and even friendly tech, while also being very scrutinizing of the specifics of the knowledge artifacts that people leave because a lot of times there is like a gray zone where it'll say something in text, but then, oh, if you're in the club, then you know that they actually meant something else or you gotta read between the lines with the methods or you have to read between the lines with the citations and that's kind of a conflation of the person and the arguments, like when people give an overly charitable reading or even try to insinuate things that weren't there in the initial text because they want to uphold their legend of person and rather we can just kind of separate those two in how we model the generative process and the generated product. I just think that there's so much you can learn when you're wrong and that doesn't mean you wanna be a serial mistake maker but I don't think anybody that's publishing is looking to do that either, but if you're comfortable, if you're actually comfortable with putting wrong answers out there and having put people kind of proved that you're wrong and put you're comfortable with that, I think that makes a huge difference at least in terms of the culture that you're participating in. It's a lot safer if you can throw stuff out in four weeks or five weeks time, somebody can say, no, Dean, you were wrong. We gotta take that citation down. I wouldn't take it down. I would leave it up. I would simply say, no, in that moment, I thought I was right. And now I've updated my generative model. Yeah, it's like showing your work. Like, here's what I knew at this time and this is what leads me to my conclusions. And then someone says, well, you missed this paper from 1855, okay, then that updates the next round of pheromone deposition. Yeah, exactly. Blue, any last comments? Good. Yeah, it's a fun start to 22. And it's the two architectures, 2022, second year of Actinflab, 22222222 is coming up. I mean, it just couldn't get any better. That's great. Thank you, folks. Yeah, so see you and everyone.