 Hello and welcome. I think that we're going now. This is the Active Inference Livestream. It's Active Inference Livestream 18 on March 17th, 2021. Welcome to the Active Inference Lab, everyone. We are a participatory online lab that is communicating, learning and practicing applied Active Inference. You can find us at any of these links here. This is a recorded and an archived live stream. So please provide us with feedback so that we can improve on our work. All backgrounds and perspectives are welcome here. And we'll be using good video etiquette for live streams. And Stephen, thanks so much for joining. Always awesome to have you on for the conversations. And this will be a very fun convo. Here, looking at the calendar, we can see at the shortened link that we're down in the end of March 2021 number 18, which is going to be about the paper, the Predictive Global Neural Workspace, a formal Active Inference Model of Visual Consciousness by Christopher White and Ryan Smith. And this video in 18.0 is going to be about setting the context for 18.1 and .2, whether participating in these discussions with the authors, if you're listening to this before those events happen, or contextualizing in sort of a bi-directional way for listening to those later discussions. So if you're in the Active Inference Community, this might be exposing you to some new questions and perspectives. If you're from outside the Active Inference Community or you're not familiar with it, then this will be highlighting how Active Inference is an integrative framework across a few different other ideas. These .0 videos are just introductions to the ideas because you could take a whole course of learning on any one of them. They're just little suggestions and hints related to the ideas. And where this paper is going as far as the punchline is that we can build on this predictive Global Neural Workspace Model and think about it in being implemented by Active Inference type models and get new insights, predictions, and hopefully explanations and applications. And in 18, we're going to pretty much go through a lot of the paper as far as their goals and the questions and many of their copious figures. And it'll be good to talk to Ryan and Christopher when they join for the stream. For 18.1 and .2, we're going to be discussing this paper, so save and submit your questions and get in touch with us if you want to participate. Okay, so here is the paper, Predictive Global Neural Workspace, A Formal Active Inference Model of Visual Consciousness. Before we read the goals and claims, Steven, maybe you could just say hello and introduce yourself and why it got you excited in this paper just personally before we dive into the paper itself. Thanks, Daniel. Yeah, well, my background is that I'm doing a practice-based PhD at the Solomon's Institute of Applied Psychology, so I'm interested in how active influence can help understand some of the processes that I've been working on for understanding pre-reflective phenomenological awareness and people's sense-making. So that kind of ties in to how I'm also interested in Ryan and Christopher's work because they're bringing this into the psychological domain but also bringing that into the phenomenological domain in ways which are practically predictive. So that's really, really exciting and I see a lot of opportunities for this modelling work and I've actually spoken to Ryan about this to try and link in with other ways of people doing sense-making, etc. And this particular paper I think is really interesting because we've got a lot of exciting philosophical ideas around global workspace concepts in neuroscience and an area that I've only partly explored, but to have this integrated into a predictive model that can be tested at certain scales in certain ways and to have people who've got the sort of rigor and the kind of the ability to focus down and choose experimental and practical ways to explore this which Ryan and Christopher have is really helpful. So I'm looking at this from a few different ways and hopefully that will be something that other people will find useful. Nice. So this is kind of a big paper and topic. So we're going to frame it first with just how the authors represented their goals and claims and then we're going to kind of pull back to just the bigger questions about the paper and then we're going to go through basically the keywords. So when reading what the authors are setting out to do it's like why would you even want to travel from A to Z? We're going to go to define them. Stephen, do you want to read the first one on six? Oh wait, you're muted. So goals and claims of the paper here, we've got formalizing ideas first introduced by Howie and White and Friston. So they argue that conscious access or ignition, which is this kind of sudden awareness that things come into our awareness, is a fundamentally inferential process that depends upon a level of processing of sufficient temporal depth to contextualize and coordinate lower levels of processing. So that's actually quite, has a few implications to that. So shall I carry on, is that okay? Sure. Then also to build on the previous conceptual contributions in this area mentioned above, they will be substantiating arguments with a series of detailed computational simulations. These simulations are based upon the first principles account of perception and action selection offered by active inference. So that's again, that's one of the real foundational benefits that this is testable. And then also they end by examining the relationships between the predictive global neuronal workspace which they've introduced as a concept and alternative models and briefly address potential concerns about how phenomenal consciousness could plausibly be situated within the model that they're going to present. And however, this paper is chiefly concerned with what block terms access consciousness, which is defined as the availability of information for verbal report, voluntary action and executive processing. For brevity, they will use conscious access, they use conscious access and consciousness interchangeably throughout the paper. So just to really, you know, but this gives them a practical starting point to work on. Yeah. Nice. Yep. There's a lot in these goals, but one of the key pieces is that they're talking about informational availability in terms of report paradigms and then they're going to be connecting big questions in multi-scale systems and cognition to active inference. So let's unpack that a little bit in terms of just what are the bigger questions, not in the author's words, that we identified this paper as relevant for. So people who are maybe working in these areas and the first one, which one of these do you want to just go for first? I might just mention the first one and then if you could pick it up after that one. Yep. So basically there is a question really that they're exploring. Can simulations using generative models of belief updating give you principled schemes in active inference to extend to other proofs for a predictive global neuronal workspace? So does this lead to sort of a plausible route doing this sort of simulation to a predictive approach to this whole paradigm? Nice. And it's related to the second question, which is what are the unique predictions and explanations of active inference? So the first question is kind of asking, how can we use the simulation, the generative model? How can that be used to get to unique evidence or unique claims or unique experimental design? And then something that's related to active inferences role is how could this reposition the distinctions that people make between access consciousness and phenomenological consciousness? So access consciousness, as per the previous slide, is the informational accessibility. Phenomenological consciousness is the awareness or the qualia. And an example of something that's in access consciousness but not phenomenological would be like a subliminal sound because you're not experiencing it. It's subliminal, but it can influence somebody. So that is within their access consciousness because informationally, it leads to differential behavior or the planting of an idea, but it's not phenomenologically experienced. So it's not just one basket with what is being met with consciousness. Yes, Stephen? Yeah, and to clarify a little bit on that as well, is that they are using this idea for this access consciousness whereby you can access it in a way that you could report on it, like you're saying, and that may be there, like I said, as a qualia, and it may be transiently available, but you may not be able to access it from a reporting paradigm. So the way they frame access consciousness is to really give a temporal depth and a sense of how stuff comes in. So the idea of access consciousness, it's either accessible or it's not, whereas phenomenological consciousness is kind of more of that flux, which is when we think of consciousness, it's hard. It's not really something you can put in a box, whereas access consciousness, you can't put it in a box either, but they're more able to, so they can model it. It's useful. Yes. They also are setting out to address how the active inference model can integrate different ideas like Bayesian brain and global neuronal workspace in neuroscience and combine that with more psychological applications. And also they're asking in this paper, how can they perhaps integrate with other existing active inference models to sort of extend other work, like related to the global neuronal workspace and also bringing it into a specific visual modality and experimental task. So we had a bunch of questions making dot zero as well. So what is just one or a few of the questions that you just wanted to raise and why did you raise it? You're muted. Some of the questions that came up for me was, you know, just how does this all fit together into what's potentially in the future? So I'd be really interested to get Ryan and Christopher sort of take on some of these distinctions, you know, just like you were talking about that potential for it to go into multisensory integration and beyond. So how does visual consciousness or awareness of that differ from general awareness? And how does the ability of the predictive global neuronal workspace give these detailed simulations and resulting predictions? So what's the potential for that in other fields and sectors? Okay, so one of them might be to help influence psychological perspectives on people with professional practice. As people are starting to make, even what it means to take a perspective, how much is that access consciousness effectively? When I start to bring something into my perspective I bring something to my access consciousness. When medical understanding of conscious awareness or when people start to understand about self-reporting in general and how people know and people who are working with knowledge and bring things up, because a lot of knowledge workers are using interviews and reporting paradigms. And there's a big question about, well, how do you deal with first person, normally it's first person perspectives, which is one thing. And then there's first person experience. And how do you even get those two resolved in some sort of coherent way? So I think that's something that would be really interesting to hear more on. Nice. Good thoughts. And we're just always curious about where active inference comes from and where we can apply it and learn it. Why does it really matter? Well, there's sort of a theory arm and an application's arm. The theory arm is that this paper is exploring some experimentally tractable unique predictions of an active inference type model and showing how it goes beyond Bayesian brain, global neuronal workspace, predictive processing. It integrates across and it goes beyond several of these theoretical neuroscience modalities. And then the paper is part of a broader network of research and work by Ryan and others that will hopefully hear more about just as to how could this shape the kinds of clinical and community psychology practices that people are performing? Because this is related to all these types of topics. So any thoughts on this before we go to the abstract roadmap? Yeah, I suppose it just comes back to this idea of ways of knowing what it means to know and be aware and have awareness. That actually is often sort of implicitly sort of in the background, but this gives a way to make some things more explicit, which I think could be really valuable. Cool. So the abstract just because I think we'll have enough to actually talk about with a paper, people can pause it to read the full thing. They're in the first part of the abstract framing the need to move beyond the global neuronal workspace model. In the second half of the abstract, they report what they did, which is demonstrating their integrative model, which we're going to be able to look at and explain what they actually did. And they find that that model is able to reproduce a lot of other findings from previously non-unified experiments. And then they close with a discussion on how they can build upon previous work in this integrative and active inference style and where that leads to as far as predictions. So that's sort of reflected in their roadmap in that they give a basic broad introduction, then specific theoretical backgrounds on active inference and deep temporal modeling. They then spend the bulk of the work presenting and many figures presenting the results and recapitulating several specific phenomena that are important to reproduce and also impressive to reproduce in a simple framework in one where we can see all the parts. And then they close with basically a discussion about where their work is fitting in relative to other work and concluding, summarizing and stating a few next steps. So how did that strike you? Or what would you add to that, Stephen, if anything? No, I think that's a good summary. I think in some ways we might be... The authors will speak a bit more to how that taxonomy came about and there's some nuances in there in terms of how and why they selected what they selected and which would be really interesting to have them really step through. But overall, the way they've modeled this out into a very practical, real-world kind of context I think shows that potential. And then what does that mean for this temporal structure? Then links back to other work around temporal depth, which is the kind of... Since I think it's 2016 has been what a lot of the active inference work has certainly in the modeling has really been yielding is this idea of temporal depth and what it can mean. So that'd be great. Nice, great point. Earlier models of active inference were sort of like instantaneous or one-step models. And a lot of the ongoing research is about the integration of multiple scales or multiple coarse-graining of temporal dynamics. So here are the keywords of the paper. So if someone was listening to it from any background and was curious about why does it matter to connect some of these things or why is it important or how does that implication arise from integrating what have previously people done, if not with this integration? That's where the keywords kind of come into play. And we're going to go through basically a slide for each. They are computational neuroscience, predictive processing, Bayesian brain hypothesis, active inference, consciousness, visual consciousness, global neuronal workspace, and the predictive global neuronal workspace. So let's start with computational neuroscience. A lot you could say about this, a lot of people's department and research, but we can just look at it pretty naively as the intersection of computational approaches and neuroscience research. So computational could mean using computers, which a lot of things are in nowadays. Using computers could be descriptively modeling statistics or it could be generatively modeling a system like with a simulation. But also the computational approaches might include using computational metaphors to describe systems. That might be considered computational as well, like computationalists. On the other hand, neuroscience is about studying brains, but increasingly non-neural cell types in the brain and the body and the niche and the society. So all these different aspects of behavior in the brain. And it uses all different kinds of approaches, some of which are computational, which brings us to the overlap, which can look like a few different things. It could look like computational brain metaphors. The brain is a computer, or it does information transfer of this type. It could look like doing descriptive or generative modeling on computational representations of neurological data. It could look like brain-computer interfaces. So that's sort of computational neuroscience. And Steve, I know that you kind of saw where that could go beyond where most people would think about computational neuro. Do you want to mention it here on the next slide? Maybe on the next slide and just... I suppose in some ways it's just to try and highlight some of the thought practices I had to go through to get my head around what this means and how I could bring it together with the active inference process. So the challenge you slightly have with active inferences and predictive processing that goes with this is that it challenges the computer metaphor that we've kind of got embedded of how the brain works. Either the model or the thought that we sort of bring in signals and then we have a computational awareness. We think it and then we do it. It's this sort of open system. So these predictive processes and this cybernetics. So it kind of... When we're talking about computational neuroscience, in relation to the systems neuroscience, the nonlinear dynamical systems that active inference is part of, it's a bit more of an open kind of process approach. So I suppose that for someone who's trying to understand this, I think I found it helpful to start to think about the computations of dynamic process that develops over time when it's used in this sort of context. And those sorts of ways that these models are, which is different to kind of system dynamic models, which are much more like flows on something which is defined, it's this kind of evolving process and of course the free energy principle then ties into that because it's all about being able to work with those dynamics. So I don't know, does that make sense? People can let us know if it makes sense or helps them. I think that the nonlinear dynamical systems is a big topic. One way that that has been operationalized in neuroscience is the Bayesian brain hypothesis because Bayesian statistics is often used to model nonlinear dynamical systems. So if we're thinking about the brain as a nonlinear dynamical system, it makes sense to think about the brain as a Bayesian device. And this is from the 2018 interview with Friston and Martin and myself. And in this paper, we summarized and characterized the Bayesian brain hypothesis, predictive coding and the free energy principle, and asked how are these different models similar or different from one another? In fact, he had suggested that these three frameworks are variations of the same basic mechanisms. So if you're wanting to learn more about the background of the Bayesian brain and how it differs in specific ways from predictive processing, free energy principle, active inference, et cetera, then read his response here or read the interview because it can help with some people's understanding. Let's just go through it at an overview level. Predictive processing is specifically looking at how error is propagated through systems which can be hierarchical or must be hierarchical and therefore are amenable to a Bayesian hierarchical modeling. So in this sort of diagram on the right, it shows how at the very bottom level, you have muscular output, action, and sensory input. And then what's happening in this cascade of hierarchically nested systems is that predictions are coming down and prediction errors are moving up, but it's not a spatial thing. It could be in or out or it's just about how it's laid out. And predictive processing turns out to be a way to get somewhat effective Bayesian-like behavior because it's even how we fit some of our algorithms. We just start with the prior and then we inch it closer depending on the error signal. So learning based upon the error signal is a really common algorithmic maneuver. And so predictive processing in the Bayesian brain are sort of like compatible theories because the Bayesian brain is just saying it's going to be something Bayesian. Predictive processing is a theory because it makes predictions that are falsifiable about what is being represented in the communications between levels and namely that you have prediction going one way and prediction error heading back the other way. Anything to say about that before we continue? No, that makes sense. The bad is that what's going to be mentioned is how there's this temporal depth piece starts as much as you could have like a level of processing goes to another level and that's something that the same basic principle is driving it. So yeah. Cool. So here is a really informative piece from the first in 2018 interview. First in wrote, the Bayesian brain hypothesis per se does not trouble itself to commit to a particular process theory other than requiring implicit beliefs to conform to Bayes' rule. So similar to how free energy principle doesn't have a specific process theory that's downstream, it's a more like a general overarching does it conform or not to this principle. The Bayesian brain hypothesis is a corollary of the free energy principle and is realized through processes like predictive coding or abductive inference under prior beliefs. However, Bayesian brain is not the free energy principle because both Bayesian brain hypothesis and predictive coding are incomplete theories of how we infer states of affairs. And then here is so interesting where he wrote, it's this inactive embodied, extended, embedded and encultured aspect that is lacking from the Bayesian brain and predictive coding theories precisely because they do not consider entropy reduction through action as well as through inference. So it's all well and good to have the inference engine that's driven in this thermodynamic way towards entropy reduction but it takes on a whole another level and integrates with control theory and action in the niche when you go from saying, well, yes, the brain is a Bayesian statistical inference device for action. And that's really one of the key pieces of active inference, which is why we went to it here. So before we, this is the definition of the authors in this paper. That's what they wrote. So Stephen, where would you say active inference fits in and what do you have to say about that? Well, I like this. This is a really helpful slide. Well, there's a couple of things that come up. One thing is the fact that it talks about abductive inference and it's talking about this idea, you know, predictive coding and Ryan Smith and Christopher, they're talking about prospective, sometimes ideas of, you know, we're predicting how things are going to happen. Well, that's really interesting because most research in terms of like social sciences, mostly, you know, retrospective or inductive or deductive and abductive is this abductive emergent thing is actually very hard to get at, but actually is showing that this predictive rather than retroactive knowing is what's driving everything. So I think that in itself is quite useful to think about. Yep, nice. It's interesting that he mentioned it there. And that active inference, how does it implement potentially some of these cool aspects that are lacking from the Bayesian brain? So where does active inference build on Bayesian brain approaches? Here's what active inference is. So the way that the authors Ryan Smith and Christopher White wrote it in this paper on the first sentence of their primer is active inference a corollary of the free energy principle is the first principle's approach to modeling approximately Bayes optimal behavior. So you can look at their primer in this paper which is really awesome as well as their very extensive tutorial on active inference and model stream number one point one through four. So we just really appreciate the clarity that Ryan and Christopher bring to this area. But basically what's happening is active inference is allowing us to formalize the process of the environment giving data through sense to the organism which in the generative modeling of that data which is like Bayesian modeling but that's inference and then it connects it to external states again closing the loop via action as well as internal dynamics or endogenous fluctuations on either side of the interface only by defining the interface do we get this kind of a clean separation so that's where active inference allows us to say yes it's Bayesian in terms of how it recognizes data and it acts in an ecosystem that also has other agents that's where active inference comes into play all right so now to sort of switch gears to consciousness and visual consciousness so let's just start with sort of the obvious what is consciousness question and just we're just talking about this paper so we're not talking about like something that's you know some sort of theological question remember that it's access consciousness it's self report paradigms and you can read more about their perspective on that in the limitations and multiple other places where they really are clear about the pros and cons of using visual modality so it's not olfactory consciousness it's not a sound it's not multimodal they're doing a sense specific task and it's reported awareness so what do you think about that Steven or in your thinking or learning about consciousness where would this sort of a narrow approach figure in yeah well I think that you know when they talk about the pros and cons you know the pros are that we are actually able to generate data through reports and it gives us a way into this whole area and by the sort of the detail of how the design has been done it negates some of the challenges and makes us get some workable insights and one of the cons is or one of the downsides that they're facing is you can't genuinely report being conscious of nothing you can't say I'm no longer conscious of any of that thing so to speak and they acknowledge that there's always something in your field of vision when you're being asked to self report around if something's present or not present so so they have to do some work around to get the results that are usable but that would be something they can speak to better than me cool I agree it will be like what are the authors big motivations what is their long term vision for consciousness research why have they chosen vision first what would be the next most tractable sense so definitely there's a lot of questions to have for them and also this is a great note from them brief note on phenomenology and the phenomenal consciousness access consciousness distinction so again I just we kind of hit this point because it's one of the key words of the paper is consciousness and a lot of times when we see scientific research on consciousness so-called it's just it's always critical to be clear about what's being discussed and the authors thankfully here really are clear which actually leads some people to be almost disappointed it's like oh what about the big phenomenology it's like maybe we'll get there maybe we won't or you could have any story you want on that but it's almost like if the sales pitch is too much hype then people it's what they expect so then when there's a more narrow focus paper they act like it introduces all these critical challenges that are just being glided over by a paper that doesn't define consciousness or doesn't go so specifically and carefully into the distinctions between access and phenomenal consciousness because not everyone will and yeah yeah I like that the way they talk about this they acknowledge this question about this threshold and start to sort of decide on where they're going to fit in with that I think it's really valuable because this idea of the question that's sitting in the background with the phenomenological consciousness there and when they talk about this moment to moment sensory flux and in a way it ties to the Chris Fields work that we were doing with quantum contextuality potentially is you know what's this kind of flux idea that's in the background and when does it get shipped over to the contextuality of awareness so just to show it linking to some of the other papers that have come up and then for this you know it shows you the importance of the experimental design and the modeling simulation design and how the way the design ends up evolving itself kind of gives us some insights of how you might structure our understanding of consciousness the sort of questions they had to ask an answer to build the model in itself is quite interesting nice yep so they're really clear about that and we can talk more about it let's go to the neuro side so we kind of talked a little bit about the philosophical question of consciousness and the systems dynamics the troll theory question of active inference but then there's this neuroscience side and area that they're in and that's what's also moving towards applications in the clinic for example and what we can do here is present some of the early and later citations related to the global neuronal workspace global workspace theory GWT there's a bunch of related ones sorry if the acronyms are not always consistent but what the global neuronal workspace I guess is probably what we should use and predictive GNW are their integrative models that are on one hand for neuronal function so through philosophically neutral frameworks like cascade theory or network theory you can describe how you get patterns like ignition or cascades through networks or propagation then it can be applied and say ah what if that ignition is so called consciousness access consciousness there's a strong argument you can say look if the data doesn't get to the other end of the pipeline it's not going to get there so if it does get there it had to have propagated through a critical system so that is the easier bar and then the harder bars is that information propagation equivalent to phenomenal consciousness experience qualia and the answer as we know from subliminal stimuli is just no it's not but that's what people often want to get at addressing which is how does the brain quote make consciousness or what brain states are associated with quote consciousness meaning awareness in our first person experience which even under the most generous vision is just a part of our total system what do you think about that Steven yeah I think that's a good you make a good point there and this little from the sort of more broader workspace to this neuronal workspace then brings in the actual understanding which again Ryan and Christopher have of neuroscience and the way that the firing happens so that gives an insight and also the thing I think is interesting is that they're talking about consciousness as being this belief updating or prediction that's fairly on off like it's like things come in they come out of consciousness whereas in other cases that might be more of a gradual flux of awareness there's something going on here which because that flips in and out of consciousness that gives you a way to use belief updating schemes to model it and I'm interested to hear how they see that yeah nice good question so here's kind of a classic neuroscience diagram from 2005 early paper left side is the single neuron scale you have receptors in a membrane and different currents that are happening at a very very micro level me so level with circuits and then more of a macro level with the activation that's propagating cross regions of the brain and then another way that was helpful was looking at this I think it could have even been 1998 paper the global neuronal workspace model of conscious access from neuronal architecture to clinical applications so lest we think that it's a new idea to apply neuroscience in clinical settings or think it's too soon or too late it's just always going to be sort of what people are recalibrating to and in this figure it's connecting sort of the multifaceted attributes of the self and consciousness so again used maybe vaguely maybe detailedly like long-term memory of the past evaluating systems of value attentional focus regimes of attention as we would say perceptual systems in the moment and then motor systems in action in the future so it's like all those things are being connected and what connects the sound in the ear to the memory to the action pattern like if someone says march what connects that and the answer is something like a system at criticality that propagates from a sensory modality and if we're a quieter it wouldn't have been heard it wouldn't have propagated if it's the wrong language it doesn't propagate so it kind of only catches when it's all aligned with the rhythmic oscillations and the prior state of the network which is equivalent to constituting a Bayesian belief network so that's sort of the global neuronal workspace model is it's going to be integrating all these features that are kind of disparate cybernetic features memory past present future self other thinking through other minds all these features of neuroscience that everyone is trying to understand and model and put that in a network science frame so that maybe it could be brought into the clinical applications just like their title is getting at even 20 years ago or other avenues like how do we discuss neuroscience with the public what kind of metaphors will they receive is it like a network that activates or an avalanche that falls or a program that halts different ways that people think about systems neuroscience any thoughts on that before we can continue yeah I think just generally speaking this this idea of this integration is really useful when people are thinking more broadly about you know often people are thinking about us as a system but you people fall into this trap of a kinesthetic thinker or a visual thinker you know some of these kind of folk psychology systems things and NLP and that and this approach gives a plausible way to understand what it's all one integration and it's not that you have someone going off and their visual system is in one part of the brain and this piece you know and in a way this is permeating how culture kind of thinks the brain works so it and then bringing that in with the free energy is a way that could actually integrate and it could be part of that and now this prediction dynamic it is quite it is quite possible that this could start to be more understood more generally as a model that is how we sort of work you know as to the different styles of learning or sensory modalities it could be that some cognitive diversity is such that the same symbols on the page will ignite something that the audio doesn't depending on where they can have their regime of attention which is a culturally inherited and co-created artifact so it allows instead of just saying well it's left brain right brain or it's this kind of person that kind of person it's about the dynamics of the system and oh would it be possible to do some sort of exercise to sensitize you or to allow you to perceive some dimension that your brain hadn't seen just like there's a good music or a good wine or something but it's something that is encultured and trained so let's get a little bit specific into the neuroscience before we go to the figures and that's just to talk about this ERP or event related potential and the ERP is a pattern that's kind of like a collective behavior of an electromagnetic field it's based upon EEG measurements which are electroencephalograms and those are just like electrical measurements around the scalp usually but it can be other places too and each of the little leads each of the measuring points on the EEG is a reference channel and then what happens is you could be taking these time series of the electrical potential at these different reference points and then using techniques like SPM and other approaches to reconstruct in a generative capacity the electromagnetic field that gave rise to that data distribution of event leads references channels then you can align time series to events like if you clap your hands 10 times separated by 30 seconds you could take that big time series and then align all the claps and then ask is there like a coordinated probabilistic not every channel does it but when you align it you can see that yeah there is a dip and so that's the ERP is when there's this spatial temporal pattern that might play out over tens or hundreds of milliseconds as shown here and in different brain regions it's known to have a canonical form and even diagnostic value in certain situations and there's a lot more to learn about here Friston's work Buzaki many others but it's a big area but that's kind of what it is it's like a pattern of space and time in an electromagnetic field in response to an event any thoughts on that because it kind of comes into play for the paper yeah nice and I noticed that I am this event related potential so yeah again it's giving you a way to think about what happens when so they've got P1 P2 and P3 they talk about P3 more I noticed in the paper as this kind of main sort of lag but it gives a sense of this temporal dynamics that are played there's an idea that there's something happening that is more cumulative maybe takes more time to go to register and this model gives a plausible way where that could hint that some of the faster processes are actually sort of more closer to this flux and some of the others are closer to this awareness it's not like the PQRST system on the heart EKG and so you can say oh there's long QT interval it's because there's sort of a normal shape with variation and then if something is outside of a certain bound you can say this sort of interval is too long and then you can have a model of heart cells and if you had a model of heart cells with just the channels and you were getting the range of expressivity observed in the biological system it's pretty meaningful so let's look at figure 1 the first half of figure 1 I guess figure 1a we could call it but we'll have figure 1b2 this is kind of the classic active inference graph this is one of our favorite figures and it's good that they put it in all kinds of papers what it describes is the relationship between at the starting of the bottom observed measurements in the world through matrix A to hidden states of the world like is it actually light in the room is the hidden state and then you get photons on your eye and then there's a pretty clear mapping there but A could be probabilistic as well so that's about states that are hidden to observations through time that are then this is actually a second level model where at each hidden state it's evaluating what is going to be the state outcome mapping under a future time series so this is a little bit not the basic basic version but hopefully you'll have listened to a few other like their model stream where they walk through from really the fundamental which is just the state outcome mapping so go to the model stream if you haven't seen this before at all then D is the prior that plays into the second level state estims and just like the B matrix at the lower level defines the transition between hidden states the B2 as opposed to B1 second level B matrix describes the transition frequency and patterns of the states at the higher level so this is kind of like that nice nesting you could have the higher level state changing and then the lower level states changing what does influence the higher level state transition matrix PI policy so that's the innovation or one of them of active inference which is that instead of just estimating how states change hidden states change through time we estimate a conditional related to how our policies influence states changing through time that's the PI coming into B2 our policies are selected from our affordances and they're sort of differentially weighted terms of their instantaneous attractability in terms of this G functional which is a expected free energy minimization of policy down here G of PI not just some abstract G floating out there in space but of a policy and it takes in arguments like C the preferences and C is the preference vector so I think that's all the letters on here good any thoughts on this Stephen can go to this figure one B now I think I think this is a one thing I've noticed is also they've got the top the variational free energy which is kind of the the broad kind of dynamic that flows as you mentioned and like you just mentions it's the when you start to got the policies involved you've got this expected free energy which is more tied to how that might relate to particular actions for the organism so I think that makes and also I suppose the other point that's there you mentioned there you know the B1 matrices it could almost be assumed that that's the that's like the flux it's kind of it's keeps churning over and it will it will it will work out it will come and start to populate its values whereas these higher levels you have your priors it's in you know it might not be that important where my retina is in its state at this moment it will just populate and start to come to some equilibrium whereas what I think I'm seeing my prior might be these higher levels a more significant parameter and that's reflected in that model very nicely said about how our priors or system priors even to remove the whole awareness consciousness thing system priors are semantic it's like I expect I'm a veterinary scanner I'm a check scanner at the ATM I expect a check it's like it doesn't expect the pixel to be in this spot and the green this way to be here it expects something semantic and part of the challenge with talking about observations and making meaning from data and sense making from large data and scaling sense making is that the bottom level data is almost by definition meaningless especially with respect to the larger level semantic question so yeah there's parts of the check that are so meaningful but the pixel can't be too meaningful otherwise you need to do a whole another model to really break that down so it is interesting that the D prior enters at the second level state but there's no D1 so maybe it's not shown or maybe we can ask where that's coming from let's look yeah maybe it's not it's not necessarily so important for the modeling and so that in a way this is quite useful for showing I mean in some ways they say that's where autism can have challenges because that there's not that ability to know where to put your priors and to resolve things so just one other point in here is they don't in the expected free energy for vision because they're looking at salience of vision or Frank the idea of risk which is sometimes shown in that expected free energy it's not in there because I suppose for this model that's not really it's not relevant for the vision system in the type of way that they're exploring consciousness so it just shows again that the way that the equations or the terms can be constructed is related to the nature of the way that what's most relevant to the type of dynamic that's being looked at and I think that's quite interesting right it's not the free energy of like the system with a no function in mind it's a calculated free energy that is related to policy as implemented by some actor in the space so it's not just like we're making one general free energy principle model of some system we're doing tasks specific behavior and I'm sure that Ryan and Christopher will have a lot to add on that so here's the second half of figure one be our figure one and it takes some of those letters that we saw in the first part of figure one a b c d and then also adding an e I think down there which is the kind of affordances prior which is not here but it connects to pie as well so there's a few other aspects and then it maps it on to parts of the brain human style brain that are tractable or plausible or where the architecture of how those brain regions are connected or functionally linked through dynamic causal modeling SPM so if there's a functional or an anatomical connection between these brain regions with that directionality it seem to be some might think of it as confirmatory or supportive evidence for that biological systems do free energy principle or active inference another way of thinking about it is it's the computational realization of a process that surprisingly or unsurprisingly can be modeled with active inference type models there's a few different ways you can think about it but they're kind of laying out this probabilistic graphical Bayesian network and they're kind of laying that out onto the brain's neuro architecture and there's a lot more papers that go into detail here any thoughts or ready to continue I think an interesting question is like where does the body come into play like this is one level of the nesting but how far is this model towards just basic group communication models or some other type of modeling anyways let's return to the sort of specifics of their experiment they included figure one I think to be just very clear about active inference which is great because it helps the paper be accessible to those who are outside the field and also everybody in active inference can always go back to the basics and the terms because it just always helps us here's what they do in the results the first part and then we're going to go look at figure two and the task so results section three as a proof of principle we first simulated 200 trials so they're going to be having a behavioral paradigm where they can have a fully in silica fully simulated experiment so it's going to be like an experiment that a person could do but they're going to totally simulate it and some of them are going to have square present and some obviously won't and then there's going to be model parameters that relate to attention and model parameters that relate to stimulus strength so off the bat you might imagine okay well if the stimulus is stronger like the sound is louder to some point it's going to be easier to say yep I heard that sound so if it's a bold line versus if it's a tiny faint gray on gray situation and then there's also the attention so if it said pay no attention to the middle look away and then something flashed in the middle you're less likely to see it than if you were to be looking at the middle so that's the kind of an experimental setup that they're going to be simulating completely in silica here is what they're going to show in figure two there's this sort of visual pattern at time point one and this is what also allows them to use discrete time models like as currently shown the active inference model where it's like T1 is here T2 is here and T3 is here so they can actually get a lot of power out of this sort of like checkpoint type almost kairos driven analysis really and then they show an ambiguous stimulus where it replaces these diagonal lines with partially with a square and then it flips back to the bars the agent is then required to either say yes I saw a square or no I didn't see a square so that's the sense and here's the action so if anyone has made it this far and was still like wait but they didn't explain consciousness of this system or of theology it's not this paper this is only about this task paradigm any thoughts on that or continue this was just useful in terms of from their think in their discussion I want to read a little more about why the time scales are the way that they are okay figure three is a Bayesian network description of the generative model with arrows showing dependencies between hidden state factors and outcome modalities okay so this is kind of like at first I was like wait what is this it looks like a an app or a game or something like a network model but it is actually so at the level one let's look at these are the hidden state factors these are the report would be I'm waiting I haven't reported yet which is actually a state seen or unseen so this is sort of one of the hidden state s factors that we're going to fit with data then there is the trial phase which is the time steps of the model and then there's going to be the sort of visually differentiating features of the model so it's sort of like you know round one any one first out it's kind of like identifying where you are in the model as a state in the model which is something that was really important in this framing of the model so check out the model stream because it matters if you're in trial 2 row 3 or trial 4 row 2 or something like that the very bottom level is the square or the lines here there's also this red or black thing so I'm not going into that but it's whether there's the square or the lines being seen and then words I silent or I see a square didn't see anything so it's sort of like the generative models if they saw it or not so we need to have some integrated way to combine the trial phases and what is actually being displayed which we control in this experimental context with their hidden perception as to for example the sequence of two different things so if it was first the square or the diagonal and then red or black you could have four different combinations and if it is held across trials you had to have like been aware of it so this thinking would go like I'm going to read you a color and then a name and then you're going to tell me both in a row that they're going to be able to say and then that's the attentional mask because that allows them to say not report on the color but report on just the square being there or not so it's kind of like everything in this experiment is like locked down not that everything the person's shirt color and how they felt about it not that that is all explained in this model but it's sort of modeling the input and the output and then leaving the rest actually for context so we can hear more about that from the authors anything on figure 3 yeah no I think they can speak to that more I think one thing is was interesting is they use this these different words rather than just like red or no you know just basically saying red when you see red or black when you see black they had a rationale there to separate it out so that it helped them have a bigger time window but I think they could speak to that better than I can yep nice the level 2 outcomes are level 1 hidden state factors so the higher level is getting connected to the lower level in this way and then also if you really follow some of these lines like wait only maps to silent and not only maps to the observable silent seen maps to different things and unseen they both are I see and what could be I see that's where the divergence point is they're both about a perspective and perception and then a swear that's only for this thin bash for seen or I didn't see anything from the unseen condition so it's kind of showing how even the lexical tokens are mapped from probabilistic estimates of higher order models and that also relates this quantization and this quantum nature of communication and it's like you're forcing them to say I saw it or not it's not a slider how sure were you saw it you could design that experiment make that model but in this paper there's a lexical interpretation of their actions because they're only given the option yes I did see it no I did not see it so by constraining the experiment you can do things again leave it up to someone to debate whether that is cutting too much away so four is split in half and here's where they sort of are just proof testing their simulation that's the foundational simulation and this is just to check some of the basis of their model and what we can see on this left one is that when the square is shown so here's the four possible hidden states it's like four horse racing lanes and we can see that there's this gray that's the red one is never even on the table because they haven't even brought that level into play yet but there's this gray competition 5050 I'm not sure that's just reflecting the prior of whether it's going to be the dashes or the squares because did you see it or not and not is going to be the dash and then the stimulus occurs and then there's rapidly a convergence towards yep I saw it and that convergence is associated with a transient period of elevated firing rates of the first level of the model so that's where we're going to get some of the cross experimental modality work because this isn't just a Bayesian phenomenological analysis of probability it's actually linked to a mechanistic process model so if you found a brain region that were firing that way and removing it did the same thing that removing this did all of a sudden you'd start to be seeing a lot of model reification and then they also tie in the probabilistic component firing rates the first level firing rates in the self report and then conversely on the square absent condition it's the same thing it's gray and then there's no transient period of firing and then it just converges towards not seeing anything so it's like it was waiting it was waiting and then it converged upon not seeing anything so that's figure four that was just sort of like because they're like oh yeah after three four five like I'm waiting I'm waiting waiting up no just didn't happen so that was one now let's look at another a little bit more nuanced way to look at that in five so here there's two axes it's kind of like a Punnett square or Eisenhower matrix a two by two and it's called a four way taxonomy so just like we kind of hinted at earlier you can tune model parameters to have low or high endogenous attention and then you can tune model parameters so that the stimulus is like shriek is weak versus strong so that would be like thick versus thin line or it could be rapidly flashed versus not so you can imagine if it's a weak signal and the attention is absent very few are going to see it if it is a strong signal and attention is being paid you'd expect a higher amount to see it so empirically what you do to make the attention increased or what you do to increase the signal strength that's a detail question but their model is doing quite a few things here and so it's a high energy high density representation because they're showing that basically they do recapitulate that pattern with high percent seen self-report and high correctness rates with an archetypal firing rate at the first and the second level that are reminiscent of ERPs and the ERP P3 looking like arises at the second level so it's not like the data have a structure that give rise to the P3 it's like the P3 is an emergent behavior of something that as you're saying is more like forming and unforming at the lower level and the semantics and then the response that we actually observe the system do is this higher level and then it's absent in the other three cases and in those cases the accuracy here is almost 100% in this it's 58, 47, 61 so it's about 50, 50 and remember they have to make a forced choice and they're not always sure their confidence isn't 100% so that just shows that like you get confidence and experience and firing rate and second but not first level ERP dynamics in the high attention, high signal strength condition very clean finding and then you can disentangle attention and signal strength and then just show that like whether you pull one away or the other you destroy the ERP you reduce the number of people and the simulation runs that said they saw it and it reduces the accuracy to a large extent so that's like a big finding of the model any thoughts on that? yeah thanks that's really good to go through that I think because there's a lot in this diagram it can be a little bit to first see and I think you know first level and noticing how all four quadrants have got this peristimulus time this kind of first level firing is present like you just mentioned in the black graph and then only on that bottom right you have this very strong so this does indicate that when you have sufficient signal in relation to attention bang you know it has a certain on-offness to it which is kind of what they're trying to show very nice to connect it to the ignition model of the neuronal workspace which is like if it's loud but you weren't paying attention sufficiently then you're not going to or the other quadrant it doesn't ignite but when you get both together when you get somebody looking at the screen it's not just a little bit different than them not looking at the screen it's so different it's qualitatively different it's a categorical difference and so that is something that's reflected categorically in this model flat versus not flat now it's a simple model but it makes me it's a simple model but this is like a lot for a simple model to show I was just going to make a little anecdote there it's the sun it's like the sky but when my daughter was I think she was like nine months old we had the fire alarm go off and it was literally like ear breaking the loud she was sleeping through it all so you know that I know it's not but the same kind of idea is there this idea that because obviously I know it's sleep but it's still an attentional state and you'd expect someone to always wake up because of the loudness it is an attentional state absolutely and it's encultured and it's developed and somebody could have phases or you know all these things can come into play it is like is the noise hitting the eardrum yes are the neurons firing that's where predictive processing has a lot to say and that's why it was important theory because the centralized processing theory would be like it hits the baby's eardrum it goes full force loud sound incoming and then there's the nap time region and that just says nope I'm sleep we just don't listen to sound when I'm asleep the predictive processing model is that there's a generative capacity like we're sleeping almost semantically and that propagates in a way that dampens the ignition of loud sounds so the net effect is loud sounds don't wake the sleeping baby but it's extremely different if there's like this natural commander that's receiving the raw sensory input and processing it's not what happens and it is much more like this generative semantic capacity of systems to have meaningful modes that are distractible or not distractible to different stimuli so pretty nice good good baby story so here's another phenomena that they're going to recapitulate so let's expand it in six so this is the inattentional blindness task and it's just another task where basically they're going to differentiate a few different experimental treatments that usually would be used to pull out intentional inattentional blindness but sort of just the summary of this is in phase three which was the self report you can read the caption of what they are but similarly depending on which pieces and phases you add in you get the existence of the simulated ERP and a high fraction of having seen it if there was attention but then in the case where it wasn't seen then it just is like a totally flat piece so this is just uh then from the caption says in phase two firing rate plots illustrate how more gradual updates in the second level beliefs do not produce p3 like ERPs so it's interesting to think about how they can actually titrate stimuli and find out was the system updating all of a sudden or was there more of like continuous updating but these are just kind of going into specific um examples that are like laid out in a clear way but it's just it's a lot of info that might be interesting for some but there's five more so let's just rapidly kind of go through the figures yeah figure seven is another layout of their experimental results and again it's a sort of spread where on the left you have no attention absent attention and the right there is a regime of attention parameter and then similarly weak strength signal on the top strong strength signal on the bottom and they're looking at the effect of a third dimension so this could have been like a hypercube of statistical results and that's what this model and the Markov decision process allows is instead of just making one model and crafting it we could make like a spectrum of 5,000 models that are just incrementally different and then report the whole distribution so here within each quadrant the left is inconsistent prior and the right is a consistent prior so here consistent prior strong signal high attention now we can kind of imagine that's like the focus you know paying only attention to that and blasts right in your face and it's like 100% saw it and 100% were correct really salient stimuli really clear self-report no one's really doubting that they perceived it with qualia so these are the easy cases okay everything else is the gray zone because if you have an inconsistent prior again it's just a model we're just experimenting and thinking about it but it's interesting that 94% saw it 100% are correct so even with high attention and high signal strength if your priors I'm not sure about what I'm seeing then 1 out of 20 people in the simulation again just loosely 1 out of 20 will still not see it but it was right there and they were paying attention to it so there's a lot of kind of psychology anecdotes about stuff like that like the whole room says that line A is longer than line B and then the last person just is like I guess it's that way even though they think that they're there to actually speak the truth so people can look at the percentages and read more stories but this is a very interesting modeling framework because we can talk about how these different components of certainty and sense and action are related in different people's perspectives I mean this could be interesting as well where it talks about this inconsistent prior when you even have a strong you know a strong signal but if you've got an inconsistent prior you may not be aware of something and there's this challenge in a way where people can be tricked you know like we've seen with the internet and Cambridge analytics if you feed things through at a level where it's not a strong enough signal but it has associative qualities you may be starting to have that feed into your consciousness at another level but it's not activating your awareness so I think that some of these questions you know that and people might not notice that happening because when they do pay attention they you know they think they've seen everything there is to see but there may be some things which are just inconsistent with their priors as much as they're weak you know they could be and there's a little bit of an interplay potentially there that could be going on which we are not aware of nice and then also like even on the other extreme if it's consistent prior but low attention and weak signal you still sometimes report it and you still might be correct even at a higher amount of correctness and so a lot of times belief and faith are as if they're to be ridiculed and that's just so far off from the way that everything is figure eight is again another two by two with attention and signal strength and they're just really going into detail now overlaying this prior consistency in consistency question and adding a neutral prior as well and now overlaying those three so to be almost like there was three cells here but now overlaying those is three traces of the ERP so what they're showing is that you get sort of the prior is like a fine tune it's like a tweak but it remains pretty much to be the case that you see the categorical difference amongst each quadrant and interestingly the one that dips down gives like a quasi P3 is the consistent prior strong signal low attention so that's like you're not really paying attention but something loud happens and you're sure that it's possible that's actually the one that comes closest to actually inducing and we can look at here percent C95 low attention high signal strength so it's pretty interesting that it's easier to have again you have to calibrate it's real people to notice something when it's a strong signal even if you weren't paying attention than it is if you are paying attention but it's a weak signal but maybe there's values for which it's one where the other but it really separates those two things like is this a subtle point in the text or is this person just not paying attention to this sentence and those are sort of two questions and you want them both to be clear and so that you can align the regime of attention but that's why this is such a cool framework okay we're not going to go into too much detail here but this is a experimental paradigm laid out with sort of classical psychologist style of quote independent manipulation of expectation stimulus strength and attention so they're able to use repetition and variation in a specific way so that you can disentangle certain attributes from the sequence of behavior which is something that we actually heard about from Adam Saffron and Colin D. Young in the first streams of this year with the way that you could look beyond the personality surveys to the cybernetic big five personality traits so this is kind of like similar it's revealing higher order parametric values from behavioral sequences that disentangle or just like a gambling paradigm might be high risk low risk high reward low reward so you're disentangling a few different attributes the person's model it's kind of like that but for visual consciousness access attention and then just to go through the a1 and a2 supplements just before we kind of hit some closing thoughts so a1 is their specification of the first level generative model matrices so let's definitely have them talk about these but if you'll remember from the model stream it's all about specifying the matrices and the calculations are all about matrix multiplication small or big or nested series of matrices or lists of lists of lists and so each of these outcomes at each level when we're looking at figure 3 this kind of colorful connectivity model each one of these are basically reflected by a row or a value in a matrix two spaces 17 matrices seeing a theme with the matrices and how they can represent different things like attention or a word so then a certain combination of rows is I didn't see a square or I see a square but there's semantic meaning to the composition of a matrix and then you can do probabilistic inference using that matrix and then the second supplemental figure is the second level matrix structure it's like you thought the movie was long wait till you see the higher level matrix and here it's similar except now we're dealing with these hidden joint states and that's why that was the second level in the model where these higher level states that also deal with the behavior of the agent so nice figures by them and they really did all out yet still it's kind of a lot to think about so let's have them walk through what they are but I guess with that I'll just kind of bring it to the ideas and questions where we can go and see when you just talk for as long as you want just what what do you think or what was like where we take that or who do you think we find that interesting or important yeah I think well now I think that we will find that a lot of people who are trying to understand how this modeling can be applied into other contexts and I think there's probably going to be more and more of a movement for people to offer modeling approaches more broadly so there's going to be people who are going to want to make models there's people who are going to want to work with people doing models people want to look at what the implications of the models are so I suppose that's something this paper is definitely from what we've been talking about and so maybe even asking Ryan and Chris a little bit there about you know how would someone maybe engage as they're working with those types of matrices and looking at the outputs to sort of put that into a transdisciplinary team in context you know that could be quite interesting because this is all it has this potential really without asking too much of them but maybe because that sort of thing and maybe just how if there's different ways when you start to think about the A transformation matrix the B matrix and the D matrix just how those things can be are there any good practices for just helping keep your head straight with all of that as people start to you know because I know for Ryan and Christopher it's second nature they can switch in between but it does take a little bit of it takes a bit of tacit knowledge to start to really embody what's going on in there and how that translates into this kind of common language but this paper I think gives a sense of that but it's still hopefully that's where this live stream might be useful for people is as we go through that especially when it got into some of the later model scenarios yeah I got a little bit it was a little bit like a lot of the processing to try and work out what exactly was the paradigm that's going on and again that might be because this is specific to people working in visual consciousness and some of these types of experiments so so that would be that type of broad question I think might be quite interesting to and maybe even it will yield other model streams in the future if they're even interested build on that but otherwise I think that covers it for me well said thanks for bringing in the specifics of the model but also all the ways in which especially through teamwork and communication we can be working on it and having an impact and that's really an interesting area I can see why you're interested in it but what a fun discussion it is always good to read the work of Christopher and Ryan it's really just informative at multiple levels so we'll close with that thanks again Steven for participating here and we'll hope that people are interested in coming on live for 18.1 or .2 or if it's after those dates then just leave in comments and kind of continuing the discussion that way well that's it