 Thanks Daniel for, oh sorry. Hello and welcome. It is February 9th, 2023. We're in active gas stream number 36.1 with Ben Fallendes. So Ben, thank you for joining. We're gonna have a presentation and then a discussion period. So thanks again for joining, looking forward to your presentation. Yeah, thanks so much for having me. I'm really excited to connect with the active inference community on this stuff. So hopefully some people are in the live stream and can we can have some discussion at the end of this. So I'm going to talk about this recent preprint that we dropped with my collaborators, Jeff Yoshimi, Bill Warren and Michael Spivey, titles right there. So I'll just jump into it. So the backdrop that we're thinking about when we came up with this paper was that most cognitive scientists think that cognition is about mental representations. And most of them think that those mental representations are constituted by brain dynamics or in philosophy jargon you might say supervene on brain dynamics. And so hopefully that's not a very controversial statement. And what we're trying to do in this work is take one step towards a non-representational account of the central nervous system, meaning the brain and kind of connected parts of the central nervous system. So why should we want to do this? What's the problem with representations? So I'm just going to present a sample of four problems. This is not a comprehensive view of what could be wrong with representations, but these are some things that might encourage you to look for an alternative. The first one is that just because you have some encoding or correspondence between internal mental activity and something out in the world doesn't get you to content to meaning of the representations. And that is basically getting at something that's called a symbol grounding problem. I'll talk about in a second. Beyond that, we can note that in central nervous system activity, there's a high degree of context dependence even down to the tunings of individual neurons. So given a different task or a different setting, you see what look like different encodings appearing. So that seems like it would be a problem if you're actually trying to use that for an encoding because you have to keep track of how those encodings are changing according to the context. The third problem I'm going to mention is a recent finding in the neuroscience literature. It's called representational drift. And the point here is that when you see what looks like encodings in the brain, if you look over long enough time scales, and these are not actually very long time scales, maybe even a day or a couple of days, you find that those encodings move around in the brain. So again, if they were to be used by the brain as an encoding, it not only has to keep track of what is the mapping, but how is the mapping changing over time? That's a problem because the system that needs to keep track of how it's changing is also the thing that's changing. So it's not clear to me if that could even work or what kind of computational mechanism you'd need to make that work. And the fourth thing is the argument that we just don't need representations, at least for a lot of tasks. So this is a major point in the ecological and embodied literature is that a lot of the major problems that organisms are trying to solve appear solvable without using any complex internal representations. So I'll just briefly give you an example of each of these. The first one, I'm going to rely on this really nice 2019 paper from Romain Breda. I apologize if I'm pronouncing his name incorrectly. And one example that he shows in this paper is, we imagine a scenario in which neuroscientists is doing a single cell recording of the tuning of a visual neuron. And they're varying wavelengths, right? So they're varying colors, presenting various colors. And you're seeing which color elicits the greatest response from this neuron. So we've got wavelength and the x-axis and the output or the current coming from that neuron on the y-axis. And what these gray lines are showing are different intensities. So the point here is that the output of the neuron is not just a function of wavelength, it's also a function of intensity. So that means that you can get the same output with multiple different combinations of wavelength and intensity. So you could have kind of high intensity, low wavelength, or, you know, medium intensity, medium wavelength, and so forth. So if you're a neuroscientist who's holding intensity constant in this context, then you might read that activity as a mapping. But the point here is that the brain doesn't know that the neuroscientist is holding that constant. Even if you tell someone, there's no way for the brain to look outside itself and see that that mapping holds. Now somebody, you know, might point out that actually neuroscientists don't typically think that single neurons are encoding wavelength, but they might say that the relative response of multiple neurons is encoding it. So on this plot B, we've got two different neurons with different tunings and you present it with some wavelength and they have different responses. And that might carry intensity and variant information. So that's helpful. But what I want to emphasize here is that wavelength and intensity are just two properties that might emerge from a stimulus of which there could be many more that we don't know of. So it's only by virtue of knowing that the response of the neuron doesn't vary according to these other properties that you can actually call it a code. And that's something that the brain can never do is figure out what other things could this neuron respond to. So this is kind of a neuroscience framing on what's called the symbol grounding problem. A lot of cognitive scientists might be familiar with the Chinese room thought experiment. The idea here is that a person is locked inside a room, all their exchange with the outside world are messages that are passed through the door that come in in Chinese. And the person inside the room doesn't speak Chinese, but they have a book that translates from one set of symbols to another. And so they get a message in, they translate it, and they pass a message out. And to the person on the outside of the room, it looks like there's a native or fluent Chinese speaker inside the room. But of course, the person doesn't know any Chinese. So what this is trying to point out is that if all you have are mappings from symbols to symbols, you can never figure out what the meaning of those symbols are. And this is the symbol grounding problem. I think it's kind of a fatal problem for any representational view. And with all the buzz around chat GPT and stuff like that, people are having debates over whether or not it could be sentient. This is something that comes up a lot, is that really all it has are these relations between tokens. And at the end of the day, doesn't know what it's talking about doesn't have meaning. And for me, and many others, I think that grounding your symbols or at least being able to avoid this problem somehow is crucial for understanding consciousness or subjective experience. So arguably, if you have a representational view, you might run into problems naturalizing subjective experience and consciousness. Second problem I mentioned is that central nervous system activity can be context dependent. So here I'm going to show an example that comes from Gilbert and Lee 2013 top down influences on visual processing. What they did is have chimps, I believe, do a task switching paradigm. So what you're seeing on the right is a setup where there's a square in the middle, and it's got a line inside of the square. There are lines on the right and left and also on the top and bottom. And so at various times, the chimp is going to be paying attention to some task that's going vertically, or they might do another task that's going horizontally, and they're supposed to ignore what's happening on the other side. And what they looked at was the tuning to orientation of individual neurons in this case. So what they found is that depending on whether the line was presently task relevant or task irrelevant, you get a different response from the same neuron to the same line. So it seems like it doesn't even have a fixed kind of tuning or preference for any orientation, but it's going to depend on the task. Again, once it becomes context dependent, then you need a way to keep track of the context, but you get into this infinite regress where the brain just can't step outside itself to figure out what the context is. The third problem, pretty recent finding in the last couple of years, this idea of representational drift, and this is starting to be demonstrated in more and more studies. And I expect there will just be a flood of these coming. The idea is that we've thought for a long time that the brain can implement certain kind of correspondences that look like topological encodings, for example, spatial encodings. So in this classic task, they had a rat doing a T maze. So it moves down the length of this T and then goes towards reward. They trained the rat, got it to master this maze, and they record the activity. And so first I'll direct you to just this top left panel. What we're looking at on the X axis are different locations within the maze. So this is designated as Q offset. So basically how far away from the reward. And then on the Y axis, we've got different cells, and they're sorted according to their response to particular locations. And so what we see in this first panel is that as the rat moves along the maze, you see this nice kind of topographical encoding that there's a series of cells that are responding across the length of this T maze. And it looks like there's a spatial mapping inside the brain. But if you look at those same cells on day 10, you can see that that spatial mapping has totally faded. And by day 20, it's completely gone. And on day 10, you have a totally different set of cells that are implementing that spatial mapping. So it's some distributed mapping in the brain, but it's moving around over time. This has now been demonstrated in primary visual cortex, primary olfactory cortex. So these are low level sensory systems that presumably everything higher up is going to rely on. And so reading the responses of neuroscientists this are, they seem kind of shocked. The question is, how could you have an encoding system that's based on a foundation that is constantly shifting? And the final point that I want to make is that for a lot of cases, we might just not need representations. And this is a big emphasis within ecological psychology and embodied cognition. And so I'm going to mention an example that is probably the most used example in those fields, which is the outfielder problem. And the idea is that there's a baseball player, they view a ball that's just been hit or launched in some way, and they have to try to run to the landing location and catch the ball. If you're a representationalist or kind of a classical cognitive science approach to this, you might assume that the baseball player first looks at the ball and they calculate some predicted trajectory of the ball, that gives them a predicted landing location, and then they run to that location. So there's a lot of kind of internal machinery and representations happening. Well, the alternative is that the baseball player can just run so as to keep the ball constant in their visual field. And that's a little bit of a simplification of what it is, but essentially if you move so that the ball stays, it's kind of acceleration left or right and your retina stays stable, you're just going to end up where it ends up. And so the idea here is that you visually couple two properties of the ball, and you can get the job done without needing to have any internal representations that you're reading. So we use the ecological framing in this paper, so I'll tell you a little bit more about the background and the kind of literature that we were trying to engage with this. And some jargon to introduce is that the Gibsonian or ecological psychology view says that what organisms perceive are not objects, it's not the environment, it's affordances. And affordances means opportunities for action. So a chair affords sitting stairs afford climbing. And what you see are those potentialities for interaction or what you perceive, I should say. How do you perceive those? Well, the idea is that those are specified in ecological information. So there's structured energy in the environment, and that energy can be revealed through relationships between action and perception. So for example, if you're swaying a little bit to the left and right, you're kind of changing your angle of view on some objects, and that's going to reveal depth cues, how far away is the object from you. So what I want to emphasize here is that Gibson was trying to point out that the environment is actually rich with information, as opposed to the standard view that sensory systems have impoverished information. So if you think that there's not enough information in the environment, then it makes sense that you'd have to do all these internal calculations and inferences to figure out what's out there. If you think that the environment is rich with information, then maybe you can just skip all of that stuff. So affordances are specified by ecological information and when you detect ecological information, Gibson says that you're resonating to it. So what does it mean to resonate? So he's kind of using this example that's related to these two tuning forks over here. The idea is if you have two tuning forks that are the same resonant frequency, which means the kind of natural frequency at which they're going to vibrate, and you strike tuning fork A, then those sound waves will travel out and cause tuning fork B to resonate. So I think what Gibson's pointing to here is that this kind of coupling of organisms to affordances is a direct physical coupling and not an inferential coupling. And Gibson described learning as a tunement. So one thing I want to point out here is that in order for two tuning forks to resonate to each other, they have to have the same resonant frequency. So if one is bigger than the other thicker different material, it might have a different resonant frequency. And if you strike tuning fork A, it might not cause tuning fork B to resonate. But what you can do is change the properties of tuning fork B a little bit. You could put a damper on it or other methods for altering the resonant frequency, such that it's going to couple with tuning fork A. So that's his idea about learning is that we're actually just kind of gradually refining our perceptual system until we just happen to resonate to useful information in the environment. So what's the what's the mechanism of this? How does this happen? What Vicente Raja and colleagues have pointed out recently is that resonance remains a metaphor. So Gibson described this idea, but he doesn't give a mechanism. If we want a mechanism, we probably need to look in central nervous system. So it's not just that any type of system can do this, it requires a certain type of system. A slug, for example, can't go catch a fly ball. It's not going to be able to resonate to that information. So that might be a bad example. But the point is that we want to know what are the specifics of the system that allow it to change its resonance properties and ultimately pick up useful information in the environment. Ecological psychology is really focused on this level of the organism environment relationship. And that's a purposeful thing, because they're trying to emphasize that often we don't really need to look at brain activity to get a more or less complete understanding. We can get really far just at this top level without appealing to this lower level information, but eventually we want to get there. So that's what we're trying to take a step towards in this work. One model that I think is useful in this domain is the reservoir computer. So some cognitive scientists have pointed out that reservoir computers are interesting as a model of cognition in general, because they give you these three properties that we might want from a cognitive system. Actually, first I should explain what the architecture of these systems are. So we have a little diagram of a simple example of a reservoir computer. The idea is that there's some input layer that has feedforward connections to a reservoir. So this is just a collection of nodes that are connected together with random sparse connectivity. And that gives it the property of not having stable attractors. So if you give it some input, then the input is just going to kind of propagate through that network over time. And then it feeds forward to an output layer. And usually they also have a connection from the input directly to the output. So what's cool about these? One is that it gives you dynamic memory. So the idea is that when you project an input, when you give an input to this reservoir, what it's essentially doing is taking a low dimensional signal and projecting it into a high dimensional space. So that allows you to separate information from a lot of time scales, but also have it be interacting constantly. So you can have things that are changing on slower time scales, for example, interacting immediately with faster time scales, that the network is kind of setting up a context for itself, and it's preserving all that information across time. And so I kind of hit on both the dynamic memory and the time scale integration. Also, if you have multiple inputs to this network, like multiple sensory modalities, it's going to automatically integrate those things. And I should take a step back to hit a point that I missed about the dynamic memory is that in the kind of flow through this network, it is implicitly preserving a trace of the history of inputs. And so this to understand this, you can think of the kind of inspiration of reservoir networks, which is an actual pool of water. So if you're throwing rocks into a pool of water, it's going to create some pattern of ripples on the water. And as you throw more rocks in, those ripples are going to start to interact with each other. And if you have a way to read those ripples, then at any given time slice, you can figure out the history of rocks hitting that network. So the history of perturbations. So some long memories kind of preserved in the rippling pattern here. Another thing that's cool about these is that they're computationally efficient because you just need to train a linear readout mapping. So you don't have to have any complex back propagation, the weights in the network don't usually update. And you can train multiple readouts at the same time. So you can have one input to the network, and you can have multiple parallel computations being performed on that. So this is cool for ecological psychology, but one problem with it is that it's still pretty representational, right? So you're training a readout mapping. We, you know, requires an observer that's going to suffer from the symbol grounding problem. And it's not very biologically realistic in the way that real neurons do. So the neurons are not updating inside the reservoir in any way. The weights aren't changing. We know that actual brains are very dynamic. So what are neurons if they're not, if we're not going to think of them as, you know, vehicles of representation, primarily they're just living organisms. So this is the focus that I tried to take here is thinking of the neuron first as an agent that's just trying to survive and that the properties of brains and the properties of the collective actually emerge from these individual level mechanisms. What are some of those mechanisms? So I'm going to call these mechanisms of homeostasis, meaning self maintenance. So neurons trying to stay within some viable space of input to point out before I begin this, if neurons don't have any input, they're going to die. So they need some input, they actually compete with each other for inputs, they're kind of putting out tendrils and trying to form connections and the ones that get good connections are going to survive ones that don't get enough connections are going to die. But also if you have too much input, it's a bad thing. So you can get something called excitotoxicity, which is a neuron is basically going to explode and die because it has too much input. So what can they do to deal with this? One thing is to adjust intrinsic parameters. So these are things like the leak or pump rate, the number or location of receptors, and there's lots of kind of gene regulatory processes going on where they might be turning on or off different genes in order to change their local parameters. And I want to point out that although this is homeostatic, that can change the set point. So if you keep input constant, a neuron can either increase or decrease its receptivity to that input, it could have more receptors or less receptors, and that can change the level of input that it prefers. It can also adjust extrinsic parameters. So forming new synapses, adjusting synaptic weights, and there's global synaptic scaling as well. So there's this relatively recently discovered phenomenon where a neuron can scale the total amount of inputs. It's basically changing all of its synapses at once. And in this view, this is a little bit of speculation on my part, but if we're thinking of neurons as agents, then what is spiking? I argue that it might be a resource management method first. And in that sense, the way to dissipate energy quickly when a neuron has too much input. And then that function, that kind of survival function could be co-opted for cooperation or for signaling. So how does this work? If you imagine that one neuron is overloaded and it's just going to dissipate a bunch of energy, well, that energy is basically wasted. But if some other neuron is connected to it, then it can use that waste to keep itself within its own bounds. So it sets up the possibility for these kind of resource sharing functions. At the same time, you could think of that as carrying a signaling function. If you know that your neighbor has hit hard times and is kind of selling all of their furniture, then that might be a signal that there's some economic problem coming down the line. So it implicitly functions as a signal that some perturbation has occurred that your neighbor was not prepared for. The hypothesis here is that from those low level processes, those neural level regulation processes, you get global behavioral control of the entire organism. So what is the argument here? First proposition is that homeostatic mechanisms are going to regulate that flow of energy through a network. They're trying to distribute resources in the most efficient way so that all of the neurons can thrive. You can't regulate the flow through a network very well unless you control the flow into a network. So if you have a constant input stream, then you can learn how to distribute that in a nice pattern of flow that works for the network. But if the input stream is constantly changing, different neurons are getting perturbed at different times, then it creates a challenge for that problem. I also want to point out that the flow into the network is driven both by changes in the environment and by the organism itself. So if an organism is in an environment, if they stay still, the environment can change and that might create different inputs to the network. At the same time, whatever the network is doing is going to potentially cause some motor movement, some action. And so the organism might turn and look at something else and that's also going to change the flow into the network. And usually both of these things, some combination of them are happening simultaneously. So how do you solve this problem? Well, you can't get a homeostatic equilibrium in this network unless you stabilize that higher order relationship. So you have to have some kind of regular pattern of organism environment relations in order to have a regulated flow of input to the network. So that's why we think that if you just put homeostatic mechanisms into individual neurons, then this global property of the collective being kind of adaptive organism is going to emerge. So now I'll tell you about how I tried to formalize these ideas. And I'll direct you first to the top left. These are the possible states of the neurons. So each neuron that I modeled had some activation level that's represented by these purple bars. And Daniel, can you see my cursor here? Or do I need to put on a pointer? I think maybe it's there. Okay, cool. So each neuron has some activation level. And we assume that there's a floor, a minimum activation that it can have. And there's also a target activity level. So this is just your homeostatic set point, but I want to point out that that target is going to vary. So we don't put that in on our own. We initialize it, but we let the neurons decide what target they want. And that target also has a minimum value. The reason for that is that we assume that neurons need at least some level of input. So you can't have a target that is zero where a neuron prefers to not get any input ever. It needs to prefer to at least get some input, but it might prefer to get a lot of input or it might prefer to get a little. And then there's a firing threshold. And when the neuron goes over that firing threshold, it's going to spike. So based on these possible states that neurons can find themselves in, we've got a set of rules. And that's this flow chart that I just had at the bottom here. The idea is that if you are under your target, so in the top left, then the way to get closer to your target is either to recruit more input or to try to lower your target if it's not at the floor. So you're kind of trying to converge those two parameters so that you can get to a stable equilibrium. If your activation level is over the target, but under the threshold, then you can do the reverse. You can try to decrease the amount of input you get, or you can try to raise your target. And actually, they're doing both here. Then you get the case where the neuron is over the target, and they're over the spiking threshold. So what they're going to do is dissipate all of the stored energy and end up with just the remainder. So in this case, this neuron was just slightly over its threshold. And so what's going to be left after spiking is just this red section at the top. It'll end up being below its target. That creates an interesting kind of heavy end mechanism where if you spike and you end up below your target, then you actually need to recruit more input to get back to your target. So you might increase the weights with nodes that just caused you to spike. So out of purely homeostatic mechanisms, we get these kind of learning properties that emerge. So here's the basic architecture of the model. And in the gray box in the middle, I'm showing just the network. And then on the sides, the kind of agent environment system that we put it into. So first, I'll talk about the network. There's some sensory layer consisting of a number of nodes. And what these nodes are going to do is spike when there's a stimulus at some egocentric location. So for example, this top node might be corresponding to your right eye. And if light hits that right eye, then this top node is going to spike. If it hits the left eye, maybe this other node spikes. So we actually have an array of those kind of representing something like a retina where there's a series of sensors that get triggered. And those nodes have purely feed forward connections to the homeostatic reservoir. These are sparse random connections. That means for us, probability of forming a connection is 10%. So we walk through every sensory node and give it a 10% chance of connecting with any node in the reservoir. Then the reservoir consists of a relatively larger number of nodes that implement that homeostatic mechanism that I just described. Just to clarify, the sensory and effector layer are not homeostatic. So this reservoir layer also has recurrent connections that are directed. They're not bidirectional. And again, we just use this sparse random connectivity. There's no spatial structure built into the network for now. So this neuron can connect to the neuron in the top right, just as well as the one that I have pictured right next to it. And then finally, we have the same process to form connections with some effector layer. And in all of the examples that I show here, the effector layer is two nodes. And what each of those nodes are going to do is normalize their input so that it is going to take on some value between zero and one for each of these two nodes. And then that's going to drive motor output. So you might imagine that the top node makes you turn left, the bottom node makes you turn right. If you're fully activating the top node and not activating the bottom node, then you just completely turn left. If they have equal amounts of activation, then you stay still. It's balanced. So that's the general idea. And then we put it in some environment where there's a stimulus that's going to perturb that sensory layer. That stimulus can move around and does in the first couple of cases that I'll show. And then motor output is going to change the position or orientation of the network. So again, I want to point out that this has that kind of action perception feedback loop where the input to the network could change either because the stimulus changes or because the motor output causes the network to orient itself differently or some combination of the two of those. So I'll show you four case studies of what we did with this. And the first thing I'll show is the original paper that I put this model in, which was in a language processing context. And then I'll show three more from this most recent preprint where we have it in an action perception context. So on the language processing stuff, some background. The first thing is that a lot of language researchers, especially in the last decade or so have started to think that the brain is a prediction machine. Why do we think that? Well, AI models do very well being trained on next word prediction. This is essentially the basis of a lot of modern large language models like chat GPT, although it gets a little more complicated in those is that they're predicting upcoming signals and getting some error signal that leads to learning. And in some cases, the activity those networks looks a lot like what humans are doing. Of course, there are a ton more examples that I could cite of evidence that people point to for why it looks like we might be predicting. This is just kind of one larger point. But there are at least two ways that you might understand what that means. On the left, one is that brains might be predictive coding machines. And that means that actually the signals that are propagating through the brain are prediction errors. And what that prediction what the hypothesis I should say of this is that you'll get a reduction in signal when you have predictable or expected inputs, and you get a spike in signal when you have unpredictable inputs. But there's another view that's related, which is sometimes described as predictive processing, just maybe a weaker version of this, which is that you can get that prediction like behavior, you can get those same behavioral signatures without specifically relying on explicit prediction mechanisms. So you don't necessarily need to have nodes that make predictions or that propagate prediction errors. So what I did to get at this issue was take this network that I just described, except in this case, it had no output layer. So it was just a passive response device. And I used 100 nodes. The input layer was five nodes. And those nodes were one hot encodings of five distinct words that I'll show you in a second. And those were is going to generate a sequence of inputs that come from a probabilistic grammar. So what does that mean? Here are our five possible inputs, man walks dog, and bites, and space. So those are five possible inputs. And we create some probability of transitions. So a sentence might start with man, the most likely flow is that it'll say man walks dog, and it always ends with a space. And then you come back to the beginning. This is going to generate a series of probabilities of different observations. The most probable are either man walks dog at the top here, or dog bites man. And the least probable are man bites dog, or dog walks man, and you get multiple other combinations. So what I want to point out here is that this gives you a simple system, simple kind of environment. You could think of it that way, a linguistic environment that has statistical structure to it, but it's not fully deterministic, right? There's some randomness and probability built in. So what I'm showing on the left are the spike patterns for the reservoir network at the end of exposing this network to about a thousand of these sentences. And so each column here contains all of the nodes in the network, and each row is a time step corresponding to an input position. And if that node was spiking, we have it in yellow, and if it was silent at that time, we have it in blue. So you see that we're getting a series of inputs, man walks dog, dog bites man, man walks dog. And then what we do is present it with a kind of prompt, one input to start a sentence, like man, and then we don't give it any further input. And because of the properties of reservoir networks and this dynamic memory, or what you call fading memory, the activity of the network keeps going, at least for a few time steps. So it's going to generate some series of outputs, even though it's not getting any input. And what we wanted to ask is, do those generated outputs resemble reasonable patterns? Does it look like the network is kind of generating predictions of the next word? One way to get at this is looking at the correlations between spike patterns. So what we have here is time on the x-axis and the y-axis, and we're showing the correlation between all of these points. And kind of on the bottom right here, these are the places where there was no input. And so this section is actually the bounded section here by the dotted line is what we're interested in how similar were these outputs when there was no input to previous outputs when there was input? And just to point out some kind of clearer cases up at the top here, I'll look at this center column where it got the word man. And if you go back in time and you find there's a high correlation, what was it correlated with? That was a previous instance where it saw man. So it seems like the network generates something that looks like a pseudo population code and I'll tell you in a second why we call them pseudo, but it has a kind of pattern that represents or corresponds to each word. So it seems. So to make this a little more clear, I'll just show you like what was the highest correlation, which we'll call the prediction of the network. And I have it in scare quotes because we're using that loosely for now. So if we give it an input of man and we cut off the network, it predicts or it creates some output that is most highly correlated with previous instances of the word walks. And of course, this is the most likely next input. If we give it man walks and then cut off the input, the output resembles dog. And this goes down the line. So in each case, when you cut off the input, the output looks mostly like what you'd expect if you had continued the input. So it's kind of like it's dreaming the next set of inputs. And this might look like it's predicting or doing sentence completion. So we looked for some behavioral signatures of what's often taken as evidence of predictive coding. And the other thing that I mentioned was an increase of activity in the network for unlikely inputs. So what we have an x-axis on the left here is the transitional probability. So how likely was that switch? And what you can see is that as the transitional probability goes up, the mean activity goes down. So that means that the more surprising the input, the higher the activity of the network. So this looks like it might be kind of having a it looks like it's error coding, right? So it's got a large error signal when it gets a surprising input. Another way that we got at this was giving it a bunch of training trials with the standard transitional probability matrix. And so we are showing the last 100 iterations of training where it's going subject, verb, object, space, subject, verb, object, blah, blah, blah. And it's kind of nicely oscillating in some level of activity. And then we suddenly switch the order. Instead of giving it subject, verb, object, we give it subject, object, verb. And you see huge spike in activity. So it doesn't like to get those things that are out of order. So it seems like not only is it kind of classifying the possible inputs, it's also learning something about the temporal structure in the transitional probability matrix. When we look at the correlations, another thing I just quickly want to point out is that the activity can differentiate the same input at two different positions. So you can get man as a subject of a sentence or as an object of a sentence. And you see different response patterns in those cases. So that's not built into the network. It gets the same input. So it seems like it knows something about the position. And it's also showing similarities between different input words that are in the same class. So man and dog are the nouns, walks and bytes are the verbs. And we see correlations between these. So we'll see that, for example, bytes has a 0.28 correlation with walks. And that doesn't seem very high. But if you look at the rest of the correlations, they're near 0, except for the ones that we're interested in here. So that's the language stuff that kind of spurred this. And then we wanted to ask if this model is going to generalize to other tasks. So in that language example, models just passively receiving inputs. What's going to happen if we give it an output layer? Where whatever goes on in the network can also potentially change the next input. And we looked at three agent environment systems just to see how well does this generalize to a variety of cases. And the first thing that I tried was an agent that is fixed at a center point and they just rotate. They can't move, just rotate. And there's a stimulus that's going to rotate around the agent. And sometimes it's going to change directions. We give this agent two eyes that are 30 degrees apart. And those eyes have an array of sensors that correspond to different egocentric angles. The idea is that this is, again, like a retina. So there's one sensor that's going to be straight ahead. If something's right in front of you and if something's off to the side, then it hits you at a different angle. Different sensors going to respond to that. We also gave it two effectors, which we just arbitrarily dub one as the left turn and one as the right turn. The idea is that these are like the motor system and that there's kind of random connectivity to, for example, your left or your right leg. So I'll show you a quick video of what happens in the top left. Here's our agent is this pink circle. This is intended to look kind of like a top down view of a head. And these red and blue dots are the two eyes. And then there's this green stimulus that you'll see move around in the circle. On the bottom, we have the network and all the nodes are blue when they're silent, they're going to turn yellow for spikes. And we can also show the output of the left and right turn effectors. On the top center are the outputs of the sensors. So you'll see as the stimulus moves, the sensors activation is going to move across the array of sensors. And then top right, we've got the average aggregate parameters of the nodes in the reservoir. So what is their mean activation is the first column. Then how, what does there mean error? So how far on average are they from their targets? And then what is the average target? And then finally on the bottom right, we're showing the distribution of weights and how those change. That turns out not much interesting happens in this case. It kind of stays as a normal distribution. So here's a little video and I might speed it up. And you can see that as the stimulus moves around the first time, the agent kind of starts to move with it, but it loses track. And same thing happens on the second run. And so we'll give it a third chance. The stimulus changes direction. And now it suddenly kind of locks on and is traveling with it. And what you can see is that it's kind of slowing the rate of change across the sensors. So you'll see that as stimulus changes direction, it locks on again. And it's making the flow of input change more gradually. What I want to point out is that this is, this is not an automatic consequence of the properties of this network. So we don't tell it to track the stimulus. It doesn't have any kind of rewards going on. There's no supervision. You can do whatever it wants. And in order to track the stimulus, it needs to find the right ratio of outputs. So if these, the ratio of left and right effectors is going to determine how fast it turns. And in order to track, it needs to turn at the same speed as the stimulus. That's essentially what it does is discover that learning pattern. And it's not perfect, but I'll try to show an example where it, here's a nice example where it quickly changes direction with the stimulus. So it bounces here and it just flips and follows it. And you can see that when it's kind of stably tracking, there seems to be a sort of limit cycle kind of flipping between two activation patterns in this one case, but we see a few different patterns. So let's analyze some of the outputs here. On the top panel, what we're showing on the Y axis is the angle in degrees of the stimulus, which is in black, and the agent, which is in red, and then time is on the X axis. And what you can see from this is that the red line is essentially tracking the black line, following the black line. It's not always perfect. So here's an example at the beginning where it loses track and then connects back with the stimulus. And I've kind of lined this up with the timing of the proportion of the network that's spiking at each time point. And so what we're looking for these kind of little perturbations, these jumps in activity, what are causing those jumps in activity? What we can see from this is that they occur either when the stimulus changes directions. So this kind of first perturbation corresponds to a change of direction, or when the agent loses track of the stimulus and then finds it again. So this next one is a case where it had lost the stimulus. And then when it comes back around, you get this burst of activity, and it starts to track again. We can also look at the auto correlation of the network activity over time, which is again mapped onto this time. So we've got time going to the right and also downward. So this is symmetrical. And what you can see is that every time you get one of these spikes, the network shifts into a new regime. So it's got some pattern of activity that's highly auto correlated for a short period of time. And then it switches to a new pattern of activity. What's interesting about this is that off the diagonal, you don't see high correlations, right? So we actually see a lot of what look like zero or negative correlations. And the reason this is interesting is because the network is actually repeating, or the agent, I should say, is repeating several of the same behaviors. So during this 7200 time steps, it's turning right and turning left multiple times at the same speed. And yet it seems to be accomplishing that with a totally different pattern of activity each time it does it. And it finds that new pattern of activity very rapidly. So it doesn't seem to have kind of recurring encodings that correspond to, you know, this is the subnetwork that turns left. This is the subnetwork that turns left. It finds a totally new set of collaborators within the network to accomplish its goals on the fly. All right. So then, you know, I was inspired by some recent work that you acted in. For instance, folks have probably seen where Kagan at all grew a culture of cortical cells on a micro electrode array. And they hooked that up to the feedback loop of a game of pong. And they found that it within I think five minutes, they said it starts to improve its performance. Now, it didn't get very good. The performance was still just slightly above chance. But it's cool that it's kind of self organizing. And they give they give a free energy reading of these results. But there's no mechanism in this case given specifically. So, you know, what is it about that network that is allowing it to minimize free energy? So I wondered if my network would do the same thing. And I'm really curious to hear from from anyone who's here at the end about, you know, whether you think this is a potential counter example or just a different mechanism for for accomplishing the same goals? Or is my network also minimizing free energy potentially? So I'm not really sure yet. There's more analysis that we need to do. But same kind of idea. This slide is a mess. But I just want to point out some features of it. We gave it a few more nodes, 500 instead of 200. The agent in this case is the paddle. The arena is 500 by 1000 pixel space. And the stimulus is the pong ball. And in this case, the input is encoding the relative angle of the ball from the center of the paddle. So it's very similar to the last case. Except here, instead of, you know, rotating left or right, the output layer is moving the paddle up or down. So what happens when we do this? So we start the ball in the center. If the paddle misses the ball, then it's going to reset at a random position over here. And I'll show you kind of cool hits it on the first try, but maybe not that hard because it came right in the middle. And it's got a few hits on the third try. There's some misses for sure, but you can see that it's going for it, at least, which is interesting. And yeah, so just a few examples of what it does. I guess I've shown some bad examples where it looks pretty bad at pong. But when I show you the summary stats in a second, maybe that'll change your mind about that. So before I get to the performance, similar idea to the plot that you saw last time is showing time on the x-axis. And in this case, instead of the angle, we have the y position of the ball or the center of the paddle. And I've marked red columns where there's a miss and green columns where there's a hit. I just want to point out that the ball and paddle can be at the same y position here without contacting because they might be at different x positions. So here's a case where they overlap and it's not a hit or a miss because the ball is just not close enough yet. So again, we see this kind of spiking of activity at certain time points. What are those time points? So they seem to correspond to misses. Cases where the paddle missed the ball and it reset, and then you get this spike of activity and potentially a reorganization. In this case, we don't see high auto correlation even along the diagonal. So it seems that the network activity is just kind of chaotic throughout. And yet it will show in a second, it seems to have decent performance. So here's the performance over 500 runs in these first cases. And I'll walk you through these. First, I want to point out this kind of medium gray. This is our baseline condition, just everything that I just described. On average, when there's an opportunity to either hit or miss, how often does it hit? So it's the proportion of hits. And we find that it was about 60% on average, some variation down to about 40%. In this case, chance performance is 20% because the paddle is one fifth the height of the space. So if the paddle just does nothing, it has a 20% chance of hitting it. So even at its worst, the network is more than twice as good as chance. And we wanted to see, does it learn? So does performance improve over time? So to do that, we looked at just the first 50 opportunities for hit or miss and the last 50. And what you can see is that there's not much change. So it seems like it's at its peak performance right away. And maybe we're missing some learning that happens in the time steps before even the first hit. So it could be just kind of rapidly picking up on something. So maybe there is learning and the proportion of hits is too discreet of a measure to get at that. So one other way to look at this is the blue column. So what we did was just turn off the homeostatic updating. So there's no learning happening in the network. What's surprising here is that the performance is already well above chance. So at its worst, it's near chance, but on average, it's twice as good as chance. So why would it be doing all right if there's no learning happening? What we hypothesized was that this is just kind of a natural consequence of egocentric encoding in this case. The reason for that is that if the paddle hits the ball, then it's going to create a predictable stream of input. If it misses, it gets a maximally unpredictable input. But also due to the laws of optics, the angle of the ball to the paddle changes more rapidly as the ball is passing the paddle. So when it's coming close to the paddle and there's about to be a miss, that's when you get the fastest change in input just due to trigonometry. And given a rapid change of the input, this creates a chaotic signal through the network, which means that it's going to be more likely to generate a movement when it's about to miss. So that seems to just automatically help it by chance to hit the ball, even if sometimes it's moving in the wrong direction. So to test whether that egocentric encoding really is important, we tried it again with an allocentric encoding. So instead of encoding the angle of the ball to the center of the paddle, we just encoded the objective Y position of the ball. And here, performance is right at chance. So it really needs that egocentric encoding. And one thing that I think is interesting is that that Kagan at all paper didn't use an egocentric encoding, they used an allocentric one. So I'm sure it's a lot more work to grow a culture of cortical cells, but I'd be really curious to see if it does much better if they give it an allocentric encoding, or sorry, an egocentric one. And so the final case study, I'll talk about similar ideas that Kagan at all study. This came from Masamori at all 2015. They also took a culture of cortical cells, put it on a microelectrode array, and used it to control a mobile robot. And what they were wondering is, does that robot learn to avoid walls? So they put it in a box of some size and just let it run around. And what this is showing is the first 15 minutes, you can see that it's kind of clustered to the right, and it's hitting the wall a bunch of times. And last 45 minutes, maybe it's more in the center of the space, it's not hitting the wall so much. So maybe this network will do the same kind of thing in that case. We went back to 200 nodes. In this case, our agent is like a brain bird vehicle, so we simulate two wheels on each side of the agent. And the movement is driven by the relative speed. So if, you know, wheel one is the right wheel is much faster, then it's just going to turn to the left, but stay still. And if both wheels are equally active, it's going to move straight forward. To our sensory system in this case is a wall detection encoder. So we have two sensors that are just going to send a ray straight out. They're going to find the distance to the nearest wall. And their input activity is going to be an inverse relationship with that distance. So when it's really close to the wall, high input, when it's far away, low input, I want to point out that input was always non-zero here. So we set the minimum activation to be the diagonal of the space. And so the agent actually can't get right into the corner of the space. So it's always going to have at least a little bit of input to both sensors. So here's our agent again, now it's a, you know, a little version of that first case you saw, and I'll let them run around for a second. I think I might have given you a bad video. Here we go. It's not the video I intended to put in here where it's tracing the line. This might be an earlier version, but you could see that eventually tries to find a circle. There we go. So it finds a little circle of movement. And I apologize that I loaded in the wrong thing here. But when we, in the final version of the model that we used, same setup and basically takes a couple hundred time steps and this agent essentially finds a little circle that it can move in in the middle of the space. And that has some limit cycle where the red and blue sensors are going up and down as it's turning around. And it needs to find a ratio of left and right output that is going to create a radius of a circle that keeps it within the walls. And so I'll jump to plots of the end results that are a little bit better. So here's a representative run. The arrows are tracing the path that that agent made. Again, we've got the proportion of the network that's spiked and then the auto correlation. This time with time running to the right and up. And I put little black lines down here for every time the agent hit the wall. So what this shows is that at the beginning of the run, it hits the wall a bunch of times. And then it, you know, an activity is kind of chaotic. And then it finds this nice circle in the middle of the space and it totally stabilizes activity. So in this case, we don't see chaotic activity. You just see this stable pattern over time. The reason for that is that the environment is not changing in this case. So the agent, now the inputs of the network is changing, which is interesting, but the agent has learned to compensate for that change in input that is purely due to its own activity. So it can fully stabilize. So given that it fully stabilized, we might want to know, you know, what's going to happen if it can't find a completely stable pattern of activity. So we tried putting in some sensory noise. So to the inputs, rather than just getting a, you know, a spike for that corresponds to the distance to the wall, we just add, you know, sampled from a uniform distribution to add some noise to that input signal. What you find here is largely the same thing, but it, you know, gets disrupted a little bit and it starts to change the size of the circle that it's making. And it ends up finding a smaller circle that's, you know, more easily maintained than this original circle, given the noise. And you see that, you know, even though everything else is stable in the network, you sometimes get these sudden switches to a new pattern of activity that's doing the same thing for the agent. Then we tried something that was done in kind of similar paper. I apologize that I forget the name of the author, but we have it cited in our preprint, which is to see how the network responds to a perturbation. So what we did here was just suddenly invert the visual field. So we switch the left and right eyes, and we did this at time 1000. And what happens in this case is the network first finds a nice circle, and it does that for some period of time. And then we do the visual field inversion and it sends the agent off track, it starts crashing into the wall again down here, and then it just finds a new circle. So in just, you know, under 500 time steps, it fully adapts to that inversion of the visual field. And then finally we said what happens if you, if you don't have learning. So in the palm case, it seemed to work pretty well regardless of whether or not learning was on. In this case, you don't get that same result. So if there's no learning, because the input is always non zero, the network gets saturated. So activity goes to a maximum, all nodes are spiking constantly. The result of that is that both output nodes are fully active. And that leads to straight ahead movement. So what you get is an agent that just goes straight and bounces off walls. You'll see that it never stops hitting the walls. Activity is just peaked right away. And we can't even give an auto correlation matrix here because there's no variability in the signal. So what are our conclusions from all of this work? The first is that given this homeostatic network, and I actually kind of have vacillated between using allostatic and homeostatic, I guess I forgot to change this here, but I want to point out that allostatic just means homeostasis with a dynamic set point. But homeostasis doesn't really necessitate that the set point is static. So I think it's fine to just stick with homeostatic. Given this homeostatic network, we get prediction like behavior from a simple probabilistic grammar, get spontaneous object tracking, plays pong substantially above chance, and it naturally avoids walls. All of these adaptive behaviors or apparently adaptive behaviors continue uninterrupted in some of these cases, even though the activity in the network is drifting around. So we don't think that there are stable representations in here because the network doesn't need to use representations. It doesn't need to read out its own activity at any point. It just needs to attune its activity to the statistics of that agent environment system. And if we look over just short time slices, you do see what look like population codes, at least in the language processing model. So this might give us a way of understanding why do we often find what look like representations or encodings in neuroscience studies, and why maybe it took so long to notice representational drift is that we're typically just looking over one or two sessions of an individual. And in that time span, you might not see these changes. So why does this work? I think that the network is not predicting in this case. But what it's doing is adapting, but it's adapting in a kind of counterfactual or anticipatory way. So what I mean by that is that when a node gets a perturbation, if they're above or below their target, what they're going to do is say, how could I change my setup so that that would have gone differently? If that perturbation were to play back, how would I change my parameters and my synapses in order to approach my target? So it does that. And because these things repeat, you get what it looks like strong anticipation. So the network is kind of generating a flow of activity by itself that compensates for upcoming input, compensates or complements, depending on the case potentially. So this, the activity of the network, what we're arguing is not a model of the world, but it's kind of a shield for the world. But it's a kind of temporarily changing shield. It's a way to protect itself that it's way to protect itself for what's coming next. And what we wrote in that original paper to describe this is that when an individual is successfully adapted based on the past, and those same conditions occur again, it will look like the agent predicted those conditions to occur. It might not be that they are literally predicting those conditions, but just that they're set up for that particular trajectory. And what we think is that just by trial and error, kind of changing parameters when there is a lack of homeostasis, when they're out of homeostasis, the network quickly generates what have been called talents. So this is a term that comes from Anderson in his neural reuse work. And talent stands for transiently active local neuronal subsystems. And I think I switched the lower case of the A and the O. But yeah, the idea is that these are temporarily active neural synergies or partnerships that form on the fly self-organized for their utility in a particular context in an immediate environment. And those do not have to be read as representational. Patterns that we get there, they could be fixed point attractors, they could be limit cycles, they might even be some kind of strange attractor if these networks can learn those patterns. And they're essentially just flows of activity that match the flow of the environment as it interacts with the agent. So the flow of input is driven both by environment and agent. That creates some pattern over time, and the network generates a complementary pattern. So the two questions I want to end on are first, the main question of this paper is, does this give us a way to understand what resonance is? Is this a step towards a mechanism or what's the best way to think about this? And I'm really interested to hear from you guys on how you think this might fit or not fit with free energy principle and active inference. So is this system a free energy minimizing system? How would we find out if it is or is not? If it's not a free energy minimizing system, what does that say for FEP, for other approaches? So I'd love to hear your feedback. And thanks so much to Daniel for having me here and looking forward to the discussion. That's a wrap. Awesome. Thank you. All right, just getting back to very cool. Thanks for the awesome presentation. Well, those who are watching live, please feel free to write a question in the chat. And I've written down some as well. Let's just start with how did you get to this research question? Were you studying ecological psychology and got interested in modeling? Or were you already involved in this kind of modeling and then wanted to come to explore resonance? Yeah, great question. So I mean, I'll tell you a little bit about my path into kagasai in general, which is I started as a philosophy and psych undergrad in a very kind of traditional, cognitivist place. And I was sort of frustrated with both sides of those views. I thought that cognitivism had a lot of problems. So I started to learn about embodied cognition. And that, you know, I got into Andy Clark's work that sort of led me into his interest in the free energy principle and active inference. But in the background of all these things and kind of the motivation, I think, for a lot of people, building embodied or 40 approaches and FEP approaches is something like the symbol grounding problem is that if we have this problem with representations, what do we do next? So that's something that's been in the background of all my work, sometimes more or less explicitly. And then, you know, in the case of this model, we were working on that language paper that I showed where we were arguing that you can think of prediction in many cases as just a kind of pattern completion, that there's some kind of momentum of a system that flows forward. And if that momentum has become entrained to something in the world, then it's going to look like a prediction. And so it just kind of based on intuition started to make these homeostatic mechanisms and see if that would do the kind of entrainment that we expected. And I was sort of shocked to see that it worked right out of the box. I mean, it's kind of crazy that random sparse network, no supervision seems to do all of these tasks language action perception, like right away. So that was kind of amazing. Awesome. So I'll give a reflection in a question, and then we'll see some questions from chat. But I thought it was a very provocative framing where you right at the beginning said the neuron is the agents, and it's trying to survive in a harsh intracranial environment. And sometimes the intracranial environment is framed, well, it's dark, it's warm, it's chill, things are being brought to you. And if it loses oxygen for even a short period of time, the energetic rate is such that there's damage, it requires nutrition, sugar, ketones, whatever it may be, it is actually a survival situation for those neurons. And so by framing at the agent level of the neuron, a homeostatic impulse, we then see functionality arise from a group. It's like individual ants in the colony are taking behaviors to normalize their interaction rate. And then the colony might explore a space. And if it's a big space, they'll spread out to explore it. If it's a small space, they'll slow down to explore less. And so that's a great collective behavior or multi-scale systems approach, which is rather to kind of put our thumb on the scale on the collective behavior, and then say, well, what would the neurons do? We can actually embody that lower level agent as the one that is the action perception cognition entity, and then ask, well, what does happen? And as you just put it out, it's a really exciting emergent outcome that these kinds of brain level homeostatic impulses arise from the neuron level homeostatic imperative. So that's just very cool. And it gives us something to work or at least build off of jump off of when we're having these discussions about like, what do neurons know? What do groups of neurons know? What do organisms know that neurons don't? And that is in some ways so core to the representation question. Yeah, yeah. Really interesting points. And I'm glad you think this is cool. And I love that you bring up the kind of collective intelligence framing. And that's in the background here for sure. And another sort of stream of my work is on cultural evolution and kind of emergent processes in cultural systems. That's something that I'm just prone to that way of thinking. And I think you're right that it often looks like the brain might be this very nice kind of incubator for neurons where it's set up for their survival. But it looks like that because they've honed these mechanisms of collective intelligence that something about the developmental processes and the regulatory processes has created an environment where they all can thrive and something is going to come out of that. What I think is interesting on an evolutionary level is how do you get that kind of transition into cooperative behavior? What are the constraints that lead to that? So with a bunch of great co-authors, I have another preprint right now on collective intelligence where we explore some of these questions a little bit. And we look at, for example, a transition from single cell states to multicellular states. And there are some species that can sort of suddenly do this switch. Others are obviously kind of obligatorily multicellular or always single cellular and sometimes they can transition. And so there's some pressure in the environment that triggers that transition. And eventually you might get this kind of path dependence where they actually end up relying on that kind of collective structure. And that gets sort of entrenched. So I think the brain is one of those cases. And it's really interesting to me to explore in what ways is the brain different from human cultures or groups of organisms and what constraints keep us from being a fully kind of cooperative hive mind, which I don't think we necessarily want. But it might be interesting still to know what would shift us into those different basins. Cool. With a multicellularity and the evolutionary origins as well as the developmental origins of the nervous system are in the ectoderm and the closure of the neural tube. And so it's like you could imagine an epithelial cell that just responds electrochemically when it's touched. And then like you pointed out with the neighbor selling their furniture, it's like if this cell on the epithelium responds, well, that can come to be associated with response from an interior cell without necessarily taking on this baggage or the symbol grounding problem, all these questions of representation. So very cool. I'll go to a question in the chat. Anyone else can add more questions. So Djingo writes question, wonder how this messes with functionalist views. Functionalism would say that these systems are still functionally doing the same things. And this is just another way of framing some other function. But then it makes you wonder whether functionalists are cheating because the function is only a function from our perspective, but not from the perspective of the system slash its parts. Yeah, yeah, really interesting point. Yeah, so you definitely can think of these as, you know, the activity of the network is performing a function for the collective, which is ultimately maintaining survival in this case. But when we think of higher order functions, like turning left or turning right, we might read that those functions are in there, but it's hard for me to imagine how you could say that they're, you know, that that activity has a specific function, if you can't find a thing to attach that function to. So if the function comes out of this kind of shifting landscape, then what, you know, what's the functional, what's the part that has that function? Can you have a function without the things that perform the function? I don't know. So interesting question. I'll also think more about that. Cool. Well, let's bring it to active inference and think about at the neuron level and at the brain level, we'll just call it the reservoir level. Think about what are some similarities or contrasts with active inference? How would we know if the system is normatively described by the free energy principle? So right off the bat, you described at the neuron level, a target set point. And that is very resonant with the active inference framing, which is that actions are selected, not based upon the maximization of a reward or utility function, which is what we might see in reward or reinforcement-based learning, but rather in active inference, actions are selected to reduce the divergence between observations and preferences. And so we can talk about those preferences as what the agent likes, but also what they expect, what we expect slash prefer, like the body prefers to be at homeostatic temperature. It also expects to find itself there. It's not surprised to find itself at homeostasis. It is surprised to find itself out of homeostasis. It takes actions to bound surprise. And then the unified imperative that bounds surprise statistically for a generative model is the free energy. So right off the bat, what the agents are doing in terms of reducing their surprise by navigating to a target, rather than proposing a reward or utility map that is then climbed, but reducing surprise and reducing divergence between incoming observations and an expectation slash preference is a reward free pattern that is just right off the bat, one of the strongest and most important pieces that make it aligned with active inference. Yep. Yeah, totally agree. So this is why I'm really excited to be here and talk with you guys, because I think there are a lot of ways in which it's really close to active inference and then potentially ways in which it's totally divergent, you know, little subtle differences that might have big theoretical consequences for how we interpret it. And so on the one hand, as you say, there's this kind of target set point which you could cast as a type of prediction. Couple of differences I want to point out. You know, I don't have much experience with active inference modeling, so I'd love to connect with someone who does, but at least in the language processing literature when there are predictive coding models in particular, you have to build in a prediction mechanism. So the, you know, there's a flow of top down predictions that are changing the lower level predictions and you're propagating prediction errors. So the question is how much of that framing can we expand? And you know, there might be a sense in which that kind of abstract framing is a useful modeling approach and maybe not an ontologically real thing that's happening here and that could be okay. But there might be other ways in which it just doesn't fit at all. So I think it's possible that, you know, one thing to point out is that the predictions in this case don't come from top down signals, they're emerging from each node, you know, the target levels. And the way that signals are propagated is not like computationally optimal, unnecessarily, right? So you might have potentially neuron level predictions that don't get propagated in a way that they could potentially form higher level models of the system. So yeah, I think those are some of the divergence points that I'd like to see if, you know, I'm wondering in particular, we were to approach this system like an organism that we, you know, are studying a species in the lab. What kind of analyses would we do to find out if it is or is not minimizing free energy? And presumably there's some set of systems we could say these are doing that, others that are not. And in this case, I could imagine scenarios in which the well of energy that creates a kind of thriving environment for these neurons is just a chaotic signal of some kind that is, doesn't have, you know, a non-equilibrium steady state, for example, it might have shifting Markov blankets. Now we can say there's a sensory layer, there's an active layer, there's internal states. But if all we have is the statistical structure of the network, then it might look like for any given subset that there's a changing Markov blanket. And, you know, maybe that introduces problems or maybe not. Cool. We'll definitely the input since state internal cognition and output action layer is something that neural networks of certain forms have active inference agents and really just classically cybernetic agents. So that kind of input cognition output model and embedding action perception loops around that kind of an agent is something that really connects all of these models. And then it's a super interesting question. Okay, view from the outside, let's test different models. And it's a point that's been brought up and is very fascinating to think about, which is that in some ways, active inference never fails as an analysis framework, like a linear regression never fails as an analysis framework. You could have any two variables, any scatter plot, you'll always be able to use some squares to fit a linear regression with a smooth optimization. You know, your computer program is not going to complain. And similarly, we can always use active inference to make a generative model. And that generative model will always be able to minimize its free energy. And the model won't complain. Now, is it an adequate model? Is it the best model? Is it a profitable business? Is it an adaptive organism? I mean, no, to any of them. So active inference will never complain by being used as a view from the outside. But that doesn't mean that it's necessarily the most computationally effective or ontologically supported view from the inside, actual mechanism. And I think we see it right here. There might be a plurality of mechanisms that lead to resonance. And the fact that you've shown one is like poking one hole in the veil. But that doesn't mean that's the only mechanism that leads to resonance, because we know that there's a plurality of cognitive mechanisms and a diversity of life forms that do it differently in different niches. Yep, yep. Yeah, I think that's exactly right. And you know, you raise the interesting point that you can always apply that modeling framework to any system. So then that leads to the question of what would distinguish different kinds of energy minimizing systems, like an energy well stability, just a rock rolling down a hill is minimizing thermodynamic free energy. But it's not a cognitive system, right? It's not forming a model of the environment clearly. Or maybe there might be people that disagree with that even. But I don't think that a rock is forming an internal model of the system. So that leads me to think that there should be some way to tell like, how much credence do we or how much concrete reality do we want to attribute to the idea of active inference in different cases? There might be some systems that we think are literally active inference systems and others that we think are just a model. And maybe the model itself can tell us something about that. Maybe we can say like, this system, if we model it using free energy principle, we find that it's doing a poor job. Or I don't know. Or maybe in this case, there might be mathematical properties of this system that just make it not fit. So if there isn't a non equilibrium steady state in this network, can can you even apply the framework? Great. Yeah, a lot of fun directions. I'll return to a question in the chat from Donarash. Hi, Ben. Really cool work. I'm wondering how and where your dynamic homeostatic points come from. Is this target externally set or emerges from the dynamics itself? Yep. Yeah. Great question. Because I really want to emphasize this point that the set points emerge from the dynamics. We do initialize them. We ended up using just a uniform initialization. Everything starts at the floor. I tried it other ways and it was robust to those changes. So it doesn't seem to care how you initialize it. And I think the reason that this is an important point is that if you build in the homeostatic set points, you kind of fall for a lot of the same problems that I was bringing up at the beginning related to symbol grounding. So this was a cybernetic school looked at homeostatic mechanisms and how they can give rise to adaptive behavior. But there's some criticisms coming from the ecological sphere that are pointing out that if you're building in those set points from the perspective of the researcher, then you still have a problem. So we wanted those set points to be emergent. And on the basis of how they're emergent, I'm still not really sure how to think about what is being controlled in this case. Because we might say the target's being controlled. But first you might say input's being controlled. But because the target can change, the input can change. You might say that spiking activity is being controlled. But that can vary as well while the neuron is still just as happy. So it might be spiking all the time and using those spikes to get back to its set point, it could set a really, you know, if one neuron is like always getting a flood of input, then it might try to, you know, lower all of its connections. And then it's sort of adapted for a really high level of input. And so it has a really high target. So I'm not even really sure what is being controlled at the individual level. And if you look at the individual nodes, they have different kind of emergent behavioral profiles. Some of them are never spiking because they are just riding activity near their targets. They never go above their threshold. Some of them are spiking on every iteration because they've actually come to rely on the spikes for getting back to their threshold. They depend on, you know, I'm going to get a flood of input, but I can dissipate most of that and end up right back in my target. And they end up using that repeatedly. So you get this range of different profiles, which makes it sort of mysterious to me. And I need to do some more thinking. But yeah, to just drive it home, they are emergent set points. Yes, very interesting. Well, the title of the paper, a potential mechanism for Gibsonian resonance. So could you speak a little bit? Who is Gibson? How has this topic of perception and action been approached from the Gibson lineages? What are some of the key features of a Gibsonian approach? And how would we know it when we see it? What are we looking for in terms of evidence? Or is this a philosophical debate? And the evidence is already on the table. We're just composing it differently. Is there some data set that people are waiting to see before they support or reject a Gibsonian perspective? Yeah, great, great question. So first, James and Eleanor Gibson were the founders of the Ecological School of Thought. And, you know, I'm not much of a historian on Gibson, so I might get some of the details wrong here. But I know that Gibson had a big focus on visual perception, at least early on, and did some work for the military, like figuring out how fighter pilots were, you know, how to set up a kind of, I don't know what I wanted to say, human computer interaction. But I guess it's not human computer interaction. But you know, control setup, how do we design that space given what kind of information is available to an individual? And so he came up with this idea of optic flow. And I think the big emphasis on optic flow is that rather than just thinking about static images, we're thinking about the change over time in input to the retina. So the optic flow is the pattern of change as you're moving around through the world. If you start from the idea of static inferences, or sorry, static images, that's kind of the classical, cognitivist approach, then you think, well, there's not enough information in the static image. Like if I give you just a picture, you can't really read out the, you know, all of the depth cues and other information from that picture. So the idea is that you need to add some background knowledge to that and make inferences about what's the actual structure. But what Gibson was trying to emphasize is that we're not dealing with static images as an organism. You're, you know, moving through an environment and certain information is revealed just, you know, through that relationship between perception and action that as I walk, I'm changing the pattern of optic flow. And there might be some structure to the change in optic flow, not just structure in the static image. So that leads to the, you know, Gibsonian approach and it's been expanded way beyond vision and kind of influenced the embodied cognition or 4e approach. And a lot of people in that camp are trying to emphasize how far you can get without any internal representations. So in addition to, you know, patterns of optic flow, there's work like the kind of passive walking machines that show that if you just have a system with the right kind of coupling of degrees of freedom, you can get like a wooden robot to just walk with no controller whatsoever. So their argument in a lot of cases is that we don't need internal representations to mediate interaction. You just need to couple with the environment in the right kind of way and discover what that coupling is. And so I think, you know, Gibson, what we're trying to emphasize in the paper and integrating it with this existing literature is that Gibson and the early ecological psychologists were trying to shift away the emphasis from internal states. So they purposely took that off the table. They said, let's just look at the relationship between the organism and the environment and look for some higher order stability in that system. So for that reason, resonance has stayed as a metaphor. He never spelled out, you know, what is the underlying system that accomplishes that goal. So that's what we're trying to do now is move into that space and figure out what is it about brains that makes them resonate. Very interesting. One active angle on that. So in active inference, we commonly use what's called the particular partition. And it's called that because it's one specific way of doing it. But also what it does is it carves out particles that are like the agent. And that includes their blanket states and their internal states. Now, there may be some agents whose behavior is fully described by the blanket. So I'm thinking of a guitar string. The resonance of the guitar string may be even simpler than like an input black box output. It's like the black box is empty. Everything about the input and output are happening on that Markov blanket layer. There's no need to appeal to some kind of internal state. So you don't even approach the cliff of representation. But let's say we had two guitar strings. And we find that one, we can fit a statistical model that explains all the variants in behavior with just the blanket states. And so we say this is basically non cognitive, surely a pan cognitivist pan computationalist could take a different angle. But whatever it is that it's doing, it's all on the blanket. It's all on the holograph. In contrast, we have the second guitar string. And it's like, wait a minute, there's there's something else happening like across trials. There's some residual that we're not modeling based upon observable input and output. So what would we do in the Bayesian causal modeling setting, we appeal to a latent cause of action that might look like planning or might look like anticipation. And so we can propose cognitive models with more and more complexity. And this work, it's kind of like digging for gold or I don't know if it's digging up for something, because it's actually showing that tasks or cognitive functions that previously may have been studied from a Turing machine, symbol, computation perspective, that might be one way to do it in certain systems, but it's not necessary. And so then the question is going to be, well, what are the limits of what interfaces can do, which is very related to the question, what are the limits of how we can model interfaces? Because whenever we have a shortcoming in our modeling, we appeal to another latent factor. So there might be some exciting things that today we do model with cognitive states, like memory. But as you brought up with the reservoir direction, memory can also be understood as being kind of carried in the dynamics and the interfaces of collectives not necessarily stored in a register like you would do it on a hard drive. Yep, yep. Yeah, great points. And so as you were saying that, I realized that I didn't fully answer every part of your previous question, which was about what kind of data would tell you about the difference between these two things. And there are at least two types that we might look at. And one is sort of proof of concept data, which is take get a system where you know the ground truth, and you can show that a task could be done without any internal representations. So that's like the passive walker machines. Or I think like this model is that we can look at this and know that there's no prediction mechanism happening, although maybe you can read homeostasis in that way. The other is I think you can give there, you know, not in every case is it really obvious, but sometimes you can find cases where the prediction or cognitivist approach versus the ecological approach are going to make very different predictions or at least discriminable predictions about behavior. So for one, you know, I mentioned the outfielder problem and I had a paper up there, which was, let me find the name of it really quick. I think it was Fink and Woo and Warren maybe. But essentially, they were trying to figure out what strategy do people actually use in this task. So what they did was simulate catching a fly ball in virtual reality. And the way that they got at it was perturbed the trajectory of the ball in virtual reality. So it did something that was physically impossible. But if you are canceling optical acceleration, then you'll still end up at the landing location. If you are predicting the trajectory, you would end up somewhere else potentially. So that's the kind of case where you might separate those two and have different predictions at a high level about what strategy an organism is using. And I also wanted to mention that, you know, I don't want to rule out the possibility that there is something like representation and sort of, you know, turning like serial processing that can happen in the brain. Because this work falls into that first kind of proof of concept location, we wanted to strip away as much as possible and say, how much kind of intelligence or adaptive behavior can you get with a, you know, purely random network that's kind of dumb, right? Now, I think there are a lot of ways in which it's probably not very realistic, like a human brain that, you know, we have sort of homogenous network structure in the brain, you're going to have different lots of different sub networks. Those sub networks can do a lot of things to the flow of information through the network. So you might have, for example, a flow that moves into a network and kind of gets, you know, partially sent into some recurring loop that's like stored in some sub network. And now that could serve as a representation that's available later to some other system. So I think, you know, you could get a system like this to do some of those things. And part of the point that I want to emphasize is that, you know, even if you see those things, you, you should be careful about taking it literally, because you can have a sort of self organized, Alan's like system that just shifts into these regimes. And those regimes might come computationally or algorithmically implement something like a turning machine. So the point is to emphasize that, you know, at the, at the base, what's physically happening isn't necessarily computation, although you can get those higher order things out of it. And I think that's a point that is, you know, cognitive scientists often get lost in talking about their models and what is the model and what is reality. And so it's sort of a recurring problem probably in every field where as you're writing a paper, you start to slip from one usage of a term to another and you start to take it very literally. So that's where we want to start is not give people any chance to try to read that sort of serial processing computational architecture into this. Cool. In the Pong example, I thought of it like a center fielder, like that's doing left, right. And then the classical lift angle and the ball trajectory is really on the, should I move forward or backwards? So you connect that wooden walker with the biomechanics to Pong playing left, right, and size of the ball forward and backwards. It's like, you're basically ready to be in the MLB at that point. If you can do that well enough. Yeah, right. Right. So yeah, that's the, that's the interesting question is what, what would we have to do to this model to get it to have, you know, near perfect performance. And in this case, we've gave it kind of an easy situation where, you know, if you're actually playing like the classic version of Pong does a lot of different things where the speed is going to increase every time that you hit it. And, you know, maybe the angles of deflection are changed in some way. And here we just had kind of elastic collisions and constant speed. And so it's pretty easy for the, you know, a human would have no problem with this. And the network is only at like 60%. So what kind of setup would you need to get it to be perfect is something that I'll try to explore in the future. And, you know, maybe that involves some kind of more modular structure to the network. Cool. Well, one last thought and then a last question. You mentioned that just because of trigonometry or optics that the optic flow is highest when the ball is coming closest. And I wonder if that might relate to kind of critical slowing of time or during pivotal action moments, a perceived slowing of time as a homeostatic mechanism for making sure that we can still act adaptively. And then when there's low stimuli, we zone because it's like we're having low flow. And then when things are happening like really fast, you're Neo in the matrix, then you have a critical slowing that helps you negotiate that. Right. Yeah, that's really interesting. So I mean, I know a little bit about critical slowing and just complex systems. And I haven't heard much about that in terms of like time perception or like the actual experience of it slowing. So that's really cool. And I'll have to check that out. But I think there are definitely some interesting relations here. But what I think is most interesting about the case where we turn off learning there, and it's still doing okay, just because of the laws of optics. So I think that's a great example of the Gibsonian point that there is structured information in the environment. So without having any learning mechanisms just by virtue of the way that optics changes with respect to movement, you get this pattern of change where it's more rapid when it's close to you on the sides, and it's slower when it's in the middle. And so there's already some information there about what you can do or what's happening in the environment that you don't have to perform any computations to infer that. And in this case, I don't know if you know, I'm not sure if we really see critical slowing down in the network and future steps for this will definitely be to try to apply some more specific like dynamical systems or complex systems methods to this to look for those signatures and the dynamics. To me, what it looks like is the network kind of just freaks out. It has a seizure when it's about to miss the ball. Everything starts spiking and it just goes crazy. And sometimes that craziness is going to make it move away from the ball, in which case it will miss, but it was about to miss it anyway. In other cases, it'll move towards the ball just by chance. So you get sort of a 50 50, but you're adding to your probability of hitting in that case just by doing something random. Excellent. Well, any final words or how can people learn more or be involved? Yeah, I mean, I'd love to hear from from any of you who have interest on, you know, working with me or just working with this model or related stuff. Got a lot of ideas for what to do next. I'd like to do some kind of evolutionary algorithms with this. I'd like to do some some kind of complex systems analysis of the dynamics of the network. I'd like to kind of compare this to data. And most of all, with some experts in this community, I think it would be great to ask those questions that we've been talking about, how do we do we see if this fits with the active inference approach or doesn't fit. So if you have any interest in all any of that stuff, reach out to me. My Twitter handle is at the beginning and the end of the talks, you can go back in the video. You could send me an email. If you Google my name, you'll you'll get my website. And all of our code is up on OSF, open science foundations linked in the paper. This is still a preprint, so I didn't have a chance to clean up my code before I put it up. So hopefully not too many of you are going to look at it. But before the final version goes up, I'll have a nice commented version. And it's pretty straightforward for anyone who wants to use it to get started. Thank you, Ben. Welcome back any time. And looking forward to how this all continues. Yeah, thanks, Daniel. This was super fun. Very well. Talk to you later. Bye.