 Welcome to the Active Inference Institute. This is the participatory online laboratory. Today is Tuesday, September 13th, and we are here with the mass stream 4.1 with David Spivak. And we are welcome to take any questions in the YouTube chat. So if you have any questions, please write them there and we will have time to ask our presenter during the course of this discussion. David is here to talk to us today about dynamic interfaces and arrangements and algebraic framework for interacting systems. Please take it away. Okay, yeah, thanks a lot for having me. So I'm gonna talk for around 40 minutes, but maybe almost an hour, I don't know. But feel free to interrupt with whatever. And I wanted to say at the outset that I'm not really talking much about Active Inference itself here. I'm mainly saying like another perspective on interacting systems or systems that might learn or something like that. And I'm more like waiting for the discussion time to kind of find these links. So yeah. Okay, so I wanna talk about dynamic interfaces and arrangements and what that means and this applied category theory point of view on things. So why am I here? I'm here to seek a valuable exchange of ideas because Active Inference offers, first time I heard about it, it's compelling and most people apparently find it unusual, but for some reason it instantly meshed with my own intuition about things that I've gotten from applied category theory. And so I think there's a lot we can learn from each other. So applied category theory emphasizes structure rather than quantitative analysis. So most of math and science today reduce all of our experience to plays of quantity. So you have graphs and you have numbers and you measure things and numbers and you have probabilities and those are numbers. And so everything is just a number, but experience what's happening now. I mean, what number would you put on the sound of my voice or what numbers you could put numbers on this slide but would they tell you what the slide was about? And so what category theory emphasizes it's still math but it emphasizes structure. It emphasizes relations between things like the relationship between what's on this slide and other things on the slide or what's on the rest of the talk, the relationship between that and me, the relationship between that and you and the kind of coherence between all that. Like why are we really here? What is it? How does it cohere? How do they come together? So applied category theory aims to bring that math to relational disciplines outside the math. So I wanna tell you about applied category theory specifically in reference to interfaces because I'm gonna say that we interact through our interfaces. And when you hear the word interface, you might think Markov blanket, I don't know enough to be sure that that's exactly what I mean. So that's kind of where this is fitting in maybe into an active inference framing but I'd rather you just think about interfaces. So I'm asking in this talk, what sort of algebra could support us in thinking about interaction and stuff like that? I think of mathematical fields as accounting systems. So I like that the word count is in there and I like how kind of humble it is. It's like an accounting system. So if you want to, if you have arithmetic, that's a mathematical field and it accounts for the flow of quantities like in finance. So if you want to your accountant to help you say why you did what you did or what you did and how the quantities flowed, you would write something in an arithmetic account. You'd write an arithmetic account of it. And they could someone could point that and say what's this $12.83 and you could explain. So there's numbers and you can account for stuff. What if you wanted to account for the teleportation protocol of quantum mechanics or something like that? So Hilbert spaces is a branch of math and it helps you make accounts for the sorts of things that people do in quantum computing or quantum mechanics or generally. Probability distributions is a mathematical field but it helps you account for likelihoods. Why is somebody winning a poker and somebody not? If you want to give that account and you want to be careful about it, you'd use probability distributions. So, but what is an account of phenomena? Like we're already kind of talking and somebody might give an account of their finance or something and just written in English but you can't really be that careful with it. You can't track it. You can't be sure you know what they're saying and that you can point at something. So the language, the accounting system has to articulate the relevant type differences and provide operations that correspond with their interactions. And that's really a dense statement. I didn't mean to, I'm trying to keep things concise but what do I mean there? In accounting for dollars and like financial stuff, there's dollars and there's numbers of things and you can add and subtract dollars like this was $5 and he paid four so there's $1 left so you can subtract them or add them but you can't multiply $2 figures. It doesn't mean anything to multiply $5 times $8 but you can multiply $5 by eight things like so I sold eight things of $5 each. So the operations that you're allowed to do they're different type difference. There's the number of widgets and then there's the price of the widgets and that's the difference in type and the operations make sense with plus you're allowed to add dollars and you're allowed to add numbers of widgets but you're not allowed to multiply numbers of dollars by numbers of dollars. So that's kind of what I mean here. An accounting system needs to tell you like, hey, that's not what dollars means what would it even mean to multiply $5 by eight dollars? Is it in dollars squared now or something? Anyway, so that's the sort of thing that an accounting system does. It gives you a way of knowing what you're allowed to do what the type differences are and how you operate on them. So if every mathematical field is an accounting system then category theory is an accounting system. So what is it accounting system for? It's an accounting system for the fact that we see in the world coherent structures. We see in the world flows of things we see in the world quantum mechanics. We see in the world likelihoods and we see in the world coherent structures. So if we want to account for them we need some language and the one that's been invented is category theory. And so it makes analogies which are similarities of structure with different content. I've got one thing and another thing A is to B as C is to D that similarity is made into a formal object and category theory has been useful in math and computer science, physics, material science, linguistics and moreover one thing I didn't see on the slide is that category theory helps you invent new mathematical fields. They might not be big ones that are studied by a million people. You might just have a micro custom built accounting system written in category theory. So you make a new system of structure. And what I'm interested in is what sort of system accounting system would could we use to account for dynamic interaction? Like things interacting. So the driving question, I really prefer to have a question that I can always be asking instead of one that I need very specific lab to do. So I want to know what we have right here. I want to account for this incredible world that we have because on earth we have all these amazing forms of life from cells to humans. Like how did that come about? And we have the built world where we have transportation systems and things like that. And we have computers. It's an amazing built world. We have language, the ability to systematically present knowledge like a slideshow or a history book or whatever. We have morality with rules of thumb for living a good life. How did that come about? I think in each of these cases, they evolved through the push and pull and struggle of living. So language is refined by continually using language and the way we present knowledge or the built world, the way we architect things is continually evolving. And in even morality, like you have a bunch of societies that have to live together and they evolve a way to do that. So I wanna see all that. I want to account for that. So what are these systems and how do they develop and how do we talk about them all at once? What language is appropriate for giving accounts of this happening in action? And can we use that language to engineer new systems? And then something we were talking about just before we started to talk today is like what constitute health or wisdom? What gives the system a sense of direction and how come it keeps finding good senses of direction in the sense that evolution, somehow our civilization is offering more today than it did in the past. So the only part of this I'm gonna discuss is the potential accounting system that I'm offering here or like the beginning of it. So in other words, I don't know how all this stuff happened. I just, and I don't wanna be the one to figure out how all this stuff happened. Because obviously it's more than even, than any group of people could really finish or even, and I don't even have the capacity to really start. But I think there are many people who do and already are doing that. And all I'm trying to offer is a way we could put that into some sort of accounting system. Okay, so this thing that we're doing now is the subject of the talk. It's dynamic interaction, morphology, the way that things form like Topos Institute formed or a person forms and behavior, all that stuff is about little systems that interact. And this interaction structure itself changes through time and I'll explain a mathematical language for working with that sort of thing. But on some level much of this talk will be a case of its own subject matter in the sense that I'm interested in the development of the life's order, like how does this develop? But in the science system, this takes place between the interaction between fields. So right now you guys know much more about active inference and I know more about category theory. And so we're participating in this science system where we're interacting and we're having dynamic interaction because after this, the part where I'm just talking I'm still looking at Blue as she's looking at me and I'm seeing her nod and that helps me decide what to say next and whether we're, how this communication is working. So we're experiencing the development, the ability for systems interacting to shape that interaction and become like sense make together in real time here. So the concepts discussed in this talk should describe the talk itself as interaction. So let me just pause for feedback and just kind of get someone else's voice into the space if you're willing. Do you see like kind of what I'm saying that the math I'm doing, I'm talking about here should be capable. I'm not going to give the description because I'd be like filling out the spreadsheet with all the facts, but it's more like that the math could offer the sort of spreadsheet you would need to talk about this sort of thing. So definitely like math related to interaction is really what active inference is all about and fundamentally describes like sense and perception and action. Daniel joined us for a minute. Do you have any extra comments? I have a feeling you have something to say here. No. I would just ask what can math describe and where do our science and other things that are not math also play roles? How do those interact? Great. Yeah. Cool. So I'm suggesting that math gives you the accounting system and that science fills it in, but let me, yeah. Yeah, so I think I'm going to go into that a little more, so there's a self-reflective character where I'm saying that you guys just actually participated in an interaction with me and like gave a kind of sense of direction for this, talk a little bit or at least one thing we want to discuss which is what's the role of math and science. And so we're kind of co-creating this sense-making work that we're doing here through our interaction. And so that's kind of what I'm saying is that there's a self-reflective thing that the talk itself, if you want to you can always dive into this itself as a case, so that's kind of hanging around. But I'm not going to push really into that, the fact that we're building something together here too much more, but it's here. So I want to talk about how theory compresses and information and then there's elaboration and there's feedback from experiment. So I think that compression and elaboration play a big role in this story. We compress our past into a form that we elaborate in the present. So we're constantly forming memories that are not as big as the thing that they're memory of, but they're compressions that we can elaborate in the present. DNA, for example, compresses a record of who died and who thrived in some sense into a language of four nucleotides. And theory, any theory is a compression of past experience. So the free energy principle is someone has compressed ideas about what they've seen over and over again into a very small form. And then, whether you're talking about DNA or memories or the free energy principle, these are elaborated in the present. So as I speak, this theory about compression or whatever, not very original, but it's here, that is an elaboration of something that I've compressed into these slides in the past where DNA is somehow read by Mestrarnay and I don't know, there's a ribosome involved and stuff and it's elaborated. Those genes are producing proteins and those proteins are elaborated to, are the elaborations that compressed code and they then create the living form of the thing. So that seems really important. And then similarly, experiment and theory, if theory is like compression and experiments like elaboration, these exist both in biological and mathematical fields. So almost everyone knows, everyone who knows of biology has knows that there is experiments involved, but maybe you wouldn't think that math has experimentation and I think that it does, but it's the attempts to articulate and compute things. That's how you elaborate it. You talk about things and you compute and we value a formalism F if it makes expression and computation easy. So that's the experiment, like how good is this thing? Well, does it make it easy to say what I want? It doesn't make it easy to compute. So with Roman numerals here, like I don't think this is very good at expressing, but I know it's really bad at computing. So if you want to multiply this Roman numeral by this Roman numeral, you're just kind of screwed. There's no procedure you can go through. The only procedure I know is to translate them into Hindi-Arabic numerals 14 and six, for example, and then use what these are built for, which is using the distributive law. So 14 is 10 plus four and 10 times six is easy and four times six is easy. So you get 84 and so you translate it back into LXXIV, but where that L came from or why there are three Xs, you'll never figure out. It's just 84. So the Hindi-Arabic numerals I'm saying are empirically better. Like they just win games, right? Like Fibonacci not only came up with a one, one, two, three, five, eight sequence, but probably more importantly was the one who brought the Hindi-Arabic numerals. So the Hindus apparently invented them and the Arabs saved them for a long time and then Fibonacci found them and brought them to the West and they're just better, they just won. They just won out. And that's the way that experiment exists in math. How easy it is to use. And that's the sort of value claim I'm making about what I'm discussing today. That it's a single accounting system. You're not constantly changing what the notation means or changing how you're thinking about things. It's all one system and it just works really well with itself. So the plan for the talk is that it'll be in four parts. I'll give a presently available case of the thing I'm trying to model. I'll give a whirlwind tour of what the math actually looks like. I'll talk about some existing applications and open questions and then I'll conclude with a summary. Okay, so the current dynamic arrangement. So this Zoom call that we're having right now and maybe with the YouTube live stream and the people on the chat and cetera is a dynamic arrangement. So how do we think about it? I wanna break it into three structures. Interfaces, dynamics and interactions. So I'm gonna have like, I think overall six keywords and these are three of them and I'm sorry there's so many but they all are different. So each one of us has an interface. So it's what I can express and take in. I'm expressing through the changes of my vocal cords and the movement of my hands and I'm taking in information through the sensations in my body and my eyes and my ears. And then each one of us has dynamics. So I have some internal state that you can't see exactly but I'm having some mood and I'm expecting to read and all that sort of stuff. And then that's updated by what I see you guys doing or what I'm reading on the screen. And then between us there's interaction which is how my outputs, my expression affects you and how your expression affects me. So those are what I think is going on. A lot of what's going on in this dynamic arrangement between us. It's really useful to draw boundaries around things. So you modularize by nesting reference frames. We have little, a reference frame is all I mean by that. I don't necessarily mean the thing they mean in relativity theory. Although I think that would be a case. A reference frame is just a box you put around things and you say I can refer to this as Alice and this as Bob and stuff like that. I want a frame where I get to have a notation or a way to refer to things. And we have littler systems interacting within bigger systems all the time. We could say that Adam with its electron orbitals or whatever is some kind of reference frame. Or so, but atoms are even molecules. We could think of the molecules of reference frame for all of its atoms. You can point to them and refer to them. Organizations and societies, we modularize and we say that's a thing, that's a thing, that's a thing. And we are putting things together. And that's useful in evolution. So for example, according to Herbert Simon, like that's how evolution is so fast is because it modularizes our liver and our stomach and let them evolve independently. But also or various systems, circulatory or nervous system, et cetera. But also it's useful in the way we think. Humans think about things. I want to emphasize again that the map here will not be numerical. It'll be structural. So as we move into the map, you'll see how it manages these reference frames. And so like geometry is about shapes, geometry doesn't have much number in it, but there are numbers a little bit. There's like how big the angle is and how big the sides are. But it's really about the shapes and the congruences and stuff like that. And similarly, you can put numbers into category theory, it accepts that. But it's really about the structure of why you're using those numbers and which numbers can be added to other numbers or what the rules are about the numbers. And really you don't need numbers at all. The relationships that are happening here, for example, are not numerical. But before we get there, let's make sure we understand our principal subject because we'd like this map to be able to describe things we work with like cells or tadpoles or us or institutes or whatever. And soon I'll define what I mean by interface as a mathematical object. But I want to ground this talk in something we can all think about together. And since we don't have cells or tadpoles here, we're the subject. So this Zoom call, let's say, and with the live stream and all that is the subject. How do we think about ourselves? Well, each one of us can do certain things and receive certain things. Okay, so I can move and listen. We each outwardly express and take in what we do there with what I'm calling our interface, as I said before. And I'm gonna call these things. The blue will always be the things we can do. So it's like the positions of my body and the green will be what I can take in kind of like what's the forces that act on me or the sensorium I have that acts on me. So any expression, whether it's a sound or attitude is a kind of position. It's a position I'm taking and kind of popular like parlance or whatever someone's expressing themselves or taking a position. But you could also imagine the position of my vocal cords moving around and that movement of position is an expression that comes out as sound. Your interstates are reflected outwardly as these positions. It's happening maybe faster than you can recognize in the case of vocal cords, but it's still an expression using the blue, the positions, the outward expression. And then the world impinges on you or in me and it directs me and it moves me by forces. So when I see there could be literal forces on me or you could just say, hey, would you mind passing the salt? And that impinges on me. It directs me to pass the salt. It moves me, maybe I don't do it, but at least it changes my internal state to being stubborn or something or to being helpful. So what your body can receive in a moment? This is a kind of weird one. I'm not going to emphasize it that much, especially later, but it's true. I think that what your body can receive in a moment depends on its position. So when your eyes are open, the type of thing you can get in through your eyes is bigger than when your eyes are closed. So more can act on you. If I close my eyes, I just won't know what you're doing. And when your car goes through a tunnel, the GPS stops receiving. And when you unplug your monitor from your computer, it's no longer, or your keyboard, it's no longer receiving things. So the position can affect what it can receive. The monitor is no longer receiving from the computer if you disconnect. So the story so far is that you output blue positions and you B-L-U-E and you input forces. And so the force or sensation or input that you receive changes your internal state. And then your state is reflected outwardly as a new position. I move my eyes or my head or my voice box and a new sense. And then if I close my eyes or open my eyes, I have this, whatever happens, it may be the same space of sensations or different, but I see new sensations and those change my internal state, et cetera, et cetera. So this is loops repeatedly, either discreetly if you want to model it that way or continuously. But then where do these forces inputs come from? They come from this interaction. So the way that we're arranged here, I'm calling it an arrangement and that's gonna be another keyword. Zoom arranges it so that my outputs get to you as inputs. So you can hear me. And as you receive what I say, your body shifts and that's your output and that comes to me. So the way this works is based on the arrangement that we're calling Zoom and so now we have another color, orange, and the program arranges it so we can input each other's outputs and our interaction with the program may change the arrangement itself. So if Daniel wanted to, he could spotlight me or mute me or change how the information is passed. And so the very things that you're doing, the outputs that Daniel is doing, or me, I could ask him to do that or I could mute myself. The part of our outputs have the ability to change the arrangement between us. We can walk away or whatever. You don't have to actually mute me. I don't know why you're smiling but I'm imagining you doing some practical jokes. Okay, so I'm gonna move into the math now. Yeah. So this algebraic theory of interfaces and arrangements, there's a bunch of keywords we're gonna use and here are the six I think, one, two, three, four, five, six, seven. Interfaces, which positions, forces, states, arrangements, enclosures, nesting. Okay, I don't think I've said all of them so far but they shouldn't be too bad at this point. So we each have an interface. It's what we can, it's that through which we interact. Our interfaces allows ourselves to express ourselves outward to our position and given a position, our interface allows us to receive certain forces or sensations. Our state is changed by the received force and then our state is expressed as a position. So our state changes based on what we see and then we express it and then multiple interfaces interact together via their current arrangement, how we're arranged together. Doesn't have to be spatial arrangement. We are, the three of us here on the Zoom call are arranged, I guess, in space but really more importantly in terms of how Zoom wants to arrange us. And we also interact with our enclosure which is the thing outside is another interface and these arrangements can be nested hierarchically. So here's a picture and it's a kind of schematic. It's not supposed to be what we really are but every box here is an interface. So I have one, two, three, four, five but I also have Y one is an interface, Y two is an interface and the Z is an interface. And the way I wanna draw it is that the interfaces, each interface has on the right I'm gonna draw the positions, the outputs. So this thing is pushing out stuff to the right and on the left it's inputting forces or directions. So maybe this is me here and I'm seeing through my eyes and I'm outputting some visual signal and I'm outputting some auditory signal and then somebody else is receiving that auditory signal and seeing some other thing and then other person is saying something and doing something. Okay, so each box will have the positions it can be on the left, on the right, sorry and the inputs or sensations or forces on the left. And my outputs become things that force you or direct you or push you or our input to you if we're connected like this. So inside of each box, you could imagine there's a state and it's changing based on what's coming in from the left hand wires. So inside of each box is a state and it's being affected by what's on left and it's pushing out data on the right. And then the multiple interfaces are interacting together in this arrangement. So Y1 is supposed to be the outer box here or this middle sized red box is Y1. The arrangement is this formation of wires here. This Y1 thing is the enclosure it is enclosing these three things like Topos Institute encloses some workers inside of it or I enclose some organs inside of me or something like that. And then these arrangements finally can be nested hierarchically. So there's me and there's you and then we're communicating but inside of us is more stuff and then we are forming this Zoom call. Okay, I hope all that terminology is making sense and feels natural. The thing I didn't say here and it's very hard to draw is that the arrangements can change through time based on what flows within them. So if I say, hey, please mute me or something like that it might change the arrangement so that you're no longer receiving from me. So the arrangement can change your time based on what flows. At the end of this call when we say goodbye it will change the arrangement. Okay, so now we're really moving into the math. So as we said, an interface consists of two things a set of positions. Maybe I can be in three positions A, B or C or maybe I have 44 real numbers worth of positions. I'm just outputting this gigantic 44 dimensional vector constantly or something. But whatever you, the point is to make an accounting system so you can model what you think are being but what the positions are. Maybe you can decide for yourself what set of positions a being of type A could have. What set of positions being of type B could have. So secondly, for every position there's a set of forces F bracket I or inputs. You could call the sensorium it's what can arrive when you're in that position. If you're a protein and you're folded up maybe you just can't receive certain things on your interior. You don't receive charge but when you're unfolded you're receiving charge. Or if you're a hedgehog as Toby Smythe says when you're folded up you can't receive stuff on your belly or something. Okay, so what we'll encode this as a polynomial in a single variable Y. Why? Why because it stands for Yoneta in category theory and because it doesn't, I don't know somehow it's kind of makes you realize this isn't your standard polynomials not an X but I don't know I just use Y. It's traditional for me at this point and this is gonna be a polynomial with non-negative coefficients. When you think of polynomials you might think about numbers and like X squared of five is 25 and stuff like that but I'm not gonna be plugging in numbers into these polynomials. They're really just ways of housing the data structure or like what we're gonna wanna work with. So I know it's strange but it works really well and it's formal and I don't need to freak out when you see the sum sign because even though I'm showing you the map I'm not intending for you to like completely follow it or need it. It's an accounting system but like Fibonacci coming to the West like you might not be able to use it right away but if you're interested you could learn. So an interface might be this would be the whole interface. If you set up positions P and this dummy variable Y that's always just gonna be there because it kind of holds the place for this exponent and the exponent in the position I is the set of forces you can get in position I. So now we've made math for this interface thing which has positions and for every position a set of forces that can depend on the position whether your eyes are closed or open, et cetera. So imagining the interface was the sum of Y to the fifth Y cubed Y cubed Y cubed 62 times Y to the zero Y to the zero you add all those up that would have one position that had five inputs so this sensorium when the person's in this position would be five possible distinctions they could make whereas in these 62 positions they could run around these 62 and you would watch them running around here and every time you would know they're receiving three possible distinctions as inputs whereas when they're over here they're receiving five when they're over here they're receiving zero they can't receive ever they can't receive. Here's another polynomial R cubed Y to the R of the million it satisfies this thing these don't have to be finite sets so in this thing it's positions it runs around R three so now it's running around not only on the surface of the earth but it's just running around wherever it wants to and some kind of fixed coordinate system of the world of the universe and at any point no matter where it is it's always receiving exactly a million inputs so maybe it's receiving a thousand by thousand image it's got some so that's what it's receiving but you just make your own it's an accounting system for you to make your own thing your own interface and you can make as many as you want so why do all this craziness with polynomials because what's amazing to me is that the polynomial operations mean things in terms of interfaces we can add polynomials you know how to do that you can multiply polynomials you might remember you do some kind of people call it foil you just kind of multiply the polynomials you can compose two polynomials like P of Q but then there's some operations you probably haven't heard of like tensor and or and internal ham and these are just these are what are called monoidal structures on and a monoidal closure on a category polynomials and I'm not telling you what that means if you notice a category theory you can look at it you'd know already what it means but they all mean something to us in terms of what you can do to interfaces and I'm just repeating this interface here so if I had one of these called P and you had one of these called Q you could add them you could multiply them you could compose them, et cetera so suppose P and Q are polynomials representing two different interfaces then what's P plus Q well it's another it's another polynomial so it again represents an interface and it would be the sort of it would be the interface that can output a position of P or Q and its sensorium in a given position is whichever if it's a position of P it would be the sensorium of that thing or the force is there and if it's a position of Q it would be the sensorium or forces that you can get there so if P was this interface this kind of thing that output these sorts of positions and then put these sorts of forces and Q was this one then P plus Q is what nine y to the fifth plus two y to the fourth plus y cubed and that would be like a machine that can be either in one of these three states or one of these three six states and sorry positions and when they are they'd be inputting five possible distinctions or one of these two inputting four possible distinctions, et cetera so that's what P plus Q represents it's somehow making a new interface but P times Q represents making a new interface in a different way something with this interface would output both because you have to like remember how you multiply polynomials but basically the sum ends of the product polynomial a sum end of this thing is a sum end of P and a sum end of Q so the positions of this thing would be a pair consisting of position of P and the position of Q but an input there or a sensation or a distinction you can make there would be one of either P or of Q because like just if you remember like what if P was five y to the fourth outputs five different things and it puts four no matter what and Q is six y cubed outputting six and inputting three then when you multiply them you would output 30 I'm just multiplying straight across here 30 y to the seventh so you output both a position in P and a position in Q a position in this five by six grid here and then you'd input one of seven things either something to P or something to Q that's just what times does you could say well I want to input to both of them I don't want to input to just one or the other and then you're using a different operation called tensor so P tensor Q takes these two things and multiplies both positions and the directions and the sensoria and lets you output both and input both all the time so this operation does that times operation does one I said plus operation does that composition is an interesting one when you compose polynomials what it would do is an output would be something that output of P and then also a strategy for no matter what the P input was and output of Q it doesn't have to wait for you to tell it what the input to P was it'll just give you the strategy for automatically outputting a Q thing once you know an input so it runs P then Q in series and then yeah and so this thing kind of outputs a protocol I think I'm going to say that on the next slide a little better so or pretty soon the OR operation runs either P or Q or both in parallel and then this bracket operation runs arrangements for wiring P and Q if you remember what that means like I'll get into this a little more but this one is going to output arrangements of P and Q and input what flows on the wires so that's going to be an interesting one for us later and all I'm really saying is that all these different operations take interfaces and make new interfaces out of them and that allows you to kind of build things up modularly we're also going to be able we can also formalize the notion of arrangement so we thought about ourselves in the Zoom call as an arrangement because Zoom lets my outputs be your inputs and vice versa but you could also think about cells and organs and or cells or cell organs arranged in a cell or cells in a tissue the arrangement is just how the information passes between those interfaces I'm not so concerned about the spatial arrangement except as it allows information to pass between the interfaces so here's a picture again and the orange thing is the arrangement so two things about this picture you can see that the arrangement is how information passes between interfaces you can kind of see it visually but two things about this will be generalized first the information sharing can be much messier you don't accept in a circuit you don't just have perfect wires the inputs here might be affected by all sorts of stuff and second the information sharing can change like the interfaces can change shape I might close my eyes but also the thing I say to you might change our interaction so that's contained in this notion of an arrangement so formally if you do know category theory then you've probably heard of categories functors and natural transformations and polynomials are in fact functors from set to set and an arrangement it looks like this picture but all I mean by it is a natural transformation of polynomials so every one of these pictures can be translated into a natural transformation of polynomial functors and what this picture would be is you take the little arrangement a little interfaces P1, P2, P3, P4, P5 you would tensor them together and then the arrangement itself would be a natural transformation from P1 tensor P2 tensor P5 to Q the outer interface here now what's happening so brief interlude from that stuff I just told you that an arrangement is a natural transformation of polynomial functors and all that's category theory and I'm not expecting you to know category theory I'm not expecting the audience to know category theory so what's up with why would I even tell you that stuff I'm just trying to show you what the math looks like let any technical person in the audience who does know category theory see that I'm what I'm talking about and kind of know where to look for what's next but for everyone else or everyone I just wanna show what the math looks like and what it's about because so much of math is quantitative or maybe it's about shapes or something but this math is really different so I want you to see it because I want you to ask like what is how is this math and what is it for and the point is it's about interfaces it's about how you manipulate them how you arrange them how you nest them and interfaces have outputs and inputs and they're captured that can depend on the outputs and that sort of structure is kind of hard to talk about cleanly but it's really well captured with polynomials not considered as functions but as like ways of holding structure and then you can do these operations to the polynomials and they get they let you make new interfaces from old and we'll be talking about arrangements and dynamics next but again they'll just be in this category of polynomial functors that it's all kind of the same story again and again we don't have to change our accounting system to move between interfaces, arrangements, nesting, dynamics, et cetera. Okay so dynamics let's look at this arrangement again what if every PI all these P's had a dynamical system running it well then what I'm saying here is that Q therefore would two so a dynamical system is a thing with states that evolved through time. My current emotional state or the state of like where I think I am in the talk or whatever is my current state and then I look at you guys and I see what you're looking like or I'll read the slide and then it changes and then I output something so that's what a dynamical system is and then it can be formalized as a system of ODE's or a discrete dynamical system or you can just think of it as a high level idea and I've said this so many times now but I'll say it one more time you're in a state that shows up as your position in other words your output and then any possible input you get would change your state and right so the math could be discrete or continuous it both works in this polynomial setup. So mathematically a dynamical system one P is a natural transformation S Y to the S to P where S is a set of states and again why am I telling you this because all the math looks the same whether you're talking about arrangements as natural transformation between polynomial punctures or you're talking about dynamical systems themselves in happening these interfaces they're all this natural transformation between polynomial punctures and then given the arrangement of all these P's inside of Q which is natural transformation and giving this natural transformation here that puts the dynamics in each of them you get the dynamics in Q. So if each PI had a dynamical system in it so would Q by composing natural transformations. So this is the currently the last math slide. I wanna think about dynamics a bit more so if you remember a few slides ago I was talking about operations and a dynamical system on P plus Q would be one that can switch between P mode and Q mode so sometimes you're outputting P stuff and then putting from it sometimes you're outputting Q stuff and then putting from that sorry I'm a little bit sick so my voice is not great today. A dynamical system on P times Q would output both simultaneously in any state it's outputting both but we can receive from just one or the other. A dynamical system on P tensor Q would output both and receive from both. A dynamical system on P composed Q would do the serial protocol where it outputs a P and outputs a strategy for taking any input on P and turning it into an output on Q and then it would listen for an input on P and an input on Q. So kind of just tells you here's what I'm gonna do no matter what you I mean it gives you a serial protocol is what I'm saying. And what about P inner-hom Q? This is the in the category of polynomials, functors there is a internal HOM for tensor. You may have absolutely no idea what I'm talking about but it's something that comes with the math that no one created it, it wasn't invented it just is part of the structure. Kind of like the complex numbers are miraculous in the sense that you say you want algebraic closure and all of a sudden you get if I have one derivative then I have all derivatives and stuff like that. So it's kind of the category of polynomial functors is kind of miraculous. It has tons and tons of structure that no one asked it to have it just hasn't. But what is this structure, this internal HOM? Well it's the sort of thing that runs arrangements for P and Q. So a dynamical system on this thing where I tensor P1, P2, P3, P4, P5 comma Q and put it in brackets this internal HOM. What it does it's a dynamical system. This is a polynomial even though it looks like a big expression it's itself a polynomial and this polynomial has outputs and has inputs as usual. And so what a dynamical system on this would do was would be output arrangements. Okay now I want you to wire to you and you to wire to you and you to wire to you. So if you're forming an institute you'd say wow let's have this guy report to that person and all that sort of stuff and you would create some arrangement and then that's what you output as the CEO say and then you listen and you see what flows on the wires and you see how Alice is talking to Bob and how Bob is talking to Carla and all that stuff and you'd listen to that. And as you listen you'd say hey actually I want Bob to report to Carla instead of Alice or whatever. And so you would change the arrangement. Your internal state would be to listen and receive the stuff that's flowing on the wires and you'd be affected by that and then you'd output new arrangements for the system. And I think that's kind of getting to this morphology thing. Maybe there are systems that are monitoring the source of transmissions that are happening between subsystems and then outputting new ways that they should arrange each other. Okay, so let me give some applications. Digital circuits and control systems that neatly here. So a computer is a nested arrangement of dynamical systems. Every transistor is a dynamical system and two of them make up a NAND gate, not NAND. And then you can get an OR gate by wiring together three NAND gates like this. I've taken my inputs and I split them and I feed them to the NAND and to the NAND and then I put them together and I put them into the NAND and now I have OR. And this whole thing is just like NAND has two inputs and one output, this whole thing has two inputs and one output, but it's got some internal structure. And now we've got an OR gate. We could nest OR gates and NAND gates and NAND gates and we can get adder circuits and multiplying circuits and ALUs, arithmetic units, whatever, and CPUs and monitors. And they're all made just with transistors. Like the memory cells are all like flip flops are just situated transistors. And so we can kind of make this nested system of transistors and get a computer. And so the arrangement here is all really simple. It's never changing in time. It's soldered in, it's fixed. Unlike us on the Zoom call, its dynamics are simple. Every state is just a function of the input only and it just uses, there's no memory in the transistors. And yet because of loops in the wiring and the arrangement, we get all possible things that the computer can do. So with enough nesting and enough interesting even though the arrangements internally are not that difficult, when you nest them they get really amazing in the sense you get a computer. And that is the dynamical system that is happening right now allowing for this Zoom call to take place. I guess there's some internet stuff too that I don't know how it's made but the computers are made this way. Control systems also have complicated dynamics. Let's say this is a power plant or something. But again, I think they basically have fixed arrangement. So your cruise control in your car isn't changing whether that's listening to the gas or the, maybe for the battery in a hybrid car there would be a changing arrangement. But at least old fashioned control systems have a fixed arrangement of how the feedback is gonna work to control the speed or whatever temperature. Deep learning also fits into the formalism of polynomial functors, but in a different way. Here, the interfaces are all the same. They're all outputting a real and inputting a real. Now, and the arrangements are all really simple. They're activated weighted sums. Now, I'm drawing this differently than most people would. Most people would put these three boxes on the left of this big box and they'd wire the three things in and it would seem like this was a box taking R3 to R. The way I'm thinking about, because I want to put this into the context of this accounting system of polynomial functors it works better to think about a neuron listening to some neurons on this left or some predecessor thing. So like a higher level thing, listening to lower level things. It makes more sense in this formalism to put them inside. And then the arrangement isn't just a fixed, isn't just a wiring diagram, but it has more structure. As I said, those wiring diagrams were just easy cases. Often it's more complex, but they're still arrangements in the sense of natural transformations of polynomial functors. And what this arrangement would do is it take the output of these sub guys and weight them. Maybe you multiply this one by 3.2, et cetera, add them up and then do an activation function and output that to the higher level, bigger thing. Then when it gets a loss in, it would take the gradient and pass the components of that gradient to the three, or pass the components of that loss to the three things and then take the gradient and change the wiring diagram. Not the wiring, but the interaction pattern. So it would change these three numbers. So the information flowing out is the interbox's current guests about what's going on in the world and the current weighted sum. Like these guys send out their guesses. We add them up. We send out our guests as the higher level manager here. And then we get some loss function. We distribute it to our sub units. We also use it to change the arrangement here. So the changing of weights in the training of a neural network is the changing of arrangements that I've been kind of pushing this whole time. Right, so the arrangement is dynamic. The loss not only is sent to little boxes, but is updating the arrangement, the collection of weights. So deep neural networks is one of four examples we currently have of a certain structure that a postdoc working with me named Brandon Shapiro and I are calling a dynamic organizational structure or dynamic operad. It's a fractal like system of arrangements that change through time where the arrangement decides how information flows through the system just like I've been saying. And yet the flowing information can change the arrangement just like I've been saying. And we formalize this using category theory. And the four examples we have are deep learning like I was talking about in the previous slide, prediction markets, heavy in learning. So this is with Sophie Lipkind and non-cooperative strategic games. So we have only four examples of this structure but I'm guessing there are many more. So I'm guessing for example the active inference can be another one or maybe free energy minimization or something like that. The point is that these are like very fractal structures where like a collection of deep learners or gradient descenders, if you put them together, you get another gradient descender. If you take a bunch of predictors and make them into a single predictor, you get another predictor. So like the collection of predictors becomes a predictor. And the way that predictors are, the way that wealth or trust is distributed between predictors is properly nested and stuff like that. Can't really go into it but I'm interested in whether, well, I'll say that on the next slide. Okay, so wrapping up kind of, many people think of math about number but it's not, a math subject in accounting system tracks certain things and what we're trying to track above is dynamic arrangements. Math is called polynomial functors and it's well-known and beloved in category theory and it accounts for interfaces, dynamics and arrangements in a really nice way. And it's about building up bigger system from littler parts. It accounts for composing circuits and control systems and deep learning and prediction markets. And then you could combine these because they're all in one formalism. You could take your control system and hit to hook it up to a deep learning system and put that in a prediction market and all of it's just polynomial functors all the way down. I think it might have, maybe I should have said it might have particular value in active inference because it lets us talk coherently and precisely about these things like changing interfaces. I might be wrong but I don't know that that's a part of the formalism of active inference today in terms of like changing the arrangement and stuff like that. But I'm interested to hear more. You guys changing how things communicate, change how the communication pattern itself changes as information is exchanged. But if we wanted to make active inference into a dynamic organizational structure, for example, you might let dynamic organizational structure, one person when they saw that they thought, oh, you should really call it multi-scale dynamic organizational structure in order to really clarify your particular brand here. But maybe not. So what will we do if we wanted to make pre-energy minimization a dynamic organizational structure? Well, we would specify all the interfaces that we're gonna be talking about that would be used to house pre-energy minimizers. Like what is a free energy minimizer allowed to output and what are they allowed to input? And you would get clear on that as step one. And then you would consider how free energy minimizers arrange themselves, like what an arrangement of pre-energy minimizers would be where if you arrange them in that way, it would constitute a larger scale free energy minimizer. So that's the sort of thing you would have to ask to make this into a dynamic organizational structure. And as long as they all cohere nicely, there's some formals check we would do, we would have a dynamic organizational structure. So to summarize, I feel like I've said it a million times now, but applied category theory is math for tracking interlocking structures, cohereing structures. It's not about how much of something there is, it's about how it's arranged. And it applies in quantum mechanics and computer science and math, material science and linguistics, but it focuses on structural questions in each one. It allows the content to be something that someone else handles later and it's just telling you how you're going to be handling that content so that you don't kind of step on each other's toes as you use it. So I'm interested in the structure of interacting dynamical systems, like how I want to account for, be able to account for this. I don't want to necessarily give the account, but I want to be able to account for this stuff that we're doing here together, how we change, how we're inputting, I'm changing by turning the speaker off or disconnecting and therefore not inputting the same or how we're outputting, muting or camera off and how all that affects what happens on this call or wherever we are and how we change and what we learned and how we make sense of things together. So again, I don't have specifics, but what I'm offering is this accounting system, this category of polynomial functions to account for dynamic arrangements. And I'm hoping that it is helpful for thinking about active inference. So that's it, happy to talk about stuff. That's super awesome, David. Thank you so much. I have a million questions, but I didn't want to interrupt because I was really enjoying just hearing you because it helps it gel more solidly in my brain if the interaction is like, give me all the things and then let me like pick apart the little pieces and maybe that works for hopefully the audience also. Daniel, are you watching the YouTube Trotters? Are there questions? No? Do you want to go over there? Because I'll interrogate him for a while. Okay, and this is a block you, does it? This thing up here? No, it looks fine. Okay, I know it seems free to be there. Lou, please begin the interrogation and I'll be ready with the live questions soon. Cool, all right. Okay, so let me just go back to the beginning. So in the very beginning you said that you feel like active inference kind of meshes with category theory. And I just am curious as, I mean, I think like I kind of got a really good idea through the course of your talk, but maybe you could articulate the why to maybe kind of help us understand where that basis is coming from on your face. Cool, yeah. When I first heard it, it was, one thing was like utility and prediction are melded. Like what I want is, I'm just expecting the world to be a certain way and what I want is for it to be that way. So if I really intend to have something, it's because I know that I kind of deserve it. I can't actually, I think people might have fantasies where they think they can get things they don't deserve or something, but it doesn't work out. I'm not sure. I feel like I'm going into too much there, but I like this idea of melding those two things because utility never really made sense to me. It's like, where do I get this utility function from? Well, what I somehow thought was like, like what they're saying is like my knees are a certain way, they're a prediction of like the fact that gravity will be in a certain range or my lungs can take in this sort of, or this way because they're a prediction that my lineage has made about what sort of atmosphere I'm going to be put into. And the things I'm saying to you right now, like utility, whatever I'm using to decide how good it is for me to say a thing is my prediction of what's actually wanted to hear. So I like that it would like reduce the number of variables there, like instead of number of entities in the universe or something from utility and prediction just to prediction. I don't remember like other things exactly, but I just remember kind of feeling like, oh, I also like that it's scale free. So free energy minimization, for example, or Bayesian predict, update these stuff doesn't need you to be at a particular scale. It's not about subatomic particles or something. I like this scale free. And that somehow it's a physics-y thing. Like I'm really looking for a scale free thing. You mentioned emergence blue, I'm going to stop talking soon because I know you're last at thing, but I'm not a huge fan of the concept of emergence because I feel like it's usually talking, assuming we're going to start from a low level and then say, and then the next level is magic. And then they start from that level and they're like, and then the next level is magic. And I'm really looking for like the principles where the inductive step, the move from N to N plus one is the thing we're monitoring. So we're never like confused or like, whoa, where did this amazing thing happen? We're always seeing, we're monitoring that part. And I really liked that free energy minimization and entropy maximization and stuff like that is the sort of thing that I could, imagine happening at all scales, no matter if you're a person or you're a cell or you're an atom. Cool, definitely. I have to apologize for this like beeping that's going on in my background and I'll try to remember to mute. But I do think that like, yes, emergence is cool, but also there's this bi-directional information flow. And when we had Mike Levin come onto the live stream, he really talked about it when he discussed this paper, the computational boundary of a self. So yes, like there are bottom up, there are emergent properties that happen in a collective, but there's also top down constraints on those emergent properties that come from the quote higher level or maybe like larger scale. So like if we're building things up and down, there's constraints that go down and information that comes up and you've gotta have this bi-directional information flow to even start to think about drawing any kind of boundary or creating like an interface around like the thing itself. Like we talked about cognitive cones, Mike Levin. And I don't know if you're familiar with that paper, but yeah. It was cool. Cool. Something else. And so I know like you've talked with Mike a lot and also with Chris Fields. And when Chris came, he really talked to us about contextuality and he started to get into some math that I think is maybe related to some of the things that you were talking about here. So you talked about operations that correspond to interactions between sets. And Chris talked to us about two spaces and like channel theory and as these like interactions that transform sets as infomorphisms. So like, I don't know if the name of that like language speaks to this or is like, it's hard for me to tell what is the language of category theory and what is like the application of the category theory to something in real life or that's not just the set of acts such that X is less than or equal to like these kinds of math words, right? Right, yeah. So the two spaces stuff is definitely category theory and relates, there's a nice way of talking about the relationship between two spaces and polynomial functors, although I don't have it on tip of my tongue right now. So these polynomial functors are kind of bi-directional because the like, if you have these arrangements which are maps of polynomial functors, this sort of thing, this quote unquote higher level thing is talking to the lower level things. Lower level things are talking to each other and they're also talking to the higher level thing. So there's this kind of bi-directional thing there. And so on that level, I'd say yes, it does sound similar to what like Chris Fields was saying. And then, but you also ask like, what is the application of category theory? So once you have an accounting system, you can build a tool from it. So if someone says, hey, I got this great idea. It's all about plus in times and like dollars can be added but not multiplied and they can be multiplied by numbers but not each other or whatever. You make this rule and then someone could build a spreadsheet that program that incorporates that rule. And maybe current spreadsheets don't allow you, don't keep you on the rails with respect to not multiplying two dollar figures but you could imagine that they did. So what's the application? The application is that once you have the accounting system, the mathematical accounting system, you can build tools from it or you can just think through it on the whiteboard or something. But if you build tools, that's a major way people use math calculus or whatever they simulate things. If you use category theory, what it lets you do is if you say, I have this monoidal category called Polly and then someone says, they wanna implement it. Well, instead of thinking about all the features they want from an interface language, and they're like, oh, and let's have it print like this and let's have it publish the web like that and like, they have all these features. What it allows you to do is not look at any feature requests. You just implement the accounting system. You just put it in there, just the math and you call it a day. And then if somebody else wants to add stuff, you do it later and you don't use that to influence kind of the basic code. It allows you to write very, very highly structured and very compressed code that does all of the major backend work of the thing. And then every other feature request is like, that's kind of the gold plating or like the thing that makes a user interface for it. But does that make sense as an application? Yeah, and like perfectly segues into my next question, actually. So you talked about category theory as like a potential accounting system. And early on, actually, I think you even mentioned morality in your talk, but even before we were talking about intelligence and like grounding in ethics and wisdom and I'm now like in kernel studying regenerative finance and potentially like new ways to develop like accounting systems that maybe involve like carbon credit or some kind of moral or ethical bound. That's like a new system for like not just plus and minus dollars, but maybe like a new system for accounting that like a ledger that allows for more thoughtful consideration of like the environment and just the world at large and humanity and our fate and future as a species. And I just wonder if you've thought about that or and I definitely think like, you definitely think that category theory can maybe jump in and rescue us here or just what are your thoughts on that? Yeah, so category theory, you have to do the work of deciding what your accounting system is in some sense. Like it helps you decide what things, since it has talked about all of math can be written in category theory at this point or at least all that I can think of. It's got all, all of math is the stuff that people have used so far to think about the world. And so it's somehow are getting all of that into a single system. So it's likely that whatever we're thinking about now with this regenerative finance, et cetera, if there would be some good way of putting into category theory, but that still has to be done. It won't rescue you, except it'll be there as like, yeah, I think I could, if I had an expert to work with, I could do it. But then in terms of the question itself, regenerative finance and that's and morality and stuff like that. One, I mean, I don't know if this is the sort of thing that jumps to mind, but one thing I think in our society that might be pretty bad and is in ourselves also or at least maybe it's good, but it's good in a way I don't understand yet. And that is that in these big arrangements, there's little things whose job it is to hide the intentions of the things inside. And so like I think of, I'm not sure I'm gonna name this person and it's almost a trigger, but Jeffrey Epstein, I think like, what was he good at? I kind of think, I know he was known for like making these financial derivative instruments that like hid the subprime mortgage stuff. And I feel like his whole island and all the stuff he was doing, like he just knew all the secrets. And I could imagine backing away from him, but like also like the meat industry, there's secrets about how meat is produced. And when you go to the supermarket, their job is to make sure you don't think about that stuff. And so when you form these big arrangements or the financial system, like there's a lot of what goes on in the financial system that's hiding the almost slavery that takes place in various places. So, and then in myself, maybe the greed, I think all this is really coming from the need to survive at all costs or something like that, that seems like our cells in our body don't have, but we somehow have gotten. I'm not sure about that. So anyway, one thing that comes to mind is like, what's up with hiding intentions? And is there any place for that? Wow. Also, just like from a very biological perspective, I do think that we have little things inside of us, maybe because that function it is to hide things, but I definitely don't have access to all of the knowledge of every single cell in my body, right, or every even organ. So, I mean, like survive at all costs, like yes, I could be like knock on some wood, growing a little cancer tumor somewhere that's not known to me. And like we have mechanisms that mask pain and that even mask like proprioception or, so there's a lot of things going on, like there's a lot of hiding even in the biological system that happens between scales, which totally leads into my next question so perfectly, which was about compression and elaboration. And so like what is hidden or what is compressed, right? And what is the relationship or function or utility of category theory in elaborating compression and elaboration? Like if I get the compressed version of like what the larger box container is, what's happening with the larger box? Can I elucidate the structure and what's happening in the smaller boxes or the even the arrangement of the smaller boxes in the, within the larger container, the larger scale say? And just, so how much, how good is this math? Is it gonna, how is it gonna help me understand compression and elaboration across scales? Yeah, cool. I first wanna say that I think of category theory as humanity's best thought compression language. I didn't, yeah, so for example, the category of polynomial functors is the whole subcategory of functors from set to set spanned by co-products of representatives. And now I've told you the entire definition and I don't expect you to understand it and I wouldn't even expect of category theories to understand it. But I would write that expression that I just said on the board and it would be, I don't know, 20 words or 30 words. And then that would be sitting there at the top and then I would go through like what do I mean by category? What do I mean by set? What do I mean by functor? What do I mean by spanned by? What do I mean by co-product? And what do I mean by representable? And I would go through those words and I would explain whichever ones of those you didn't understand and then I would explain those in a compressed way and then you'd say, yeah, I get that or no, I don't get that. And we could unpack this thing. So the category theory itself before we get to your question about the interfaces and arrangements is this amazing thought compression language where all the stuff in this talk, all the math behind it, and the stuff is an all on elaboration of this very, very compressed few words because we really made this language for compressing big thoughts. Then there, if I remember, I hope I'm not missing the main part of the question but you were also talking about the arrangements themselves as compressions. And yeah, like an adder circuit is a compression of a bunch of and or logic circuits that are themselves a compression of NAND gates which are themselves transistors arranged in a certain way, a certain way, a certain way. And the NAND gate, if you think about it why did you arrange those two transistors that way? Because you knew that it would always act this way. You can kind of like everyone's an ending all the time but or is a pretty good thing to think about and and is a pretty good thing to think about and plus is a pretty good. And so these compressions out of all the possible ways of arranging transistors into the number of transistors in an adder circuit that can add two, three digit numbers or something like that, that might, I don't know let's just pretend that took a thousand transistors. If out of all the ways of arranging a thousand transistors there's like a few that you would actually like that where you're like instead of adding two numbers and getting like some random thing you get the actual sum. And so we compress into, for some reason some compressions are useful because we can predict them. And like I can kind of predict your behavior based on what I'm seeing out of on your face and what you need to know from me or what I can do to help and vice versa. And like that ability to compress all this stuff into just something you output on the interface is really important. And the same is happening in the or gate and the adder circuit and stuff. So you kind of create things that have a well-defined usefulness even though I have no idea what that means. And I don't know, does that answer the question or did I miss a major part of it? So let me just repeat a part of it. So if I am a compression, say like let me just do something really simple. I'm the number six and I'm a compression of, you know, three numbers like, and it could be so can I, can I, I guess what I compress is the set of the possible numbers that make up six like the three, like that's what I compress, right? So that's what translates to the compression. So you can never really go from the compression number six to figure out which of those three numbers were, right? Right. Right. You can, what you do know is the remaining thing that sickness gives you. There's lots of things that use six and they're all happy to know about six and same with or like, I don't know how you made your or you could have made it with 43,000 fan gestures and wasted a bunch of them. But like the point is I don't need to know all that stuff. All I need to know is that this thing's gonna take two Booleans and produce a Boolean in a certain way. And so the compression is outer interface is saying I don't care about all your private methods, just tell me like what can I expect from you? And I think part of like, you know in our air society we have identities. I'm guessing that identity, which is such a major thing today is like this compression of like what's expected from the person. And we are working out like how those identities really do help us expect things from each other. So just on the topic of identity and like to follow up and maybe continue this train of thought a little bit. So like I am blue, I am a human, right? Like I'm a human that's alive, that's here talking under the compression of, you know mass stream number 4.1, which all of this conversation will eventually compress to active Institute mass stream 4.1, but like the fact that I am blue and I am human like I am one among the set of many things that is all human. And like we described the set like living human in a certain way, like I have a certain number of cells but many of the cells are not human, right? Like so more cells at sometimes of your, you know digestive cycle than others, more cells sometimes are not human and actually are human. And so it's the set of like things like how many human cells, how many, you know a different E. coli, how many like, you know different types of bacteria and microorganisms and like all of these things compressed to the set of like living human. I would like to be able to unravel like what is like go from living human or like even just find the limits of like, okay. So now like too many E. coli was like some decomposing human that's not alive. So like I would like to be able to really push the boundaries of or even find the boundary of what makes the set of cells that is a living human versus like when do you, it's like the phase transition like the critical point at what critical point are so many of these microorganisms alive that I become not a living human anymore. Yeah, yeah. So in the same way, like an and gate is not made out of and gates, it's made out of smaller things or, you know, a logic, all right. So we don't really expect the high level thing to be made out of little things. But the reason I think you're saying that there are these things called human cells is because we share this DNA and the DNA that you're tracking when you say that, hey, the E. coli is not me and the bacterial gut, gut bacteria stuff is not my me because it doesn't have our DNA but it is you in the sense that it helps you as a pattern exist. And so if I was putting an interface around you I definitely wouldn't personally want to like only use your DNA cells and call those you because that would be, you wouldn't be a very good you. But yeah, so what can happen is like these E. coli or these cancers or whatever inside of you, let's say you're this gigantic arrangement of stuff that what can happen is that they can start to say things that start to mess with that arrangement and start to change the arrangement. So remember the arrangement is changing based on what's flowing through the wires. So your E. coli things in there might be saying some stuff or spreading some stuff that's changing the arrangement. And now this cell is no longer talking to that cell when it should have. And now that it's no longer talking to that cell because the arrangement has changed. Now that other cell is not receiving what it usually does. So it starts to say different things. And slowly this changes the arrangement so much that you're no longer a coherent whole. And the things that we can expect from you, your identity has changed to one that's like not as interesting. Definitely. And so can you see that in the, with the math, like where this maybe phase transition takes place or like at what level of rearrangement? Like maybe it's 20% rearrangement or like, you know, 20% incoherent arrangement or measure like the information in the circuit transferring from one to like from one piece to another such that like when the information flow is so disrupted then it like ceases to be a functional arrangement. Yeah, I think that problem is as hard as like being able to look at a spreadsheet and seeing if there's corruption. Like people can do it, but it's a judgment. It's like someone decides like, no, that's not real. Or yeah, it's like, I know it when I see it kind of thing. So there might be rules of thumb you could create in this accounting system. Like, but I doubt, I don't feel comfortable creating like the definition of the phase transition. And I wouldn't expect it to be any more than a judgment, but yeah. Well, thank you. And thanks for letting me kind of ramble on a little bit there. So on the slide while we're here, this is totally changing the subject. If you were Roman, like if you were ancient Roman using ancient Roman numerals, would Hindi Arabic numerals still be empirically better? Or does that depend on context? I think they would. I know that it must not be in the current culture, but I think it is just empirically better. Like in the sense, like is a better baseball player better? Well, they're better at baseball. Like I'm not saying that it's better. Like maybe you just love these things and it makes you feel really good to see X's and stuff. And then for you, like it's great, like use this thing. But if you want to be a math person that multiplies things, like apparently in Rome, like they knew about square root of two back then. They figured out like 1.414 apparently. I don't know. I don't know if that's true, but they were, Pythagoras was like back then, like can you imagine taking square roots if this is what you're using? It's just, you can barely multiply. Like how could you take e to a power and do, could we do quantum mechanics today if we were having to do this? Could we do even compound interest with this? Like I would never ever want, like I would, so it just wins. Like it just, it's empirically better because those who decide to move to this system will out-compete these. And in life, there is some sort of competition. So I do think that it's empirically better, but I'm happy to hear if you disagree. I mean, I do think like survival of the fittest, right? So clearly like the Hindi-Arabic numerals have one. But also, I think that if I had never seen a Hindi-Arabic numeral before and was trying to multiply that like I would prefer because it's like just what I'm familiar with. It's part of my model is to use these Roman numerals if I was in ancient Rome, and I would be like, what is this one for? And so it's just whatever I guess that you're familiar with, but it's need to hear your perspective also. I mean, like if you were a moth and it was turning, you know, everything was turning brown because of the new smoke, you might be familiar with being a white moth, but like any moth that changed to being a brown moth would win. I don't know, yeah, there's familiarity and that is a thing because we need that. We need homostasis and stuff, but there's, I don't know, yeah. I think that if someone was a more capable person in terms of being able to change, they would appreciate that faculty in themselves. Like they'd be like, ah, I'm glad I tried to change, yeah. Well, and so now I have a series of questions about the interface and the dynamics and interactions. I don't know if you want to skip forward to those slides. So are these like terms? Like you said, you had some key terms, interface, dynamics, interaction, arrangement, and I'm tracking. These terms, are they part of like the formalism of category theory? Is this something that like everyone is a category theory? Studying dynamics, they're not, okay. So that was my question was, you know, because the way that you discussed these interfaces, positions, forces, states, arrangements, enclosures, really spoke to active inference so much, which is why like I really get why you're like, oh yeah, that seems to like kind of mesh with my current ideas. Like so like we have the interface, which we talk about as like a Markov blanket, like a separation of boundary, and in the Markov blanket are like the sense states and the action states. So like in the interface, it is like you would call it positions and forces. Like that's where I act on the world and the world acts back on me, like in the Markov boundary. And so similarly like you talked about like the dynamics of a system as being hidden and only like internal to the system. And we have like hidden states or your internal states. And then, yeah, the interaction is like, what I would refer to as like maybe a state update. Like, so there's like, you know, we interact and then I might change or I might, it might just influence my like future trajectory. So I just wanted to like highlight the great degree of similarity between what you talked about and it's like what we talk about very regularly and active inference. Great. So what I did was I took, I started noticing polynomial punctures wanted to do what I was trying to do. And then I took the feeling of using them. Like when I, the intuition I get by asking myself what a map of polynomial punctures is or whatever and came up with these words as the things that most track and like most intuitive English words that tracked how the math was changing or how the math like developed as I made a story about these things. So these are really just the words that I found to most give me intuition for tracking what the math was doing automatically. It looks like Daniel has his hand raised. Yeah, thank you. Just great questions, Blue and David. Thanks for the very powerful and creative talk. I think on this topic of the active inference ontology mapping I think that was really quite well stated which is this idea of an input output system or an interface and things that are on one side of the interface are shielded from that on the other side of the interface. Otherwise quite literally it would be a different thing is a really powerful idea. And it comes up in computer science with APIs. It comes up in graphs and topology and we can even think about things like the arithmetic operations, like order of operations or nested parentheses. It's like, well, what happens in the parentheses? It stays in the parentheses until it's compressed and then it goes into some other area. And then also even how you described searching for intuitive words and I'm sure actively inferring with many learners for terms that have a natural interpretation or a conversational interpretation and also point towards more technical usage which I think in active inference we see a lot about system delineation and partitioning from the environment like such as the ones that you've brought up nested systems and interacting systems and also regarding some numerical quantities like surprise, attention and so on. Although also we have recently seen some symbolic active inference implementations from JF Closier that are engaging in this action perception loop without sending statistical or numerical messages but rather communicating about logic values. So just wanted to note that and if you have any thoughts otherwise I'll read a question from the chat. Yeah, quick thought is like somehow when I, we take language for example, Matt and we translate it into motion. So like if you say please pass the salt then like 10 to the 15 atoms will move through space. You know, you just say this word in language and then 10 to the 15 atoms move through space and like some salt crystals land on your food or whatever. And so by having the math attached into intuitive ideas in our minds we can then take the mathematical ideas and translate them into the movement of atoms so like me being able to talk or whatever. And like it's really important that we have intuitive ways of thinking about these things because otherwise they won't translate down into the movement of atoms that actually the action that we see in the world that actually accomplishes anything. So I'm worried about all the mathematicians that are doing math that they don't have any sense at all about how it could ever be used. And this is kind of the point of Tobos Institute is like you can, I don't mind if you do like okay you can do whatever you want but like I don't see a point in doing math if you have no idea how that would ever be useful to anyone, even in a thousand years. And so I'm, I don't know. So yeah, to connect the point is like to to connect with the audience here have this possibly be useful to anyone for real. I need to find the words that take the math and translate it into intuitive concepts so that they can come back to the math that they want to or whatever. But yeah. Lou, anything to add on that? Go ahead and read the chat questions. Great. All right, so this will pull us back generalize a little bit and then I'm sure we could talk more active inference or institute talk. So Ali wrote, it is said that category theory is the most abstract area of mathematics out there. Perhaps we can even call it a kind of meta mathematics. Daniel Dennett claims in intuition pumps and other tools for thinking that we can theoretically go meta on every level of abstraction. So is it possible to conceive a level of abstraction even more meta than category theory in mathematics? Yeah, great question. So there is a category of all categories. There is, the thing is that when you have a category of all categories, first of all, it's like really big. And second of all, it has an even higher level structure than each category does. So it's called a two category of all categories. And then there's a three category of all two categories. And there, but there is amazingly a thing called infinity categories that finishes this thing where there's an infinity category of all infinity categories. And I think that the abstract, like it's very hard to think about this thing. But, and it's still an active area of research to create like a nice infinity category. But yeah, people keep getting stuck on, hey, I've got all these categories and I want something more abstract than it. And then I need a two category and I, ah, I'm done. I got the two category of all categories. But then eventually someone wants the next one. And so, so yeah, this is a great question and it's an active area of research right now. Okay, interesting. I'll ask one active inference related question that connects also to enclosures and nesting. So we've in the generative models that we see in active inference, they're Bayesian graphs and they're describing control systems and action perception loops and so on. We've seen nested models be used in at least two different ways. One is to describe systems who are physically nested within one another such as an organism and then the organs, the cells, the organelles and those interfaces and what that nesting or enclosing can be interpreted as has a natural spatial interpretation. However, there have also been nested models of cognitive modeling flavors. And in those cases, it's never been clear whether the nesting or enclosing that's happening is reflecting spatial boundaries in some cognitive space if one chooses that direction of interpretation or in fact what that nesting entails and then importantly, why it would have a structural isomorphism with the nesting of the matryoshka and just things that are physically nested inside of each other. So what does nesting and enclosure mean here and is it referring to spatial boundaries or other conceptual boundaries? Yeah, great. So this, so like same with airplanes or something, you have the physical airplane but then different parts of the airplane are really talking to each other that may not be like the radio of the control tower and the radio of the airplane are in some sense more closely connected than the radio of the airplane and the engine of the airplane. The radio of the airplane and the pilot are strongly connected. And so yeah, there's always this logical, what I would call physical arrangement and logical arrangement and those might be different. But what they are from this point of view or one thing you could do is just, so if you look at, I don't know, this X12 and X21, you might imagine that there's a logical way of arranging things where these really seem like they're quite close. And so I probably should have shown this but that picture is the same as this one. I've just undone the nesting. Let's see if I can try to show that. So yeah, let's see. Oh, no, it's not. Oh, darn it. I didn't take the same one. That's confusing. Okay, well, I think I added an extra loop on this thing or something. Anyway, my point is we can undo this nesting and have this one be closer to this one and some other nesting. So you could use this organizational system called an operad to handle the fact that you might want to nest things differently for different purposes. This picture here is supposed to be either physical arrangement or logical arrangement. It doesn't matter. And it's really just about who's talking to who. That's the most important thing. The thing is that in your liver, in your brain, in your cell, maybe the latency of communication is much better than across the boundaries. Sorry for the voice issue. Well, very interesting. And in neuroscience, it speaks to the distinctions of structural, functional, and effective connectivity, which if only it was dominoes hitting each other and they were all identical. But even in that case, I don't think they would be identical. So we're dealing with something, and that's a great example, with the informational and the causal connectivity after all the Bayesian causal graph architecture is pointing towards something that is presenting itself as cause with a technical specific definition, although a natural word. And so it is interesting when the causal architecture may not reflect the physical architecture and vice versa. Right. And the notion of opera, which I never said during the talk, but this nesting thing is designed to allow you to rearrange how you want to consider inside and outside or whether you want to consider physical or logical to allow you to do that zooming on your own time or in your own way, without affecting what the thing is going to do as a whole. Thank you, Blue. So just to add on to the possible layers of nesting, there's also the layer of temporal nesting. Right. So as things happen at like a very fast pace that we don't observe, like the light is flickering as long as it's like greater than 60 Hertz. I don't see it flickering, but I know it's flickering. And as long as it's like not in my perceptible range, then it doesn't really matter to me. So there's, and just similarly, like I don't think biological evolution maybe cares all that much about mass stream 4.1, but cares about like, you know, how many children we each have and, you know, all these other things, right? So the relevance, the salience and across temporal scales is also something to think about. And we talked about like the changing interface. I think you mentioned that. And with the question of like, I don't know if active inference does this changing interface. Well, so it does in the Markov kernel, which we just actually like learned about and talked about when we were talking about quantum systems with Chris and Mike and John. So that was like, I think live stream number 40. Also on that technical note, I would suggest that the plain essence of the blanket definition does not entail any sort of dynamical rearrangement. However, recent work by Dalton S and others has pointed towards a definition of a blanket index, which reflects a weekly mixing dynamical interface zone that is on a zero to one continuum. And that may point towards the kind of rearrangeable interfaces, although not with the kind of category, theoretic nature as far as I know. But again, this is like a very interesting area. And I can see many applications in systems, including online teams and organization. And so I really liked that you used, what was that hands and eye for making some of these connections clear. And just one last connection. You had mentioned your applied, applied like trails that you're chasing right now. And you mentioned prediction markets. And I do think that, well, first of all, you talked about like, before you talked about that, you talked about the free energy minimizers, right? Like so if each little component in the system is a free energy minimizer, and how like within the FEP it can scale to form like a container free energy, free energy minimizer. So yes, and that like will stay a free energy, a free energy minimizer, as long as it decreases the free energy of all of the components in the container for it to stay a container, right? Like so that like speaks to self assembly and perhaps more for Genesis. And then in these, so yeah, consider how free energy minimizers and range themselves, there we go. And how they constitute a larger scale free energy minimizer, if it lowers the free energy. And then just in terms of application prediction markets, I think is a really good way to maybe align category theory with the FEP. And I think some of that has been done. I want to say like the multi armed bandit problem is maybe like a rudimentary example of that. And I don't remember which live stream that was. It was a long time ago, but we did have a paper dealing with the multi armed bandit problem and using active inference to solve it. And that might be like a way to start to align more. On the just on the prediction market area, then David really curious to hear what you think. I see a lot of parallels with what you described about transistors. Again, you said a thousand pieces components in this transistor. The space of the 1000 pick and connections might be vast. And so that's where like selective processes and abductive logic come into play with this two stroke engine of like diversifying and exploring the space and printing down based upon function. And so we could imagine prediction markets where the nest mate level, the colony participant is making great predictions, but then they're being compiled in a way that's actually inadequate. We could also imagine the other extreme where no nest mate knows how much food the colony has or what the weather is outside, et cetera, but it's natural actions still participate in a colony or in a market that makes adaptive decisions. So in that structuring of multi scale sense making, the composability and the ability to create systems that are valid across scales and don't enforce any kind of implausible constraints, it becomes essential because people point sometimes to wisdom of crowds and all these other related areas. And there's probably some areas and some ways of doing it that are just incredibly effective. And I'm sure we can think of crowds and ways of doing it and questions where it is just not that way. Yeah, so in this work on dynamic organizational structures, there's a kind of fractal like nature to it where the same thing, like there's one formula for the whole thing. And so in order to be one of these things, like in this technical sense that we wrote up, it can't have all sorts of crazy stuff at one level that doesn't exist at another level. It would like we, this is not supposed to be an indictment of it, but we tried to make a more socialist example of the prediction market where the wealth, the updated wealth would be more evenly distributed. And it wasn't hierarchical. Like it didn't have the fractal like property where what would happen at the high level was coherent with what was happening at the low level. And so it could have been because we weren't creative enough or whatever. But my main point is that there are rules to this system that say when it's going to kind of align across the entire structure. And when it does, then you get a very short description that happens at all scales. And so what this kind of tracking system allows is to say, I mean, you don't have to trust that dynamic organizational structures are a good system. But if you were like, well, we need something to guide us here, then you could use it to say, well, this way of assembling predictions to make a higher level prediction just doesn't work as one of these dynamic organizational structures. And therefore we're going to deny it as the way we're going to do stuff because they're just not going to work when we do multi-scale things. These four examples do, so in heavy learning, for example, what we mean by that is spike time-dependent plasticity where like what happens if I hear you say the same kind of valence as me, but you said it like one second ago, whereas I'm saying it now, then I want to wire to you more strongly. So this kind of like just this kind of time horizon where I'm listening for your agreement with me and if you did before me, then I want to wire to you more that itself. There is a way of putting those together that has this kind of fractal structure. But there's tons of ways of doing it that don't. And so that's kind of what's going on here. In what you just said, with a short description across scales, I see it reflected in that definition. You wrote up on the chalkboard that could be uncompressed and in past assault. Past assault, please. And then in the organizational setting, that kind of multi-scale decision-making and ability to address a kind of integrated stack of scales such that there is a compression like I am running, which is describing a coherent set of actions ranging all the way down to the cellular and molecular, that can be semantically compressed in a verbal or linguistic community in a certain way. And there's other people who you say I am running and they're going to look at you like, what language are you speaking? So it's still as contextual, yet we can have these multi-scale descriptions. And then I have a note from Kevin in the chat. In case it hasn't been noted, it might be worth mentioning to David that Markov blankets in active inference are just a kind of Bayesian network directed acyclic graphs with probabilities assigned to the vertices for all those who will be following this trail and working to build this relationship, hopefully between category theory and active inference. Thank you, Kevin. Cool. Thanks. So very cool. I think we're coming up on time. David, do you want to tell us maybe a little bit about, or if you have any questions for us about active inference, we would be happy to maybe attempt to elaborate. Or also if you want to tell us a little bit about Toplus Institute and what your goal and function or how you came to self-assemble in that form, that would be great. Well, sure. Yeah. Basically, in academia, there's no... My co-founder, Brendan, and I wrote a textbook. There's no interest in textbook writing in academia. I especially like to help people learn a new subject. There's no reward for it. There's no reward in math for making tools out of the math or like trying to help people either learn it or use it. And so basically, I'd been at MIT for eight years and then my department head said, we now have a new policy that you can only be here five years if you don't have tenure. And I didn't have tenure. I was a research scientist. And so he gave me two and a half years to go. And so I was like, well, okay, that makes sense because academia doesn't care about, about the stuff that I'm really trying to do. And so Brendan and I made Tobos Institute because we actually want the stuff we're doing to help the world make more sense of things. Like we are this gigantic, we're a network of people that the world that has to steer, that has to decide is climate change real, you know, is Obama a Muslim? We don't know these answers, apparently. We don't, we can't figure these things out. And that just, I think it's because we don't have the communication to do that. And so I just want like, I think there's just, it's not because I think there is this greed thing in everyone or this need to survive or whatever it is, this hiding of information that like is probably causing problems. And I don't really understand it. And you're right, like we don't need everything out in the open, but mostly I think what's really causing problems is just lack of communication. It's like my greed, if I understood how it was affecting you, maybe I wouldn't need to do this particular one. And like if you understood how the scientists came to the idea of climate change and you could like track that to the level you wanted to, you could like actually, you know, understand these things, then maybe you wouldn't question it. It's just that like you haven't done the experiment for yourself and you're not trusting that it's done the right way. And so I was hoping at least that the more we can help people communicate and understand each other and have high fidelity accounting systems so we can like be accountable for the more we can solve problems together. And like since that wasn't being rewarded in academia itself, we needed to go outside and position ourselves kind of between academia and industry and society. So that's how we decided to form Topos. That notion of accountable communication is really powerful because we see cases where communications are accounted for metadata logging and peer review and where are we accountable socially? Where is it a digital accountability? And the accounting schemes are a skew. And so people are looking at their shred of the spreadsheet and uncertain on how those numbers were derived whether it's the whole picture. Do you have all the sheets in that file? Like with a partial picture and yet the space of communication strategies just like that thousand part transistor board was vast? Well, there's more than a thousand people. And so the space of communication simply edges, let alone quality and tone and content is so vast that again, it emphasizes the need for a vocabulary and a grammar and a syntax of composable dynamic organizational structures so we can even begin to be working not just in this one very local potentially reason in organizational space. Yeah, I totally agree. I mean, in computers and the Internet we have very composable things that you can really make math out of and like at every scale in some sense there's like the same sort of thing happening. And this allows computer scientists to talk to each other. And the guy working, someone working on like Internet protocols and someone working on transistors on some level they can kind of talk. But like an urban planning person and a biologist, geneticist and a mathematician and like, I don't know, whatever, like these people across society they're like talking in wingings at each other. Like they're just like, you know, mailbox envelope ampersand and like they don't understand what each other's talking about. And that we're never gonna, if we can't cohere, like there's a solid little languages and they're not cohere together. And in category theory, there's tons of languages. There's a language of topology. There's a language of algebra. There's language of logic. There's logic. You know, there's number theory. There's tons and tons of languages. And each one needs to be isolated, like needs to have its own space so it can develop notation for itself. So it can develop techniques for itself and not have to like ask anybody else's permission. And what category theory and what category theory and same with like urban planning and genetics, like they don't, the geneticist doesn't want to say like, Oh, sorry, I used the word, you know, structure. And I know you're already using the word structure. So we'll call it something else. Like they don't want to like have to do that. And yet in category theory, what it allows you to do in math is take topology and make an analogy with number theory or with algebra and actually send information that can lands intact because the functor tells you what you're expecting to keep and what you're expected to lose across this connection. And anything you're expecting to keep, it keeps. And so what we're hoping for is that by creating like finding languages that you can use across lots of disciplines from, say epidemiology to chemistry or whatever, we can create single like languages and then connect them to other languages, database languages and physics languages and stuff like that and allow them to each evolve independently and create their own independent, you know, notation and techniques and yet be able to have that interface that compress thing, send something across the boundary to somebody else and have it arrive intact in a trustworthy and accountable way. Awesome. Well, David, thanks for hanging out with us and talking to us about category theory and maybe come back and talk to us somewhere about it. And maybe we can align deeper on the relationship between the type of things that you're discussing and to active inference. That'd be great. That's great. Yeah. Thanks a lot for the invitation to be with you guys today. Great. Thanks. Thank you, David. Goodbye. Bye.