 Hello, I'm Hayato Saigo, okay. Just wanted to make sure it's going, it's all good. All right, welcome to both. It is June 13th at zero UTC. We're here in active guest stream number 44.1 with Hayato Saigo and Nausugu Suchia. So really looking forward to this presentation and discussion. Hayato, please say hello. And then we'll continue with Nausugu's presentation. Hello, I'm Hayato Saigo. I'm a mathematician and a mathematical physicist also interested in consciousness. I have worked a lot with Nao by using category theory and application to IIT or other consciousness theories. So Nao. All right. So my name is Nao Suchia. I'm a professor in Monash University and also having a lab in ATR in Japan. And as a sort of the introduction to this, to the session, I'll give a brief talk on our recent project called the Korea Structure, which Hayato and I will be collaborating and also a little bit on the quantum-quadra hypothesis. Both of them are hopefully relevant to active inference issue. So the Korea Structure project is the Japanese funding that we received recently. And it's going to continue for five years. This will be to establish a super interdisciplinary research program to tackle the program of consciousness, especially Korea through the interdisciplinary research. And we are planning to also host summer school for two to three weeks every year. And here's the fantastic members for this grant. And Hayato is basically working with me in the part of the mathematical approach to the consciousness and the Korea. And so one of the problems, for example, we want to address is, for example, when you see this square, Daniel, what do you see here? Blue. Yeah. So you say blue and I say maybe owl in Japanese and Hayato may say suni in Russian. But we could mean the same experience, Gloria. But how do we know that we are actually experiencing the same thing? And I'm actually curious whether this kind of question is something that the free energy or active inference kind of style approach can ever tackle. And my approach is to go into the category theory kind of inspired direction. And here's the first study that we published a few years ago with Hayato and Shigeru together with also another category theorist, Steve Phillips, that in the category theory, there is an interesting lemma or the Yonezarema that proves that all the relationship characterize its internal intrinsic property. And we are using that idea to study the Korea. That's the kind of inspiration of our approach. So instead of trying to characterize what the blue is or red is, we are trying to characterize all the relationship between a particular color or a particular quality to other exhaustive. And then by finding the correspondence of this relationship, we try to identify what the actual blue is. So if you would say blue is similar to green but also similar to red, then that's going to be very different from my blue. And that's the way how we detect this blueness. It's same or different. And one recent application of this kind of idea is this unsupervised alignment that we are using with another collaborator, Masahumi, always me, in University of Tokyo. And I'll skip the detail, but basically what we are doing is to do some kind of experiment and then collect data to obtain so-called similarity matrix between all the colors. And then without really knowing what color maps onto what, we try to use only relationship to predict what we are seeing. So in a sense, this is a purely category theoretical kind of approach to the problem of consciousness. Then I'll just skip the details here. And then, yeah, so we are doing this kind of approach on the other types of the Korea. And then this is the thing that we are planning to do for the next five years. And then another just brief kind of introduction. If you have some other question, we can come back later. So this is potentially more to do with that active inference aspect of our approach in a sense. This is called the quantum Korea hypothesis. We published this preprint really recently and so you can access it later. So there are many programs in the current models of our Korea or our consciousness where we tend to think that experience or consciousness is like one point in the high dimensional space. And we use that kind of idea for our previous alignment experiment approach. But it lacks dynamics and it also lacks the effects that we tend to experience where the observation or report or introspection itself sometimes changes our Korea. And that's the sort of the active inference aspect that I'm talking about. Attention or introspection changes our experience. But the traditional approach is very difficult to deal with this. And one way to deal with that is this quantum formalism that we think is appropriate. So here we think that there is some kind of Korea experiencing system that is defined as this blue boundary. You can consider that as sort of the Markov blanket like thing in the free energy kind of idea. And then we consider a particular type of the quail like redness or blueness itself as an observable. It's a terminology in quantum theory. Something that in principle can be observed within this Korea. And then we consider this measurement as a process that probes this Korea's observable. And unlike classical theoretical thinking, we think that the measurement or probing of this observable through it's called state changes its states and also observable itself. And that's something that we can actually formalize mathematically using a concept called a very general concept of this conditional probability called instruments. It's very technical for the quick talk. So I'll leave it, but this is a modern quantum theoretical idea that captures both how we can affect on the world and that quality itself can be affected by that measurement. And then using this, we think that we can explain probably Korea in a much more reasonable way with the mathematics and mathematical formulas. And so far it's just a kind of theory, but we already think that it's probably better than the existing kind of a model of the, for example, attention and the consciousness and also it also provides some kind of prediction and explanation of why some of the two concepts or two experiences can feel different depending on the order that you experience. So this is just a conceptual kind of example. But if I ask you, Daniel, that is China similar to North Korea, then you might say probably no, right? I don't even know the reference point. Yeah. Okay. But let's say, you know, just subjectively, you know, China doesn't feel similar to North Korea. Okay. Yeah. However, if I reverse the order, is North Korea similar to China? Then many people actually start to say that, ah, yeah, it's kind of similar. Hmm. Does that have to do with one being larger or what is it about the order that primes people to respond differently? Yeah. So many people have already, you know, came up with many different kind of ideas to do, explain this kind of individual aspects of this, you know, order effects. But as we mentioned, you know, we are actually computing or, you know, measuring this kind of order effects systematically through all combination of the colors, for example. Then it's hard to systematically explain why this order effect emerges in one case, but not the other, and in which direction, right? But that's what we found basically. And but using this, you know, quantum combination kind of framework, we can explain it really nicely. Hmm. In this case, it's just a projection substrate, but we are probably going to extend much more in the more instrument like, you know, a framework so in the future. Yeah. So, and also we can, we are also predicting that there may be some kind of a violation like a violation of inequality like situation in, you know, perception. And if we can confirm that, that also, you know, gives a credence to our proposal that the quality is like, you know, needs to be framed or explained in the quantum framework. So I think that's probably enough to start up the conversation, I guess. Awesome. Yeah. Okay. Great. A lot to jump into. I'll share my screen so we can take some notes. Okay. Hato, do you want to give a first remark? Or I'm happy to, but do you have a first remark or what parts of the work do you want to reflect on first? I think maybe the categorical approach, the importance of the categorical approach is one issue to talk with you, I think, because you first pointed out that paper with Shigeru Tabuchi in 2060. So I'm interested in why you and your colleague are interested in that paper from the active inference side. So can you explain some feeling? Yes. So this is using category theory to assess the relationship between consciousness and integrated information theory with the two of you and Tabuchi from 2016. So I've long been interested in scale free or scale independent ways to describe systems and information theory has been recognized as one of those approaches. And information theory has been proposed to not just play, of course, a central role in the functionality of information systems, but with IIT and related theorems to also strike at the heart of the most quintessential aspect of our conscious experience. So that seems like an incredible synthesis of the functionality of systems that don't necessarily seem to have anything qualia like at all merely performing calculations like a super fast abacus basically. And so I am curious and feel free to address either of these things. When we describe information systems, what is it that would or could or is experienced? Are they experiencing their information what they are processing or what is it about information integration would grant a system the ability to move beyond a behavioral functionality into again strike at the heart of one of the most ineffable questions that the mind can really consider. Okay, interesting. Actually the category itself in mathematics originated or created in mid-20th century to breach the two different fields that is geometry and algebra. More specifically the topology you know something about topology maybe which is a very general kind of mathematical methods to capture the essence of the figures like a circle or the complicated figures and the fundamental problem of the topology is can you identify the two different figures essentially that means you can you transform continuously to the other figures or not. So if the one figure goes to another figure with a continuous transformation and back then it is viewed as essentially the same. But to determine the two figures is the same is a very difficult question. So like great mathematicians figured out the utility of the algebraic method that like making the one figure to the other from the figure you can construct the algebraic system like groups or something and then the transform the problem into the algebraic region. Then the point is very essential point is taking one figure to the constructing one algebra from the one figure is not sufficient. The point is the relation should be transformed the other regions. So one region here other regions here but one to one correspondence of objects is not sufficiently but the relations if you can transform the old relations into the other regions then you can provide you can prove something from one region to another region because yeah this is a notion of the factor like the region intuitively I talked about the term using term the region it is mathematically called category. Category consists of objects and arrows or morphism which is a generalized kind of relationship between these objects. So topological region is some kind of category of topology like a top that objects are topological space and the continuous functions continuous mapping is arrow or something the other regions for example algebraic region like groups and group homomorphism or the module vector space or something so something algebraic and the relationship between them. So category is defined to formulate the intuitive concept of bridging something, functor. So first the Arlenmagen-McBren define category to define this notion of functor by the bridging to region then if this is established then you can infer about topological region category by using the algebraic the relational something. So we that is now Shigeru and me talked about the possible application of this kind of argument into the consciousness study that it is just a problem of bridging different kind of regions like a physically testable region and our role experience itself or something so it is a apolia it seems apolia it is a too difficult question too hard problem to connect to these regions but if you only focus on the object but the relationship if you can compare the relationship then you can infer something about the you cannot directly touch you directly manipulate. So this is the idea maybe I'll just add a little bit so yeah as Hayato said under you you are taking really diligent notes so that may be quite good but from my end so this paper was inspired by really talking to Hayato and so Shigeru and meeting in this summer school on consciousness and there I encountered in this category theory for the first time and there were several shocking kind of you know concepts that I learned from this first encounter one is that there isn't a way to compare two completely different things and then extract the common structure between them and Hayato first introduced me this you know very strong strange paper which I didn't understand at all but it was so interesting that I could read for hours and hours it's called the Rosetta Stone it points out the commonality between topology, quantum physics logic and computation yeah that was the title and then I didn't understand what the heck is this paper but that uses this category theory and as a whole I feel like I see so maybe if we understand the structure behind each of this concrete example we may be able to find the commonality or explanation at that level and that naturally applied to my program at the time which is how to bridge the gap between the quaria and the information structure derived from neural activity this looks also completely different but if there is any possibility to bridge the gap it must be some kind of structural mathematics and that's this category theory and one point is another point is that you know I think you can get the ideas and functions by bridging two regions then the really interesting thing another really interesting concept is called natural transformation natural transformation is the kind of deformation or transformation between transformation so you imagine some theory bridges some region to the another region so there is a one meta-level R code but another theory predicts another thing or something but can you connect these theories or models this is a program of natural transformation so one representation one theory or something may map the one region to another region but there is a there may be another various kind of mappings, functions so the problem is to compare or to unite to connect this transformation is a function is called natural transformation if the function is naturally isomorphic that means there is a reversible natural transformation between functions then these theories seems different but essentially the same so kind of natural transformation is maybe the hint to connect the theories on consciousness or something and also the our consciousness experience itself will be connected to the concept of natural transformation because I see something the different thing something but there is no divided discrete kind of structure but there is continuous or relating something so this view is something like another view is something like but there is no division there is some relationship so our experience maybe itself a kind of product of natural transformation like maybe now you can explain like a maybe now you are muted is it alright if I draw some kind of figure of course would you like to share your screen and also if you can give me the access to the annotation okay I will make it so everyone can edit and I will put it into the chat here and then also if you want to draw that would be awesome looks good so just to kind of recap what Hayat was talking about is that we tend to think about something like a category and then another category is there and then there is some kind of object which could be quite abstract and then it's related by arrow like this and then if this category C is somehow possible to map onto this category D while kind of preserving this arrow for example this arrow goes to here and then this arrow goes to here and then each object is also you know preserved then it's much more than the usual kind of idea of a function between the sets right because it also imposes the correspondence between the relationship and in our idea basically if this is let's say the quality of the category and if this is a neuronal activity or maybe information domain and then maybe you know we can have back and forth of the function that would be great usually the most of the mathematical theory of consciousness tries to go from here to here right let's say in the case of IIT arrow here and then what Hayat was kind of talking in a way as a natural transformation is this kind of you know secondary level arrow let's say I don't know whether free energy or active inference can count as a prediction from you know neural activity to that particular quality but let's say if there is something like that and then maybe there is some kind of a translation between these two theories if that exists that's called a natural transformation awesome well a few comments on this the idea of mapping the maps between the spaces is so important it reminds me of metaphor where we're conveying relationships and saying well it's like a hammer or is a hammer for this other situation how we get few shot learning and applicability and how we grasp concepts is perhaps developmentally or from an evolutionary perspective in a large extent by generalizing our spatial and embodied understandings and then certainly communication if you want to have some semantics that are being communicated across minds through an interface through a classical interface then certainly you want the structure to be preserved and then I think the one that you have here is one of the most provocative for those empirically studying consciousness because it's kind of like this side is empirical and let's just say we don't know what the Q side is for now but let's just say it's another space that isn't that and then if there is a certain landscape that neural activity patterns flow through like some kind of winterless competition or you get some sort of limit cycle and you find that there's a limit cycle of neural activity and then there's also that is being paralleled by experiential cycles then is not just an accounting system that's a very strong finding about potentially the real underlying causal system especially because it can suggest unique other explanations or manipulations to make Do you want to mention anything about the metaphor? Before that very hard to say the technical remark is that for the original problem that connects topology and algebra in that case the algebra is more concrete more manipulatable in that case and topology is something mysterious in this case so in general the function is a kind of generalization of mapping in sets so there's something mysterious into something concrete so now picture C and D should be switched maybe in that case but of course there is a back and forth so the idea is your blue and my blue program is for example you can prove that your blue and my blue is different if there's something informational or something detectable structure is completely different then your blue and my blue is different of course you cannot prove the sameness between your blue and my blue if there is no how to say back and forth function but even if you don't have the back course function at least you can demonstrate that your blue and my blue is different seems to be different if the correspondence is functorial or something so it is a big step from the experience side and with me and colleagues do such a thing so your blue but this is not by focusing your blue my blue only but your blue read something or you don't have to say even do not have to say the name of something so just no labeling for example but you can teach the relationship these things then compare that so very interestingly in general the structure is surprisingly coherent between persons so it is very nice and also maybe it is suggesting that the non-isomorphic case then your blue and my blue is something different so I think this approach is very interesting even if there is no bi-direction only one direction then even that case it is useful it is my remark please this is one of the slides that I didn't show today what Hayat was talking about is kind of explaining this figure the way you experience red is related to the other experience the color but shape and sound in a very complicated way and then we can measure how similar they are by asking the people and then you can also do that for the other people and then these two similarity can be directly compared with our new method of non-isomorphic learning and IIT for example may allow us in the future to go back and forth between this quotient structure and the information structure and then mathematically we believe that the information structure should be potentially translatable in some way and if that is possible in some way eventually we may be able to say the way you are experiencing red is same or different this way or that way that's a sort of idea can you go back to your drawing awesome so for the top arrow just exploring what might it mean that IIT is going back and forth and also that active inference is going back and forth what does that mean C and D represent in IIT is that a quantitative evaluation like the five variable or what is being accounted for with that arrow I have to do you want to say something there is various possibilities for example the sameness or difference so if you have some kind of gradation of sameness so yeah sameness gradation but to formulate the gradation gradation of sameness you have to move on the enriched category not a simple kind of category but it's generalized kind of category called enriched category in that case for example the simplest example enriched category is generalized generalized matrix space matrix but in matrix space for example the point P and P itself has a distance zero of course and kind of P to Q R is there is a triangular inequality or something so yeah then it can be considered as a generalized kind of compositional structure called enriched category it is a typical kind of enriched category so but symmetry does not always hold or something so this is one how to say one small step to grasp our core structure because like now pointed out North Korea, China distance and China North Korea distance is in general different or something so almost all studies assume that some symmetry we need to go beyond that and then maybe the one number or something is too poor to represent everything so you have to go more and more richer structure like not just numbers but some kind of projection operator or something then you enter into the quantum region so relationships we feel maybe should be represented from some kind of relationship between the notion of state in quantum theory and state transition between that so our approach is step by step and we use the many arrows to study something and then goes to unify the more and more general kind of arrows and categories and this is just for the speculation but this kind of quantum theoretical like as a mathematical framework do not need to but the kind of mathematics can you use unify these kind of things awesome just a few comments on that when we're doing free energy minimization we use the KL divergence which is very importantly not a distance because a distance would have the symmetry characteristic and so the KL is chosen because it's easy to fit optimization on these variational distributions but it's almost like it's a feature not a bug that the KL divergence between Q and P is not the same as P and Q because it turns out that that kind of a divergence based quantification actually has some advantages over distance measure and then the mapping between observables and hidden states and the cognitive operators that we bring to bear on that kind of mapping problem those are some of the most fundamental pieces of the partially observable Markov decision processes that are used in the active inference generative model which is first the mapping between observables and hidden states and then how do hidden states change their time and how does action selection policy selection influence how hidden states unfold through time so this mapping between hidden states and observables and between hidden states and the hidden state at the next time point are and how again how action influences that if you understand those operations then it's like a game of chess and the ball just rolls downhill and the variational distribution works exceedingly well but when those are off base then you're fitting your way like right off a cliff so maybe then I'll just add one more point so you asked in the original like 5 minutes ago what each of these points or arrow correspond to in the case of the IIT and so on and then this is kind of related to all these ideas but one of the fundamental insights from IIT but also from philosophy especially some kind of philosophy called process theory it is important to really understand that the process or flow or movement is really fundamentally important kind of aspect of things and then that may be really distinguishing some kind of static sets or the thing in the static world when they start to interact and then when the process emerge then something starts to potentially generate or support subjective quality like proto-quadra kind of thing in the case of the IIT it is deeply entrenched in the formation of the transition probability matrix of an entity like how this thing neuron or logic gate it might have come from the previous state and it may go into the next state so any kind of a moment is already embedding the possible previous and the possible future there is some kind of a width in the moment because it's a process and when the process is considered then as Hayato said that this kind of dot is a thing kind of structure to think about so we need a more elaborated structure of a dot itself but the category theory is great in the sense that each dot can itself contain complicated structure it's just you know represented as a single dot but it can contain something more so in the upshot what could correspond to a collection of the neuronal states and the connectivity pattern and things like that which can go from one to the other and then this kind of movement may correspond to something like we see redness or whatever in this way but that's also with respect to other kind of arrows as well the wave of the relationship that we are trying to characterize both in the domain of the aquaria but also brain is also the wave of the relationship gigantic ones right and as you said that you know transition from one state to the another state that's pretty much like dynamical state dynamical system like you know approach and then the simple idea called is applicable to this kind of dynamical system approach correct me Hayato if I'm saying something out of the bounds of my comfort zone but any kind of state can transition into another state and then this transition itself can be represented by the arrow and yeah so kind of the combination of that particular kind of arrow composition may correspond to one experience as well so that's one thing I wanted to say the other thing that Hayato also wanted to mention about observable states and why we are interested in the quantum you know framework quantum theory by the way it's not a quantum brain hypothesis we are not saying that you know quantum phenomena in the brain is critical for consciousness but it's just a mathematical structure of the quantum theory that is important to understand the qualia and the essence is most likely to be this you know non-commutativity any kind of process that you do like you know cooking you know putting the water and then model is different you know model and then putting water right and so any kind of process we do there is some kind of temporal dependency that commutativity doesn't hold and that's also probably the case for the qualia and if that's the case then for you maybe it's a bit surprising but you know Bayesian kind of statistics and Bayesian probability may not work as you might wish yes so in other words using the quantum mathematics there is a quite good generalization of Bayesian innovation or Bayesian kind of inference yes actually the one step generalization is called projection in kind of mathematics there is more generalization called instrument so instrument is general notion of measurements like a system and environments interacting then more precisely the measurement is something like the interaction feature has some probe of the apparatus so probe is some kind of the apparatus is some very small tips or something which can be considered as a quantum, this quantum system and this probe system interacting quantumly then we can measure this probe by the macro side of the apparatus so this is a very general kind of measurement which is called which is identified mathematically with the notion of an instrument originated by the Davis and Louis and Ozawa one of our collaborators we begin in the cooperation but Ozawa who is also famous for the Ozawa's inequality Ozawa's inequality as a generalization of the Heisenberg universal value version of Heisenberg's inequality but he identified this general notion measurement and instrument called completely positive instrument in mathematics but this CP, completely positive instrument is quite general and generalizing very beautifully the Bayesian influence into the quantum side so I think the using, focusing on this kind of instrument kind of concept is the key to bridge the RTP per side and our side and quantum side or something so it is my speculation but I think it is a very fruitful direction It's awesome, just one comment on that the probe with the instrument and the influence frame I believe as Chris Fields might describe it is an experimenter's choice and so the way that we could talk about policy selection and inference on which experiment to run would be in terms of which sequence of observations is going to minimize expected free energy so the sequence of observations is going to have high pragmatic value and epistemic value that's the expected free energy functional now as researchers we might seek out the most informative experiments and not prefer to see a certain measurement so we can say well all measurements are a priori equally pragmatically valuable to us so now expected free energy is going to be driven by maximization just by pure learning but this recognition that action is woven into the process of reducing uncertainty about other systems across blankets it's absolutely fundamental because if you don't have the constraint that you need to actively select which experiments to run and to probe and to interfere in this quantum way, this way described by quantum mathematics you're kind of in a delusional alternative world where like you get information from photons but you don't have a sensor so it's not an empirically tractable world unless we take on this constraint and then once we do our mathematics are the quantum mathematics and again it's kind of a subtle point this isn't the orco or this isn't necessarily taking a stance one way or the other on what kinds of electronic scale quantum phenomena exist in any system but rather just sees quantum mathematics as fundamental to decision making and uncertainty which is quite a change from how it's been discussed for a hundred years actually as Nao said we're not dependent on the hypothesis like there is a quantum physical process behind the Korea but why we can use the quantum mathematics is very very simple reason because conventional probability theory classical probability theory depend on the probability space classical one is too narrow to capture for example all the effect or something but you can generalize it so kind of mathematics is just generalize the classical mathematics and I think I believe that why quantum mathematics is found for kind of physics is very simple reason because in classical physics some kind of determinism classical determinism is very universal and the probability only comes from some kind of ignorance of us but in quantum physical context there is some kind of very fundamental randomness fundamental accidents or something is found and they are all surprised because if you go into the micro side the determinism should be stronger they thought but actually there needs some kind of fundamental non-committativity in the micro side but without a bore and Heisenberg or some people pointed out that even in classical or even in a macroscopic side but this kind of the importance of non-committativity is very universal not especially in micro side but it is a historical reason that we first find in quantum mathematics in the micro side but there is many many phenomena which cannot be captured by the determinism or the classical probability theory so I think the courier or something is just some kind of example by the way maybe Daniel can you replace this quantum consciousness with the quantum courier the quantum consciousness has been used to imply this quantum brain hypothesis which may be potentially true but so far we don't really see the strong evidence about this particular thing what's funny I'm thinking of the classic qualia question what is it like to be a bat and it's like what if the like is interpreted as a metaphor what if there's a metaphorical mapping to bat which is an observer dependent mapping amongst spaces because we've moved the focus from the objective standalone experience of the bat to considering mappings amongst cognitive systems one critical one being for example whoever is listening to the question and a bat who's the one who actually answers that question and you asked also about the inspiration from the category theory relevant to the issue of the metaphor of communication so now it's a bit far from this quantum courier and jump back to another topic so the metaphor is something that Hayato has been working on so but the other part which is communication and that's something that we also wrote together last year and using this concept called adjunction in the category theory which may be interesting to discuss so in 2016 paper we already mentioned that category theory is very promising for the study of consciousness precisely because it provides different levels of sameness when I heard this from Hayato I was like what do you mean but simultaneously it makes a lot of sense and in fact many controversy or confusion among the field of consciousness but not only for consciousness, other fields like ecology as well classification of the animal species and things like that all depends on this what do you mean by the thing and this is something that also relates to what you were talking about a moment ago which is observer dependence so even at the very bottom of this reality what matters is some notion of this subjective choice of what it means to be the same equivalence or adjunction is a particularly interesting kind of situation where for example integer and real number is definitely different but in terms of the relationship if you start to think about the relationship there is some kind of one-to-one correspondence between them and also Hayato's book also written in Japanese but recently I read it and one of the interesting examples is also multiplication and exponentiation at the level of object or number it doesn't match but there is a very nice relationship between them that is very fundamental and something like this apparently completely different thing but finding really meaningful relationship that is very fundamental in communication we are talking different kind of language with different kind of meaning but because when I say something you reply and there is some kind of systematic difference between what I expect and what is the kind of prediction error I think the prediction error may not need to be minimized but as long as it is consistent and systematic then it actually works as a very nice translational kind of device after living like 20 years more than 20 years of living outside of Japan very strong English accent either in Japanese or living in Australia or in the U.S. I can't minimize this prediction error but because I have this systematic prediction error for you you totally understand me something like this is the interesting thing about that junction this makes me think about nested reference frames in communication and let's just say that the expected free energy or maybe even the integrated information content is such and such a level but it varies within some range it is 100 plus or minus 10 instead of just trying to naively minimize that to zero we can create a nested model such that we expect that to be 100 plus or minus 10 at which point the non-zero value is stabilized or buffered and so then that becomes the kind of stabilizes or wraps the fact that we're not perfectly transmitting information but we can be perfectly aligned that we're not perfectly transmitting information and also our language understanding for fluent speakers we're able to map the phonemes a little bit deeper we're able to read between the lines or listen between the phonemes so that people who say words differently map to the exact same syntax let alone the semantics the sound gets kind of abstracted out but it influences our qualia so something is being passed forward with like tonality and the timing and I wonder about with of course large language models how much context they're able to garner from the string of tokens when for our experience sending and receiving natural language communication it's all about the timing and the tone and the setting and so finding ways that we can describe that type of information the category theory capacity for the different levels of sameness seems like exactly the tool that we need I have to do you want to add anything? yeah I'm thinking about about that you know the multi-level kind of free energy I actually have a question to you actually so I recently discussed with a roboticist in Japan Jun Tani you know his work is using this predictive coding kind of scheme for constructing the self-autonomous robot and he thinks this intermingo of the top down prediction and the bottom up prediction error is the place where you can see a lot of Korea emerges and things like that but one of the most interesting aspect of his model to me is that he has this different time scale for his neural system the most peripheral motor level and sensory level has a very fast time scale so that tries to match with what they want to do towards you know outside and the intermediate level has a slightly longer time scale so there is some kind of inertia and even higher level has a much much longer time scale and so in real probably neural system there is also different level of the time scale and also different levels of the mechanism acting at the same time changing the activity state in spiking and also release of the synapses, synaptic transmitters that could increase or decrease over the time and also even the new neurons emerging and going out so the minimization when you are talking about minimization probably that kind of minimization is actually happening independently and also coherently potentially or contradictory across different levels and also regions and there may be some kind of effort or whatever to also coordinate these kind of contradictory demands of the minimization and what do you think about this issue? Do you still think it's a useful way to think about or use the term minimization in this situation? That's a great question well one thing that stood out to me was you described the slower moving levels as having inertia and from the Bayesian mechanics perspective that's exactly how we would model it they're slower moving, they have more inertia and they move more like classical objects whereas the faster time scales are moving more like lighter particles more like that quantum probe in the limit and then this sort of center or the highest nested model would be like the most cognitively massive potentially the slowest potentially integrating information over the longest time window and certainly this is underlaying by these multiple time scales of mechanism spiking neurons from a few hundred milliseconds time scale, neuromodulatory processes over seconds and minutes synaptic plasticity even neurogenesis and the development of our brain. So what does it really mean that minimization is occurring when we have this heterarchy where we have similar types of free energy minimizing units or active inference units within a level like cortical columns or grid cell units and then also we have these nested models that are optimizing across spatially separated scales. Well one paper that looked at this is our live stream 42 on the slam simultaneous localization and mapping also in robotics and so they had a two nested model with two steps and the lower level was like the tactics and that was where you had the motor and the sensors and then the nested model was slower that was like where do I want to be that was more like the strategy and you can create an expected free energy functional that sums across those two levels. How you tune the parameters is going to matter because how do you wait? Yeah yeah like and one what this comes up with like preference learning like if you have if your preference if you have to left or turn and right turn and your preference vector is like two and one versus two million and one million then that is the ratio of the preference is the same but the amplitude of the preferences is a lot different which is going to swing the model towards valuing pragmatic outcomes whereas if you had 0.02 and 0.01 the ratio of the preference would still be the same but it's such a small the amplitude preference and so similarly with expected free energy minimization you could make it so that the joint minimization was driven by just the top level of the model and then the sort of contradictions or even increases in expected free energy of the lower sensory model would basically always be accepted in the service of the top model or vice versa and it seems like biological systems with their sort of precarious or self organized criticality operations they're poised where you can have like propagations where small changes in the bottom up model can propagate and induce like phase changes of higher levels and vice versa so they're certainly poised parametrically at a special place where the composition of the faster sensory motor models and the slower like cognitive or narrative or metacognitive levels it's all poised in a very integrated way potentially even one that is where the IIT measures tell us something useful but yes the contradictory demands of subunits within a level of collective behavior and across levels as a biologist I think we just point to ecology, evolution and development just distributed systems that don't exist at an adaptive point or implement adaptive dynamics amidst uncertainty they just fail to exist and so the complex biological systems that we observe today do manage that trade off and then in robotics they have this unique challenge of developing systems with one shot or with iterated development that do have some of those properties but for sure there's no reason to think that just by laying out a nested active inference model that you would actually get like a more adaptive decision making agent in fact as you nest models the state space of these models becomes vast and with some back of the envelope calculations you can see like well if I do this level of nesting and have this kind of sensor array we need this absurd amount of data to actually parameterize this model and by the time we collected that many observations maybe the causal structure that we learn to the world 200 hours of video go don't apply yeah I mean that could be quite a huge difference between probably IIT and free energy I think the IIT explicitly states that this time scale that matters but in this case it's just about in Korea is a particular time scale that maximizes integrated information and that corresponds to the time scale of our experience like TANI's model that slowest time scale on the order of 100 seconds or something like that and also you know synaptic plasticity is on the order of hours and most likely spiking neurons seems to be more mattering with respect to our particular experience at each moment right and why it is the case that the provisional answer provided by IIT is that because at that particular time scale integrated information is maximized and across the scale and across the regions but this is a huge conjecture and nobody knows whether this is true or not and when we use the fly model we weren't able to confirm any kind of clear peak in the integrated information using a different time scale so it's still something that I'm very interested in looking but still not clear whether this is the case I don't know if Hayato has some idea or insight into the appropriate time scale for consciousness or something from the physics point of view I have some thoughts but maybe it becomes a long story and also connects to the observa dependence program so I was thinking about how to explain these very important things in one thread and I found that first the very simple fact that there is no there comes no identically same two events every event is different in some respect but to do science or more generally some recognition of law we need some criterion of sameness for example you want to know how to heal the disease you should identify the different persons people as the same kind of samples of course there is no identical person but you have to identify the same in the members of the same group so all the scientific studies depend on some choice of sameness in that sense all the science depends on this kind of choice of sameness as you both pointed out and there is no I think there is no canonical choice of the sameness you can choose every sameness but if you choose some sameness then you can talk about the observables observables mean some kind of quantities you can observe or something maybe it randomly changes the value or no value like a kind of case but by measuring process it becomes half the value or something so there always may be indefinite nature of such kind of quantities but anyway we can imagine that quantities has some kind of structure between the quantities so in mathematical terms it will be some algebra of the observables then how to connect this idea theoretical idea to the experimental thing is the key is expectation value so theory provides the expectation value of something and we can connect the theory and the experiments by comparing the theoretical expectation value and the average of samples so all the quantitative science depends on this structure like algebra of the observables and some mapping from the observables algebra to the sum number system so this is called the state is in mathematics and the algebra and state state is something like the mapping of quantities of random variables of observables into its expectation value so this mapping of expectation is technically called state so if you start from observable algebra and state this is the starting point of non commutative probability theory that is algebra is commutative then the conventional probability theory can reconstruct from this commutative algebra and state but if this is non commutative as now pointed out if the process is essential then this algebra should be non commutative then we have the non commutative probability space which I called intuitively counter mathematics so in this case the observable dependence is described the other kind of observable dependence can be described because you can't have the classical probability space all the observable but if you choose one observable like position or momentum you can choose this then you can construct the classical probability space so the choice of context or the choice of measuring something by choosing something so context you can have the classical kind of probability so this kind of observable dependence is well formulated by using non commutative probability space so I think these multi-layered observable dependence can be described by using non commutative probability and the last thing I would say is the connection of the category theory and non commutative probability very interesting if the category is given like a causal system give us some category like causaliation is arrow and the composition is A is cause or B because C then indirectly of course at least the A cause C or something so it can be considered as a category there are many kinds of categories but if the category is given then the algebra reflecting this category is necessarily non commutative if there is a non-trivial arrow between two objects so actually the category theory and non commutative probability is from the mathematical point of view it's kind of the two aspects of the same thing like the fundamental observable dependence or contextual dependence or something and this contextuality in topos theory actually this choosing concepts and the transformation between the context can be formulated in terms of topos which is a very powerful kind of category sorry for the too long explanation but without that this very fundamental question will not be organized sorry for the long explanation it's awesome I mean empirically if you have a sensor that's getting measurements at a certain time scale then there's a binding of the time scale of the observer and the processing can I ask a question from the chat alright Dave asks at one time professor Tanini was representing the shape of the relations between complexity and consciousness such that two richly connected systems are non-conscious they're overwhelmed by the complexity recently the inverted U that shows this relation seems to have become scarce leading to undue confusion are IIT people abandoned the idea that there is a bounded sweet zone for consciousness can a system be either too simple or too complex to act consciously so is IIT lead us to a kind of monotonic consciousness concept or could there be a U shaped relationship with information integration and qualia I can also see the comment here so in the end the question is can the system be either too simple or too complex to act consciously in the sense from an IIT point of view there are several things that needs to be said but inverted U is still there in a sense and here this person seems to be confusing between the axis of the complexity and the axis of the connectedness in IIT original formulation and it's still the true for the IIT 4.0 which is the latest one when the system is too connected let's say 100 neurons and each of them are connected to everybody and in a uniform way then IIT predicts that it doesn't have much complexity or compositionality or uniqueness to arise from that kind of system and even if you cut the system it doesn't matter much it leads to most likely a very low level of consciousness I think that's what yes I should have typed connectivity yes if that's the case that's fine but higher complexity which is granted by some small world or some sparse connectivity structure higher complexity is monotonically created with more consciousness that's not the case in IIT and that is true from the beginning and the easiest kind of example is to have a two highly complex kind of system that is loosely connected between them it's a kind of a situation where you are interested in the sense of ant colony even though it looks extremely small the number of the neurons they have is more than 1 million in that order yes around a million so IIT would naturally predict that the causal connection or integrated information arising from this 1 million neurons within an ant are going to be really massive it's very difficult to disconnect in any way however interaction between a single ant to the another ant has a much less direct causal relationship therefore the local minima or local maxima of the 5 values is going to be very huge within each ant but when you combine or when you treat all these lots of ants it definitely does bring some kind of intelligent and complex behavior but phenomenality is not there at that level phenomenality, consciousness within this maximum level of integrated information pretty much like what we talked about in the time scale stuff we have a huge complex and very intelligent behavior and artifacts and also short time scale but what matters for phenomenology is this particular several orders of 100 milliseconds that corresponds to the maximum of the integrated information so you still have emergent intelligence at the colony level but that doesn't necessarily mean that you have qualia arising at that level for analogous reasons why our qualia are at the spiking time scale despite that being only one of the nested time scales exactly IIT predicts that intelligence and consciousness qualia is completely dissociable and that leads to the commentary about the large language model probably the large language model is probably structurally built to be very simple almost like probably in the cerebellum in the human brain which accounts roughly like 80% of our neurons in the brain out of 10 to 11 neurons in the brain 80% are in the cerebellum but even if you remove it or lose it or bone without it consciousness is not that different and that's a simple kind of prediction that the cerebellum is doing maybe something similar to the large language model is doing it's very modular, it's parallel it's not integrated, it does a lot of intelligent things but nothing to do with consciousness very interesting well maybe in our closing sections of course anything you want to share or ask but how does active inference benefit and grow from what you're sharing and how do you see things unfolding for people interested in these topics I have to do you want to say something first? I'm thinking how would you know then from my I used to have difficulty thinking about active inference or things like that but by broadening my perspective becoming more generous I started to feel maybe some aspects of thinking and formulation can be relevant at least for the explanation of the biological systems in various domains and if it's also useful to explain intelligence or construct intelligence that would be great and from our work of the quantum quality of hypothesis, if quality is something that is influenced by its measurement then that aspect maybe emphasizing this usually you know quality of consciousness is considered a passive construct at a given moment what I'm experiencing static passive but now I'm more thinking about because of this collaboration with the Hayat and other people that can be the case and it's more to do with the process it may not be that kind of level of active inference that you are thinking not the level of acting on the word but more to do with this state itself influencing itself might be some kind of important insight to understand the quality ok my comment is ok so it is related to the concept of states again because expectation value I pointed out the importance of expectation value but for example the strength of causality or something then the problem of time scale is very important so you can cause grain for example brain activity for one time scale another time scale so I think if IIT is correct there is a good time scale which is of importance of choosing this structure then I think the brain information or something causal structure should be the good category and you can construct non-commitative algebra and state is defined or something so the time scale is quite fundamental and on the other hand the state on the category that is state on the category algebra can be represented by some kind of weight on arrows the positive kind of weight on arrows so the change in the state is the change in the weight of these arrows the positive definite function I think the transition from this arrow is very from at least mathematically very interesting to compare the active inference as a mathematical point of view so the good weight to the better weight or something is very how to say interesting to compare I'm not I don't have much knowledge about active inference but it seems to me it is quite useful to connect this level the state change or something so yeah it is very interesting for me and the other thing is I forgot the third point but yeah and yes as now already okay points out I talked about the measurements, the instruments but the point of this notion is the probe system so measurement always often considered as a very direct kind of interaction like the system and the environment but environment has some kind of probe into the system then the interaction is occurred with this probe and this system I think it is very analogous to the prauria, the case of so of course outer world is relating to the brain inside of course at least a process or the history or development but I think through perception or something the brain has its probe of the world for example how to say, interesting so measurement, the analogy to the measurement theory may be that some physically within the brain but some kind of probe of the outer system and this prauria is interacting so I think it unifies the picture I think maybe what do you think that's actually very much related to what I was actually constructing this morning so I was reading this Ozawa paper that described the instrument and now I'm getting better understanding what it means what he says is that every observable A of a system S is identified with the observable A tensored with the instrument I of a system S plus S prime so S here is the original system and S prime is the one that with any system S prime external to S and what Hayato was meaning is that this S prime is a probe system which is a part of the environment and so here what this quantum in this case quantum kind of system description but it's also true for the quantum cognition and in my case it is also true for the quantum quality hypothesis is that some kind of quality experiencing system is interacting with the world but through sort of membrane or sort of that like a blanket like thing which is already filtering as a sensory input and attention and this part itself is kind of interacting with itself these arrows are the critical part and then if we translate that Ozawa's way to this corresponds to every coil A of a quantum expensive system S which is in the case of IAT it's called complex Ozawa A times instrument of a system S plus S prime and this S prime here I think will include all the neural processes that supports and underlies quantum but it doesn't really directly include the external environment many people start to do that but if you do that then suddenly hallucination dreaming and things like that becomes very difficult to explain including too much I think that corresponds to what you wanted to say exactly well this is really awesome I very much appreciate that we could have this conversation is there anything else you want to add or ask? thank you I also enjoyed and it was also good for me to formulate myself as well yeah I'm very surprised surprised me in the how consistent our thoughts are how to say it was just a surprise for me maybe the conversation often goes how to say diverse but our conversation has a very surprisingly conversion I think it's quite interesting it's I learned much from this conversation thank you for inviting us thank you so much hope to join on your channel and hear updates on your research as things develop so thank you again thank you bye bye