 Hello and welcome. This is active guest stream 64.1. It's November 29, 2023. We're here with Elliott Murphy returning guest. And we'll be talking about this paper rose a neuro computational architecture for syntax. So for introduction and describing how today's stream is going to go, Elliott, thank you for joining to you. All right. Thanks so much, Daniel. Yeah, I work at the UT health in Houston in Texas in an epilepsy monitoring unit where we use intercranial recordings to map language in patients with epilepsy. And but today I want to present something a little bit more theoretical, a little bit more kind of philosophical almost. And this is a new paper that's just come out in the Journal of Neural Linguistics and I thought it'd be good for me. And just to kind of walk through the paper step by step, give some motivation, give some description, give some kind of outline of the general kind of framing for the paper. So the framework for this paper is trying to propose a general model for how to research language using intercranial recordings. It's basically kind of move beyond research that comes from other kinds of extracranial imaging methods. And it's trying to really figure out what is the most appropriate level of description in terms of neural complexity for specific aspects of language. So there's many different parts of language as we know, language has words, it has sounds, you know, it has you can read language, you can you can sign it, you can construct meaning, and you can do all sorts of things. But the real kind of question for me is, at what level of neural complexity, and therefore at what level of appropriate recording resolution, based on what tools you have available, is going to be providing you the most reliable signatures of particular levels of structure. So, in this model, Tim, the Rose model, I invoke representations, operations, structures and encoding, and I'll introduce them step by step. And so for the purposes of this kind of introduction, I'm just going to kind of outline what those general kind of frameworks are. But to begin, the most kind of basic assumption that we have about language, in terms of like the finding some linguistics, so the study of language, like the scientific study of language. One of the really basic introductions that you find in every introductory textbook is that linguists have kind of arrived at the conclusion that the human mind brain applies a set of rules to recursively combine linguistic units into larger objects, which derives an unbounded array of hierarchically structured expressions with humans in faring sentence meaning based on syntactic configuration. Now, that's a lot of jargon, but essentially what that means is, when you hear a linear sequence of words like, you know, we watched a movie with Jim Carrey, you can pass that not simply as like beads on a string. You can pass it in two ways, depending on how you chunk the individual phrases together, and how you establish relations between different elements. So for example, you know, we watched a movie with Jim Carrey can mean that you watched, you know, Eternal Sunshine of the spotless mind or it can mean that you're sat there next to Jim Carrey watching, you know, the new Napoleon film, for example. So there's this kind of ambiguity there, but the ambiguity arises due to structural configurations. So that's kind of how you rearrange the sentences kind of in your head internally. And that is obviously a process that's independent immediately from sensory motor transformations or experience or speech or sound or orthography. So these are very abstract processes and they're clearly mental processes. They're not processes of the visual system. They're not processes of the auditory system. They're clearly a hybrid of mental computations. So what that phrase and unbounded array of hierarchical expression expressions means is it just means that humans have this generative capacity and people like Carl Hristen and Chomsky talk about generativity quite a lot. What it basically means is the ability to just do exactly what I'm doing right now, use a finite set of elements. So lexical items or, you know, memorize words and then recombine them again and again and really novel combinations such that you can say sentences that have never been said before in, you know, the totality of human existence. But which are immediately assigned a meaning. You don't have to think too hard about what meanings are usually unless you're reading, you know, Finnegan's Wake, for example. So typically this is a reflection process. So it's automatic. It's quick. The brain resolves this very well without much effort, without much energetic cost as well. So the question is how does the brain do this? So, you know, an additional point to make here is that this capacity for hierarchical occasion has also been linked to human specific cognitive superiority. So that's a more general background here. So people like Stan DeHaine, Howes and Wattamool, they've long stressed that this facility for constructing hierarchically organized linguistic structures is really potentially a human specific function. And that's a very long and, you know, it's a long and difficult topic to get into. So I'll put it to one side, but it's an interesting topic to explore the possibility that this, whatever this process is, it might in fact be unique to humans. So in the theoretical domain and turning to neurobiology and a major consensus in neuroscience is that complex behavior and cognition rely on coordinated interactions between brain regions with face synchronization being a major candidate for a mechanism implementing this coordination. By gating information transmission in the brain. And but unlike for models of attention and working memory, there's currently an absence of oscillatory phase coding in models of natural. So what does that mean? So in the neurobiology of attention and working memory, there's been a lot of really exciting accounts by people like L Miller, early ensign, and a lot of other people trying to explore the relationship between neural synchronization metrics and some kind of isomorphic kind of relationship to some properties. So the property of an induction is computation, like a working memory process or some kind of property of the environment that you're kind of, you know, passing in real time. And but in language, the best we have just to boil it down for a very, very generally, the best we have at the moment in language is basically cartoon called cartoon model models. So that's kind of localization as models in the brain. So we have a decent idea, thanks to the really brilliant work by people like Greg Hickok and Willie Machin and Anglia Friderici and I think and David Popol and really, you know, a really incredible roster of people from the last few decades, who very carefully mapped out which specific brain regions seem to be sensitive to different types of linguistic structure. So lexicality, meaning, you know, the meaning of a word and syntax, the generation of a sentence structure or phrase structure. Semantics, the meaning of words, phonology and so on. But what we really have right now is just a kind of map. So we have a map of the brain that you can pull up on Google and you can Google, you know, language syntax brain and you'll get a decent, you know, rough architecture of which parts of the brain seem to be sensitive to different parts of language. But the main reason why I wrote this paper is because that's of essential, you know, mapping out the terrain is obviously essential. But the next step, and people from working memory, vision, attention, they did this, you know, a while ago. The next step is therefore to transition to a kind of how question, which is which neural mechanisms are involved in language. So rather than simply saying, these are the brain regions involved, we have to then say, OK, these are the brain regions. But what is the brain doing? Right. What operations are it is it's performing at what level of neuro complexity? And is it sensitive? So just because this part of the brain is sensitive, we don't really know in what way it's sensitive. Like, how is it what type of neural processes is actually recruiting to yield that sensitivity? And what does that sensitivity really look like outside of both fluctuations or, you know, a high gamma amplitude increases and other kind of difficult measurements. So that's in the theoretical domain. In the empirical domain, invasive intercranial recordings have recently mapped out some of the feature space associated with how much phonological information can be transmitted by different neural mechanisms and some citations there for you to look at. But within the domains of syntax and semantic, because there's been less clear progress. And the reason for that is pretty obvious. Phonology, sound, auditory processing, you know, the kind of a finite feature space involved in auditory processes. That's a very non trivial, you know, field to explore. But it has been mapped out very clearly because the, you know, processes are kind of less abstract, more well agreed upon, more transparent. They involve more clear sensory motor transformations and things that you can detect very clearly with very nice clear signatures in different types of neural states, state space, state space, or in fact, different reasons of the brain. And these are also things that you can disrupt very easily. So it's very easy to disrupt phonological processing. We know which parts of the brains, which part of cortex is disrupted. But it's less clear how you specifically disrupt things like syntax and semantics. You can knock out, you know, language comprehension in general, just all language comprehension. You can knock out and disrupt certain aspects of working memory performance. But when it comes to knocking out, you know, disrupting the brain's ability to specifically process syntactical semantic information, that's been very difficult. And there's only been a few papers on that, which I cite in this paper over the last couple of years, but it's very difficult to really try and, you know, provide causal evidence to try and really stimulate the brain in the OR or to find, you know, lesion patients who've had a stroke and who exhibit extremely specific niche, you know, deficits around this aspect of syntax or semantics. So that's kind of the obstacle that we have, right? The obstacle we have is that we have this very strange, potentially human-specific, seemingly very simple, you know, process of just building hierarchical structures, but it's very difficult to isolate from all these other things going on. Because as soon as you build a sentence structure, as soon as you build a phrase or a sentence, you are also doing loads of other things. You are also using your attention. You're also using working memory. You're also saying things or reading things or engaging your eyes or your ears. So that, you know, it really is very difficult to isolate it. So anyway, that's the problem that we have. So I'll just skip this part to the next section here. So I'm just going to read through some of this text and maybe try and unpack it a little bit more. But the past decade has seen the emergence of low-frequency phase coherence as a feasible index of hierarchical syntax structure. So what that means is that if you read these papers, there's low-frequency oscillations in the brain that can be detected using MEG or SCALPEG. And they basically show little fluctuations, little peaks aligned with certain moments and periods of or windows of structure building. I'm just thinking of more metaphor, little islands of structure that come along when you pass a sentence in real time. Whenever that moment of structure building occurs, you seem to get this low-frequency peak in activity in the brain. It tends to be around, you know, classical language cortex, like left inferior frontal cortex or temporal cortex. But the actual spatial temporal isolation is still a little bit iffy. We still don't really know, by frankly. So that's the kind of background for that aspect of language. At the same time, though, recent workers also examined local cortical processing using intercranial recordings. So these are recordings inside the brain. And what they do is they often focus instead on high-frequency power. They can detect low-frequency power, absolutely. But most of the analyses and novel findings within the intercranial space have prioritized looking into high-frequency research. For the simple reason that's the only real way that you can get high-frequency recordings. So this works exposed signatures of syntax in other kind of areas outside of low-frequency dynamics. So there's a big question here. How these two distinct recording scales, low-frequency dynamics and high-frequency dynamics, could be combined in a coherent model of natural language syntax and cortical computation? That has been addressed yet. So that's kind of one of the arguments that I try and build here. So yeah, neural oscillations are effectively, like I mentioned, they reflect synchronized fluctuations in neural excitability and are typically grouped by frequency, mainly for convenience, but not always. There seems to be something else going on. Common rhythms are often termed delta, theta, alpha, beta, and then gamma. And broadly speaking, high-frequency activity reflects local neural processing, whereas low-frequency activity reflects regional synchronization. So high-frequency activity, you get that local cortical computation in a specific brain region. This brain region is doing something as opposed to low-frequency activity, which is often more widespread. And it's more to do with the coordination of regional synchronization. So the low-frequency say, okay, it's time for you guys to all come online at this particular time to, you know, engage your spike activity in this particular time period, but over a larger portion of tissue typically. It's over much larger areas of the brain. And then there's a bunch of other kind of computational differences seemingly as well between these lower bands. So for example, alpha is very often implicated in cortical disinhibition. It kind of helps to shield and protect ongoing representations from decay and transformation. And then theta oscillations often implicated in hippocampus. I've been involved in learning in memory. There's a big literature on that. Don't need to go into the details, but essentially there's a whole panoply of low-frequency rhythms, and they seem to be sensitive to different cognitive processes, or at least recruited, or they seem to index a bunch of lower-order processes potentially. Okay. So one way to kind of introduce this is through a model that I proposed in my book in 2020. And in this model, again, using invoking low-frequency power, the combinatorial power of language is indexed via various oscillatory interactions, like forms of cross-frequency coupling, which is basically when the phase of a lower-frequency rhythm kind of coordinates and dictates the firing of a high-frequency rhythm. So you kind of get a potential facilitation of what's called multiplexing, or kind of packaging representations in one order, and then sending them downstream and kind of re-accessing them in a different part of the brain. So in this book of mine, there's a bunch of kind of literature that I've reviewed to motivate the idea that delta-theta phase amplitude coupling constructs multiple sets of syntactic and semantic features. So you get this initial low-frequency phase interaction between delta and theta. And this occurs when the phase of delta is synchronized with the amplitude of theta, and delta represents superordinate syntactic categories, whereas theta represents feature bundles generated via lexical access. What does that mean? It means that these theta oscillations, there's something called theta-gamma coupling, famous in memory research and navigation. Theta-gamma coupling, and in fact there's some really nice work published yesterday, I think, by O. A. Ensign, looking at how attention might be also implemented through alpha-gamma coupling. But the basic idea is that you have individual sets of features that are basic mental representations, like things responsible for animacy, plurality, basic conceptual categories. They seem to be triggered by high-frequency activity, and then they're coordinated and packaged into a single chunk by theta. So you have a single low-frequency theta wave, and within that single wave you get a bunch of individual 5, 6, 7, 8, however many you can squeeze and fit in, and there's a trade-off between fidelity and the number of gamma oscillations you can package inside a theta cycle. But that packaging of theta-gamma helps you to coordinate and unify an individual representation in the attention and visual spaces that's been referred to as coordinated visual attention when your visual system is attuning to particular properties in the environment. In the language space I'm invoking it to unify different individual features that compose into a lexical item. So a lexical item is just a fancy word for a word. So something like Wachtop, bed, I'm just thinking of random words. All of these random words are obviously composed of individual features, conceptual features, right? So a bed is inanimate, it has a particular geometry, it has a particular function, a purpose, a design and so on. And all these different conceptual features compose into what we mean by a bed. Whenever you think of a bed or a chair or a mirror or whole foods, it could be anything, anything from simple to complex, you have a constellation of conceptual features that are being composed somehow. And the idea here is that these low frequency theta oscillations are there to coordinate the firings of those individual features, which is why you get a lot of lexicality effects being found in, for example, the theta range. And then what you do is, in this model, you get those theta bundles of flexible features and you embed them further. So you nest them again. And what you do when you nest them again within a delta rhythm is that the delta rhythm then takes over from the individual lexical items and it composes a syntactic identity, so a structure, a structural identity. So for example, it's been, you know, well studied in linguistics that when you put two items together, two words together, you get a kind of symmetry breaking operation where one of the items kind of determines the category of the phrase. So for example, if you say red boat, a red boat is a boat that is red. A red boat is not a red-like quality that has, you know, boat-like features, for example. So there's a kind of immediate asymmetry. And you get that all over the place. If you say John Rand, John Rand does not mean that there's a special kind of John who is exhibiting a running feature. It means that there's an event of running and John was its agent. So there's always an immediate kind of asymmetry in language. And that's, you know, people like Stan DeHing and others have kind of speculated that one of the interesting things about language is that unlike music and mathematics that kind of allow symmetry in their structures, language doesn't really like symmetry. As soon as you get symmetry in language, the language system tries to break it somehow because there has to be some kind of categoriser for you to identify what's called the label or the category of the phrase for you to get a single unified syntactic representation. So in other words, whenever you get two words being combined together, one of them always wins and one of them is the loser. And that's how you build syntactic structures. You get a kind of categorisation process. So just to kind of recap, you have delta, theta and gamma. The delta assigns or it somehow derives or facilitates the syntactic inference of what the category is. And the reason who knows, right? I'll try and explain some potential reasons here. Ultimately, it's unknown. But we do know that in the empirical domain, we get a lot of these syntactic signatures being found in the delta range. So delta seems to care about chunking on some level, maybe not syntactic specific chunking, but I think it's possible for reasons cited in the paper. So delta and then packages within one of its waves, multiple theta rhythms. And the reason why is because the same mechanism that you invoke when you get theta gamma coupling. So it's a series of nesting and nesting and nesting. And the whole Rose architecture in this paper is basically trying to construct what some people might think of as rather crude, but I think rather empirically motivated in fact. Isomorphic relationship between structure and neural kind of signal complexity. It's common for people to think, well, the brain is very complicated, language is very complicated. It surely can't be the case that a particular level of linguistic structure maps directly onto a particular resolution of, for example, a solitary phase coding. I think there's good empirical evidence to actually assume that. And not everything takes place at the lower coarse grain level at the spike level, for example, or even intracitally. I think it's unlikely that all of these complex processes from making inferences about whether or not read or about wins becomes the categorizer. We have to go adjunct the phrase is all taking place at the intra or intercellular level at the kind of global, you know, the kind of stuff that you can detect using single unit recordings, for example. And I think something more global and interactive is happening. And that's why you kind of need to invoke some kind of phase code because the phase code, you only get that once you transition to more global synchronization metrics of the kind found with low frequency dynamics. That's the whole point of low frequency synchronization. So I think that's kind of the kind of basic natural progression that you find here, at least in the way that I've motivated it in using the literature that I review in the book and kind of is recapitulated here to a certain extent. OK. So then we turn to something called phase resetting. So phase resetting alongside concurrent encoding and storage of the products of these generated objects permits a facility for recursive self-call. So that's kind of where you get the recursive mechanism of language. You're consistently renesting objects into this more complex delta rhythm, chunking out more and more and more low frequency complexes, essentially. And just on the side, we also know that from empirical work by people in my lab and in the Netherlands as well, that you get a bunch of different dynamics alongside this. So for example, we know that beta rhythms and other types of gamma rhythms are involved in all sorts of different types of processes, like conceptual storage and maintenance of a particular structure and memory. So there's a very kind of complex interplay here between different rhythms. And it's not intuitive. It's not easy to grasp on first. And I realize I'm potentially not doing the best job of explaining it because there's a lot of things that unpack. As soon as you move to another kind of concept, you have to unpack it to a certain extent. But the basic idea is that there's a lot of interacting low frequency rhythms that then dictate the firing rate and coordination of these faster rhythms. And the faster rhythms are responsible for the smaller, more atomic features, like individual lexical items. And the slower rhythms are responsible for the more complex, larger structures, like syntactic identities and syntactic workspaces and things that you construct. So the output of the syntactic generative engine is found at the lower frequency end, whereas the input, the kind of stuff that you need to provide to the system to get any syntactic structure building done in the first place, like, for example, lexical features, you need at least to access some features before you can combine them in order to combine something. You need to have something to combine. Those individual properties are hosted by the high-frequency oscillations, essentially. OK. And then finally, traveling oscillations are also going to be relevant here. So these are, essentially, these slow frequencies that I mentioned, except they exhibit a particular directionality. They migrate in phase coherence across the brain. And what that means is they coordinate the spiking of neural clusters across the brain, across fixed points. So they kind of move from one part of the brain into another part. And as they do so, as the phase of the low frequency is shifted across a particular path, they then trigger particular spiking profiles of these neurons that they travel across. So it's a coordination process. And it's also, of course, given the nature of a migration path that naturally leads to the conclusion that, as is often commonly known, different parts of the brain are innately sensitive to particular representational features of the environment, particularly conceptual features, things like faces, places, orthographic information, emotions, and things like that. There's a very big difference, for example, between inferior parietal cortex and anterior temporal cortex in terms of which conceptual features they seem to be sensitive to. So if you get that traveling profile, you've got that low frequency phase code moving across a particular extended path, then that immediately gets you, essentially, for free. It gives you for free the mechanism of activating distinct conceptual representations during that process. So I think there's a lot of other things to say here, but traveling waves have been implicated in a number of kind of behaviorally relevant processes in humans and also non-humans. And they seem to support brain connectivity and function more generally. So yeah, that's kind of the background for the traveling wave mechanism. And then finally, one final thing to say here. I mentioned the delta, theta, and gamma complex. There's also been some interesting work outside of one, which looking at delta gamma feature combinatorics. So that's kind of skipping out the middle man and simply involving this low frequency delta complex being face coupled with the amplitude of high frequency gamma oscillations. And what that seems to be involved in is feature combinatorics in general. So for example, it's been involved in fluid intelligence. So when you have to just perform a particular task or solve a problem immediately and call upon various cognitive representations from different core knowledge systems in your mind, lots of different, you know, aspects of intelligence, you have to kind of recruit here and there. You don't necessarily have to, in fact, you don't at all, construct a hierarchical structure. You don't construct a hierarchical recursive representation. You just combine things together into a set, into a conglomerate to help you solve a particular puzzle. So that's what fluid intelligence is all about. You also get that with, say, in early stages in phrase composition in scalp EEG. So people like Andre Martin and Jonathan Brennan have done some nice work showing this. So what that tells you is that there's, you know, in language and also non-language, you can combine features for all sorts of reasons. When you combine, if you need to do that, delta gamma coupling seems to be sufficient, and it seems to be a very efficient way of doing that. Now, if you want to construct an unbounded array of hierarchical structure expressions, as I mentioned earlier, that might not be sufficient. And so the empirical argument I'm making here is that delta gamma coupling will not be sufficient for the kind of rich syntactic phenomena that are often documented in standard textbooks, for example. So anything beyond a basic phrase unit, I suppose, anything with long-distance relations or dependencies and things like that, or in fact anything beyond a single phrasal structure will probably need to be called upon all these other mechanisms that I mentioned earlier, the more kind of richly layered levels of low-frequency interactions. So delta gamma is fine for feature combinatorics, but it doesn't seem to be sufficient. Again, that's an empirical claim, could be wrong about that, but it doesn't seem to be sufficient based on the literature for the more kind of complex structures of the kind that we all know in the linguistics world. Okay, and then we can also make some interesting claims here that I've made with my colleague Antonio Benitez Braco in Seville in Spain. All these aspects of oscillatory dynamics obviously result from somewhere. They don't come from nowhere. They result from genetic guidance, and some recent workers provided a list of candidate genes for this particular guidance. So we have a particular phase code, a neural code for syntax, but it's got to come from somewhere, unless you're, you know, well, I'm not going to name any names, but unless you're the kind of person who's so in love with deep learning and large language models that you think that it's sort of just some kind of a phenomenal domain general learning process. But if you don't believe that, then there's got to be some kind of genetic structure for language. And so we've mapped a bunch of these potential genes using a bunch of different techniques to particular functions. But that's a very much an ongoing kind of far off project. It's just the very beginnings of trying to map out the kind of genetics of the neural code for language. But at least it's a coherent project, right? It's a legitimate coherent project, although rather nascent too, to be fair. Okay. So just to background this discussion a little bit more, I realized we're already quite far in, so I'll just kind of skip ahead a little bit. One of the most important questions in the neurobiological syntax is essentially, you know, what type of neural activity, low-frequency activity, high-frequency activity, spike rate, can track very long or very brief phrases. So my basic answer, which I've already essentially given to you, is associating roughly the respective scales of recording and neural organization with particular levels of linguistic complexity. So low-frequency gets you structures, high-frequency gets you operations, and spike rate individual spike trains give you basic representations. Okay. So this section here is slightly less important for our purposes, so I'll just skip over it. But essentially my claim in this section of the paper, is just to kind of review existing models of language in the brain, neurobiology of syntax in particular, and just to show that they essentially tick many of these boxes, but not necessarily all of them. So certain models emphasize the R or the O, but they don't talk about the S or the E. Or in fact, in my own book, I talk about the O and the S, but I don't really talk about R or E at all, in fact. Other work talk about potentially a combination of all these features but don't provide a particular mechanistic relationship between how these features relate. So for example, you can talk about the neurobiology of R and O and S and E, but even if you have that, you still need to have some way of causally relating them, which is the kind of final section of this paper, which is how you get... So for example, I mentioned that the representations of the R unit is hosted at the spike level, and the E unit encoding is represented in low-frequency traveling waves. Well, how the hell do you connect those two things, right? How do you directly... What mechanism do you coordinate them? So we know there's some recent work showing that, in fact, traveling waves can, in fact, directly coordinate spike LFP coupling. That's some very promising work there to kind of unify all these features together. But you've got to have a mechanism to do that. So you've got to have a separate mechanism in between R and O, O and S and S and E. It doesn't just come for free. Okay. The previous models are very promising from a chaotic graphic perspective. They really do help us understand which parts of the brain are involved in different aspects of syntax or semantic. But in terms of providing the how question, right, like which neural mechanisms exactly are involved, that's typically lacking. The best kind of that you have is definitely the work of Stan DeHane. He's done some really nice work showing that signature syntax might be in places that most people have not really thought about, especially at the lower level in individual spiking activity or cellular bar codes and so on. That's very promising and it's something that I invoke in this paper in fact as well. Okay. So just to introduce this a little bit further, these four components of rows, R, S, E enter into every syntactic phenomenon, everything. As soon as you get a syntactic structure, you need to have at least all of these things. Okay. So in terms of representations, I define them just like any old mental representation essentially. They are an instruction to provide a particular conceptual feature. They are composed of features determining constraints on operations as well and that's a really important point by the way. I do want to stress that it's well known in linguistics that words have things like selectional restrictions. So a verb will often co-occur or will need to occur with a certain other word. You can't just put words in any order that you like. So words are kind of like magnets they're kind of kind of attached to certain other words and often that's independent of statistics by the way. There's some good work by Andrea Martin and Sophie Slatz on the relationship between statistics of language and structure in language and they're not the same thing at all. They often correlate and they often aid each other in various ways in very intricate ways but they're not the same thing at all. You can't simply derive all possible structures in language just from the frequency properties of words in the biogram and trigram frequency rate. You need something else, you need kind of structural information. So the fact this is kind of a maybe a subtle point but the fact that individual representations do in fact have selectional and agreement and movement constraints already tells you that there's got to be some kind of relationship between or dependency between the nature of how we neuroly represent these representations and how they then transition into the operation stage, the O stage and there's got to be an initial dependency. So some of these features can be maybe a little bit baroque but linguists often talk about things like demonstrative, noun, plural and so on and these are of course human defined the way that our conceptual systems work we've tried to define these things using these terms are these cognitively plausible or neurobiologically plausible? Maybe not but there's at least some kind of atomic unit that's being involved here a unit just called X, right? So this feature X and Y they are being composed into particular configurations. So maybe these terms, these English language terms are just convenience to kind of place hold it for what these conceptual features really are so to speak but I think that language is slightly misleading for the reason people like Chomsky and David Popel have talked about a lot of these things really just come down to the kind of classic Marian levels, David Mars three levels of computation, algorithm and implementation. All of these things are totally kosher and legitimate at the computational level and so therefore it doesn't really we don't really have to abandon them necessarily just because we can't find what looks like a noun feature in the brain a lot of this involves category errors I think so which is why I try and break it down into more atomic stages so syntax builds structure through recursive applications of merge, like I said you have this basic merge operation which I haven't introduced yet I realized but it's basically just a binary set formation with some kind of categorization so you form a set you just put two things together and then you categorize it, you give a category, a noun phrase or a phrase or something like that and there's much more to be said there but that's essentially the basic property of language just building sets, set formation okay so this is the outline of the model at the R level, as I mentioned you get single unit encoding of conceptual features and what are called formal syntactic features and these are basically features that I won't bother going into but there are certain features like Q and Phi and other things that are other features that are interpretable or interpretable, things that are features that only enter into syntactic relationships and don't have any other role they don't have any other role in providing instructions to phonology or anything else but so these features involve a what's called a cellular bar code for distinct features that compose into syntactic objects coherently bound by these high gamma activity at O so the O stage is the later stage and it also involves vector codes this is invoking a lot of work from John Hopkins University and people like Stan DeHaine again the R level also involves vector codes for ensembles, neural ensembles cellular ensembles, hosting features common to objects represented at O and which are ultimately coordinated by the S level, the low frequency level and at the O level high gamma activity, so this high gamma activity is like I said it's a neural activity that is around 60 to 150, 200 hertz amplitude in the brain these, this high gamma sensory motor transformations so things like high gamma activity provides access to sensory motor transformations and they transform these sensory motor representations like you know sight and sound and orthography and so on, things that you read, things that you are exposed to they transform them into lexical objects and if you're interested in the particular regional development there's a list there and then they are there for accessible as I mentioned earlier, they're accessible to delta or theta phase locking so I mentioned that the theta gamma coupling where the phase of theta dictates the amplitude and the firing rate of the high frequency oscillations so you only get that once you have high gamma activity because that's the sufficiently correct resolution, the optimal resolution it seems, which the brain likes to coordinate long distance relationships and you know, transfer information from one part of the brain to the other, essentially so this level can implement the semantic composition of language specific concepts that coordinate the firing of our units, so this high gamma level has also been implicated in basic semantic composition so in a paper of mine I've looked at the composition of basic adjective known structures intercranial recordings and high gamma activity seems to be sensitive to this basic aspect of semantic composition not necessarily syntactic composition but certainly semantic composition, it could also be involved in syntactic composition but certainly semantic composition syntactic composition again would necessarily involve the S level your S for structure okay and then of course the high gamma activity itself also activates these ensembles that I mentioned of the R level that are composed of distinct units hosting this barcode or vector code for the finite list of units that compose into the feature bundles that you get whenever you access an individual word, every given word is literally just a linear list of these features that you kind of trigger whenever you access it and again you know all of this stuff is only talking about the syntactic semantic side, the side of phonology and sound and gesture and orthography and all the rest of it, that's a separate issue that's a totally separate issue I'm only talking about interpretation comprehension, you know planning thinking, that kind of thing I'm just talking about internalization here okay turning to the S level as I mentioned this is a low frequency neural program for generating structural inferences over the O level so delta theta phase ampere coupling as I mentioned gives you categorical inferences that allow you to modulate the representations of these feature bundles so the delta can coordinate the theta gamma complex and it has some say in the sequencing and the triggering basically the triggering of these features and how they are accessed basically the readout and then finally the encoding phase this is what I mentioned earlier with the traveling waves local and global workspaces for bottom up lexical memory and top down hierarchical memory traveling waves implement delta theta coupling for hierarchical memory whereas the theta gamma coupling gives you a kind of bottom up lexical memory it's kind of linearized memory just for lexical features whereas the hierarchical memory the memory about hierarchical relationships in a structure abstract relations in a tree structure independent of any linear order or independent of any century motor transformations triggered by gamma there's a separability between the structural inferences made and the lower level century motor operations that you get and are accessed by gamma and then as I mentioned earlier as well alpha power can be involved in the shielding of this process and increase in alpha power towards the end of sentences is often indicative of like some kind of accrual of semantic attention or semantic working memory or something like that so I think that's potentially related to this issue too and then finally beta power also codes for syntactic predictions there's been some mentioning work looking at how beta power seems to be sensitive to successful or unsuccessful but basically triggered or not triggered syntactic predictions or at least anticipation of building a syntactic phrase or you've been exposed to certain words and now you're about to generate a structural inference or you're predicting a specific syntactic item beta power seems to be involved in the indexing of that sensitivity but notice something interesting here right that's beta power this whole notion of prediction and anticipation that is a separate process from the core structure building inferential process involved in that's been exhibited by Delta and Peter and has already been documented in Delta and Peter so again this is a kind of neurobiological support for the separation of predictive algorithms predictive processing and more old fashioned traditional structural inferences that are made by the brain so just to go back to that a little bit more it would be a problem for me it would be a problem for a lot of people if all of these processes anticipation prediction coordination and attention and also syntactic structural inferences were all indexed and all coordinated in this Delta and or Peter rhythm that would be a problem because how the hell do you separate it's very difficult to think about that if that were to be true then we would probably have to abandon the notion that we can construct a coherent a solitary phase code a language it might be impossible you just have to give up on it and look elsewhere but the fact that we can in fact find the separability I think speaks to this potential which is why I wrote this paper because I think there's some good potential here okay and then this figure down here if I can maybe just zoom in a little bit potentially this just depicts exactly what I mentioned in a variable form in visual form so the representation level again these individual features here correspond to individual cells sensitive maybe not selective but sensitive to each of these features the question of selectivity versus sensitivity that's purely empirical I enjoy you know philosophy and theorizing as much as anybody else but these questions are at this stage purely empirical it's possible that there are certain cells that are selective or sensitive to all these features but at least that's the framework there's a particular cell infrastructure that facilitates these representational accessing essentially and when it comes to operation again I need to be careful with the term operation because I maybe this is the one that I haven't explained well enough so I should maybe spend a minute just explaining this there's the brain you know carries out phonological operations it carries out semantic operations it carries out syntactic operations it carries out all sorts of operations in the context of rows I'm focusing on how it carries out the combination the initial set formation process involved in merge so the early stage of merge essentially the O versus S kind of captures the distinction between merge and labeling essentially the distinction between set formation just building put in two units together and then transferring it into a category so I guess what I'm trying to say is the categorization process will not be found at the O level at the high gamma level you might see some category sensitivity in high gamma you might see different gamma profiles for you know verb phrases versus noun phrases and in fact we've already found that people like Andrea Mara has published papers on this so we know that there's a syntactic or semantic sensitivity at the O level again the difference between verb phrase and noun phrase and syntactic it's also semantic right that's the problem big issue here of trying to separate syntax from semantics is that whenever you modulate syntax you also modulate semantics and whenever you modulate lexco semantic content you also therefore modify everything else syntax and semantics so that's an issue and there's you know we're trying to well we are in fact you know running some designs at the moment to try and experimentally tease those apart but for this purposes for the current purposes the term operation is referred to unifying different lexcal and semantic features together essentially into a coherent unit into a set and once you've got that set represented in gamma then you turn to the structural level and the structural level is more higher order, more complex, more portable, more interactive and therefore more low frequency right that's the kind of goal that's the kind of framework here and then finally the encoding level I mentioned very briefly maybe a little bit too briefly the two types of memories that you get in language there's been some really good work by people like David Adger Queen Mary and a bunch of other people who've kind of explored the different types of syntactic workspaces or linguistic workspaces like when you comprehend a sentence you don't just have a single where you might in fact have a single workspace it's possible but it's also possible that you read it out into different separable workspaces that are responsible or sensitive to different types of memory representations so it's often been implied by people like William Machen I think very legitimately but in free frontal cortex is really involved in syntactic working memory specifically we don't know exactly what features of syntax but certain types of syntactic information are held there and maintained there in short-term memory but so for lexical memory I'm invoking theta-gamma coupling and then as I mentioned briefly for the hierarchical memory I'm invoking delta-theta coupling and then unifying the both of them or connecting the two was this other form of processing that I mentioned earlier the delta-gamma coupling so the delta-gamma coupling as I mentioned is just involved in basic domain general very domain general combinatorics just combining two different representations from different parts of the mind but nevertheless you'll see the relationship between delta and gamma between both of them now when it comes to the spatial temporal denounce of this process the actual kind of how this is implemented in the brain in terms of which networks, which white matter pathways and so on that's a purely empirical question but we do know enough from the MEG from the scalp literature, from the intracranial literature that these codes are legitimate codes to be implemented across some portions of cortex I have a bunch of speculations empirically supported also in terms of which parts of the brain these particular encoding workspaces are going to be found and going to be documented but that's at least this is the general abstract schema if you like this is the kind of very general schema and the reason why I've emphasised that is because as I mentioned earlier every other paper in neuro-linguistics focusing on syntax is almost entirely cartographic it's almost entirely localisationist in terms of let's figure out if it's, you know, is it IFG is it PT, you know, is it ventrotemporal they're the kind of questions that are framed by most of these models in the literature and though essential that's not my concern here, right? my concern here is yes localisation but let's try and focus on which particular abstract neural code is going to be generalisable enough in different parts of the brain and also by the way, small point here also generalisable across cases of plasticity you know, I'm involved in a neurosatury case today we know that the brain is highly plastic when you resect certain portions of tissue recruited or not in language you get fairly rapid organisation over a period of weeks and months sometimes years but that tells you immediately and in fact there's also really fascinating work in congenitally blind people and or deaf blind I think too showing that portions of their yeah, in the congenitally blind there's portions of occipital cortex that come online and are sensitive to syntax so because you know these people are not using their occipital cortex for vision, they don't need it they end up recruiting it for other purposes too sometimes linguistic purposes so that's why I think focus on a more abstract code is more important maybe not more important maybe that's unfair but I think it's potentially just as important if not more important than trying to figure out the kind of cartographic landscape for how language is implemented also by the way, another small point which is maybe not so small these days this is also very important if we care about things like brain computer interface devices so, you know, neuro technologies so there's a bunch of what's called BCI brain computer interface devices that you implant in patients' brains to help them with either their epilepsy or to help them with speech processing you get quite a few patients who lose their speech and there's some other efforts recently to try and help people with, you know, sorts of language deficits by using neural implants there's a you know, a factory not too far away from me north in Austin and there's places owned by Elon Musk as you know, who owns Neuralink these guys are trying to do something similar but, you know, that's a speech when it comes to designing a brain computer interface for things like syntax or semantics I think we're going to need to pay attention to these aspects, these questions to do with the neural code rather than just the region because if you're going to have somebody who has a syntactic deficit or a semantic deficit and you're going to try and help them get their language faculty back it's no good just knowing which part of the brain is going to be recruited for their language function with a BCI device you need to coordinate particular stimulation profiles and particular ways to coordinate and trigger and suppress brain activity and so in order to do that you've got to have the right parameter space and to figure out the parameter space you therefore need to have a question of neural code involved almost definitely a phase code some kind of high frequency or low frequency phase code so all of these questions that I'm asking right now don't just have kind of theoretical, you know, basic science academic implications obviously that's my interest, that's my main concern here but they will also definitely have implications for patient treatment in the BCI space moving forward okay right then, let's move on let's skip past some of this I've it's getting on to almost an hour now so I'll skip through the next few sections are just outlining representation, operation, structure and encoding so I'll just pick out some of the most important moments from here but again this section number 5 just redefines again maybe exhaustively what I mean by representation some example features and in fact what these gamma oscillations and high frequency oscillations might be indexing more generally in terms of their cognitive scope and their computational kind of facility this is just a little brief definition of the kind of scales that are relevant to the different levels so we have single unit activity, multi-unit activity the envelope of spike and the LFP, these are different aspects of remote organization and recording resolution that will be relevant to the different components of rows and that kind of just outlines the scope of inquiry here this was something that I put into certify to make it clear that I'm talking about representation in this sense and not that sense basic units again I'm just talking about what these basic units are and then I'm turning here to concerns about how to distinguish it from things like operations okay and then oh yeah so this is an important paragraph I think so this recent work by Nelson FL discovered preference activity for content words over function words in the anterior temporal lobe unit recordings again I should stress sorry so that already begins to tell you that at the single unit level at the silo level different parts of the brain I mentioned ATL already in this case ATL does seem to be sensitive to lexical type and again that could be due to other reasons in this case they looked at the independent of word length and way position the big confound here is word frequency but you know there's some of the kind of indication here that potentially something to do with lexicality is driving this difference this sensitivity okay and then more generally what's called what term concept cells that preferentially activate for stimuli that occupy a certain position in some category axis seem to support a declarative seem to support the government information what does that mean it just means that there's a really some really nice work by Bowertow showing that there's a kind of feature space in primate and also human temporal cortex that is sensitive to a kind of axis of animacy so whether or not an object is animate or inanimate like a chicken versus a laptop there's some things that are kind of in between and vague and then whether or not a surface is spiky or stubby so kind of you know like this or spiky so kind of a kiki boobah kind of situation for linguists so that's kind of a nice conceptual geometry that you have going on and there's different cells concept cells that preferentially activate the stimuli that occupy a space within that axis so that tells you very nicely that at the unit level we have a potential neural code for this very you know generic but also all encompassing object space this is Doris Sauer's lab the baracle paper so you know the kind of promise the promise that this gives us for linguistic features I think is quite interesting because linguists have spent a long time you know reminding us that the feature space in linguistic semantics can also occupy something like animate inanimate and then some other abstract concrete kind of feature spaces too so I think it's very reasonable to expect and to predict at the unit level a similar kind of you know conceptual geometry in place in a different part of the brain potentially posterior temporal cortex potentially inferior frontal cortex now who knows we don't know until we explore but I think that's that kind of you know reinforces the the definition of the of the R level but I'm defining here okay right then okay so this next section here mapping from auto this gets at the very tricky issue that I mentioned at the beginning at the beginning of the paper I lampoon and criticize all these other people for not doing this so hopefully I do it you know a decent job otherwise I will embarrass myself a little bit but the kind of motivation here is that we don't just need a definition of what R and O looks like in the brain we need a way to relate R to O we need a way to feed the information at the R level the unit level all the way up to the O level and so I mentioned just a quick a quick reminder R is a single unit you know spike-train activity and the O level is high gamma activity right so they kind of slightly more easy to detect using intercranial measures for example okay so how do we relate these these ideas the O to the R so in this section and I kind of proposed that the relations between multi-unit activity and population signals hold the answer okay so I might just have to focus on this for a little bit oh this is only a short section very short section that's good okay this will be quickly so local field potentials reflect the common synaptic activity of a population of neighboring neurons and spikes are short time high frequency content signals reflecting individual cellular activity right so individual cells neural synchronization can be evinced one of my favorite words there they can be evinced by temporally relating spiking activity to the background oscillations of LFPs and this relationship has been documented across multiple brain regions and cognitive functions so this spike phase coupling has functional consequences and there's some examples there if you want to if you want to consult okay so that's the coupling between the LFP and the actual spike the individual spike so that's kind of an analogous right if you think about it to what I mentioned earlier in the low frequency space right I mentioned the delta theta coupling and the you know the delta gamma coupling etc this LFP phase coupling is essentially the same thing just on a smaller scale right so it's kind of nesting all the way down and so LFPs are highly effective means of exposing what state a given cortical region is in since they capture general dynamics not specific to any individual cell and this is really a really cool point actually about LFPs they actually host you know types of information that are not detected at the spike level so that's a very kind of an important point there and also of course you know there are cells that don't spike too right so in the same way that there's likely much information available at the LFP level that's not represented at the unit level like dynamics in the major properties exist that can only be detected at the level of some activity of a million neurons so too is expected to be the case that there are aspects of cortical computation that are only represented at the international global scale even at the state scale like you know hidden Markov scale kind of ultra-aggressive hidden Markov models and so on you can kind of generate these more abstract state spaces across networks of brains networks of reasons I'm sorry and I think that will also be relevant at some point too in the future but maybe I'll get to that at some point later and this and that information is not represented at the LFP or spike level so that basic finding empirical finding is another core motivation for the Rose model the fact that you do find this dissociable level of information representation across these scales already tells you that it's very likely to be this scale of computational complexity at the linguistic and non-linguistic level too and like I said it's a basic position of Rose but it's also a presupposition of other major frameworks in cognitive neuroscience not just me I'm not the one to kind of propose this so an example I counted it here is working memory which seems to involve discontinuous bouts of spiking activity as opposed to steady state neural dynamics so often it was initially believed that working memory in order to maintain something of working memory there's got to be some kind of intuitively direct neural correlate which looks like it's maintaining or staying stable level or amplitude steady state or increase over time etc it turns out not to be the case okay so relating R to O is far from trivial yes that is absolutely correct consider how communication through coherence has typically been assumed to reflect phase synchronization between oscillators recent work has offered an alternative mechanism through which coherence is the consequence of communication and emerges because spiking activity in ascending area causes post-synaptic potentials in the same but also other areas that makes it more complicated so these authors identified afferent synaptic inputs rather than spiking and training them as the principal determinant of coherence opening up new directions for framing the relation between units of coherence so LFP coherence appears to be determined by two factors coherence due to the direct contribution of afferent synaptic inputs and coherence between the sender LFP and the sum population spiking activity in the receiver so therefore coherence therefore depends on connectivity strength and oscillation power and does not need purely oscillatory coupling or spike phase locking in a receiver and then further complexities arise here I mentioned some examples from our neighboring field too but I'll kind of skip over that it's an interesting topic but still you know successfully relating the two fundamental signals in the section spiking LFP can provide us with a comprehensive explanation regarding the neurobiological cognition and since many signals picked up by the LFP will also very likely be able to be found at the unit level care must be taken to map out assembly level effects from single unit responses and that's kind of an empirical caution so this kind of section is very I think it's very honest I think it's very upfront about the limitations and potential you know countervailing trends in the literature that provide some obstacles to this grounding but nevertheless I think these things you know are true there's still the possibility for multiple types of phase codes multiple types of neural codes within the same space being involved in different types of cognitive operations or across different core knowledge systems and of course you know given that we know that human syntax is in fact a potentially species defining property I think it's you know care must be taken to really separate out be separated out from these constellation of domain general neural mechanisms which might be relevant but might also not be relevant right it's again that's you have to kind of weigh up the likelihood of a domain general versus non-domain general process being relevant for your concern and again you don't know until you know you don't know until you do the empirical research and but I think you know the purpose of this section is to kind of open up the space of you know possible alternative mechanisms okay I'm going to skip over this section this is just discusses gain field mechanisms I've written a few papers about this it's a very interesting topic just kind of grounding the kind of lower level accounts of why you know what it is what it is we know already about what the computational properties of individual cells might in fact be and that might sound a bit abstract but there's some really good whereby W to come to Fitch and Randy Gallister and a bunch of other people that I cite here just exploring the potential computational kind of you know classical Turing level architecture you know Chomsky hierarchy kind of computational you know facility that these individual cells might have in terms of the operational power okay and then this section here just introduces kind of a more classical distinction between sharing Tony and hot field interviews that's basically just a distinction that focuses on the transformation of signals by nodes in a point of view in cognition as the result of patterns of node to node connections as opposed to the viewing representational spaces through which computation is considered to be the transformation between spaces so it's kind of a more different ways of viewing populations and units, the relationship between units and their population dynamics and I kind of speculate that's you know evolutionary older brain areas might be explained better via sharing Tony and accounts whereas more recently evolved structures in hidden language right might not be so that's a potential avenue that could also direct a way of exploring the kind of empirical space beyond this and then when it comes to the operation space I've already mentioned much of this stuff I'm invoking high gamma activity and I am talking about the different dimensions gamma is not just a unified you know construct either neuro chemically or genetically in fact it's a very complex structure indeed across species across brain regions you get physiological gamma responses in subcortical structures you get kind of high rate of responses in cortex gamma itself is just a range but it's a very complex manifold and so I kind of this section kind of takes care to break down what I mean by gamma a lot of papers in the neurobiology of language just kind of invoke you know gamma representations or particular arbitrarily cut off you know range band range this section try to take a little bit of care to just unpack the different types of gamma different types of aspects of gamma in terms of their sensitivity to different aspects of representational complexity so lower versus high gamma you know broadband narrowband gamma etc. if you're interested in that this is a good section for you to consult okay so turning to structure this again kind of review some of the things I mentioned earlier to do with the fact that you know every linguistic structure is sensitive to some aspect of lexical and semantic information like go here into some kind of phrasal or sentence structure there's a phrase by Archimoran there's no escape from syntax that's absolutely true you know everywhere you look in language every time every kind of phenomenon semantic pragmatic even phonological there's often a very direct impact of syntactic information in terms of the coordination and the instructions and the influence it has on these other levels syntactic structure really is the most kind of fundamental level of language that has influences a bunch of different domains and I've already introduced Merge so I'll just kind of skip over this this is just a kind of more set theoretic classical definition of what we mean by a featureal combination of the kind involved in language and of the kind involved in the kind of work that I mentioned earlier and of the kind involved in the the hydrogensigamma and theta dynamics that I mentioned earlier too so Merge is basically the most fundamental operation in language it's the basic property of language as Tromsky calls it again from a more philosophical perspective I always found it interesting that even though this is a very rudimentary set formation operation if you think about the cognitive consequences as people like Peter Hallour have in a very nice paper the computational system of language seems to be related to neural signals integrating perception and action providing humans with novel modes of planning and interpretation whereby lexical units and unification processes like Merge provide an imaginary space that transcends the influence of direct perception action cycles that's a really key point there because of course we know that one of the real benefits of language and potentially why it was evolutionarily selected for is because it allows us to plan, interpret, think consider personal responsibilities consider our own future our past and so on construct a notion of one's past and where you've been constructing narrative and so to speak we know that other animals have memories but maybe none of them have a past maybe only humans have a past it's a unique kind of concept that involves a different epistemological transition and a linguistic transition too it's a fundamentally linguistic notion same with the self the concept of self who I am and who you are are you and so on the existence of our system in language for example all of these things are extremely cognitively rich and semantically very important concepts they're not just there to facilitate communication they're not just there to facilitate information transmission they really are part of the mental architecture that we have as human beings so I think it's an important point in that how good it makes there the fact that this really is the centre of the kind of cognitive neurosciences okay excuse me this section just introduces syntactic features what we mean by a particular syntactic feature I mentioned lexical features quite a lot but this section kind of just briefly touches on syntax specific features and it also reviews some recent work looking at the role of endogenous oscillations in neural computation I mentioned some of the stuff earlier to do with low frequency sensitivity and entrainment to particular moments of syntactic structure building this section here just kind of unpack some of that more exhaustively and give some more recent references there so this section kind of just reviews that literature skipping through this part here and again just kind of to recapitulate what I mentioned earlier the role of low frequencies in indexing supra lexical structural inferences is very well empirically supported so the previous two pages that I just scrolled past they really kind of propose summarise a lot of literature suggesting that lower frequencies indexing supra lexical so stuff above the individual weird level it really is pretty undeniable at this stage I think okay and then we turn to the more issues of timing so what is the precise timing of syntactic information processing this paragraph here just kind of breaks down some of the particular periods of sensitivity that have been already documented and how it is compatible and maps on to the predicted time course dictated by the high frequency rhythms right so it's no good me just saying oh delta is involved in syntax and you know gamma and you know semantics or whatever you really need to have a motivated time course in real time like a real time passing constraint on how these high frequency oscillations can embed themselves and be entrain and phase locked to these higher these lower frequency rhythms and so I kind of break down the time course of activity here and it's all kosher it's all you know empirically motivated it's all kind of aligned with what we know about both about delta and about the intermediary frequencies too okay and then this section turns to again the mapping process that I mentioned the mapping from the O to S structure so this kind of invokes again a separable mechanism I'll just scroll up to the top here just to have a quick refresher so we had phase amplitude coupling as I mentioned right so this is spike phase coupling mapping R to O, mapping O to S is phase amplitude coupling so that's the kind of classical process that I mentioned earlier to do with theta gamma and delta gamma coupling because I get you directly to the higher order structural influence okay so this section is just a short review of the phase amplitude coupling dynamics in this particular process and also talks a little bit about some other kind of extra cranial recording measurements that have kind of unveiled this in other kinds of domains this section here dissociating structure from meaning this is very much for the linguists this is kind of trying to figure out how we can carefully account for signals that are specific to the semantics of a phrase versus signals that are maybe maybe not specific but certainly sensitive to the syntax of a phrase that's a very tricky issue you have to kind of basically design an experiment or stimuli that is getting participants to pay attention to structural inferences and syntactic information and syntactic acceptability and not pay too much attention to other kind of features but also the stimuli itself have to be carefully weighted across you know words that preferentially engage syntactic information over and above conceptual information or conceptual semantic information versus words that very much do trigger heavy semantic content so that involves a lot of careful kind of empirical consideration our lab is currently carrying out some research into that space as are some other labs too but it's basically an appeal for focusing more laser like on these ways to dissociate syntactic from syntactic syntactic from semantic processing using empirical novel kind of experimental paradigms because a lot of the paradigms that are used at the moment have a big problem which is they don't really carefully separate out moments of syntax from moments of semantics so it's very difficult to kind of realistically and you know kind of confidently make it clear that a given brain signal or a given response is really being driven by either syntax or semantics okay in terms of skip over this, no I didn't okay and then this outlines the mapping from the S to E levels and it talks a little bit more about some of Standard Hane and other people's work into vector codes for population dynamics to do with what level of representation we might find individual syntactic features at and again I transition slowly here to talking about traveling waves, how the kind of the more kind of static code that I've presented so far in terms of just you know a delta, a theta, a gamma how that static code might be transfigured into something more really neurochemically plausible in terms of what we know about the transformation of information across white math tracks and how different portions of cortex cortical cortical information is kind of transferred in real time across different portions of the cortical mantle it's not just stationary okay so I guess what I'm calling for here and as I call for in the next section is a more kind of spatio-temporally dynamic model or code for language essentially excuse me so all of this section essentially needs it kind of calls upon the notion that you need to map on an important you know important findings from the psycholinguistic domain in terms of the timing of language the kind of you know when we know that language is coming involved which periods of activity are going to be important and the actual neural architecture that's kind of a very tricky question how you map on those so for example you know if a psycholinguist says okay we know that when people using eye tracking or using scalp ERP responses we know that the period of 300 to 400 milliseconds is when most people kind of detect semantic violation effects we really then have to say okay if that's the case does that time period at the behavioral level map on to moments of you know neural activity also at that level or maybe essentially the neural response comes on a little bit earlier before it's detected the scalp level or maybe at the single unit level it's even earlier or vice versa or maybe there's a kind of top-down effect whereby some kind of intermediary mesoscale you know neural signal can be detected first before the spiking information is it you know there's different hypotheses that arise here in terms of the directionality and causal relation between the R to D levels and in this in the rose paper I make it you know very clear what my empirical predictions are but you know other alternative ways to modify the rose model could be in terms of the directionality from different levels right so for different linguistic structures it could be the case that with some of the more mesoscale levels in fact influence the coordination of spiking rather than spiking information being read out at kind of high order levels which then facilitate the inference is being made the directionality is very clear in the paper but it's kind of open it's very much open for empirical you know resolution essentially okay so in this figure figure two figure one is the basic kind of you know components of the model figure two is the basic mechanisms of the model so this just goes into some more technical details that I won't go into to do with the actual nature of spike LB coupling what the kind of mathematical foundations of it are and how you can you know map that on to a kind of classical kind of low frequency response and how these different signals could be coordinated in real time okay and then turning to the encoding section as I mentioned earlier we know that the human mind needs different workspaces for different aspects of cognition so the big question is does language share a workspace with other kind of you know non linguistic processes is there a specialized unique workspace for linguistic information it's possible and I think it's likely in fact which is why I have it in my paper and so in fact we know that and there are some non human primates that can execute a very primitive and combination and processes like potentially even more flexible composition just combining to calls you know like primary vocal calls together into a basic primitive and unit and can they then do it again and again and again can they do they have the workspace that allows the facilitation of storing this one unit and then creating a new structure and then you know assigning that whole complex and identity that is independent from its discrete parts which is what merge is all about in language they don't seem to be able to do that so that tells you I think quite reliably that we do need some kind of notion of human specific syntactic workspace which is what the actual encoding level is meant to be in rows and okay so this section kind of just plays out a little toy example of what how to derive a particular structure and what the whole process would look like going from R to O to E so I think this is important for me to kind of outline since it's the part of the paper where I kind of really do give a concrete example so let's consider the sentence old men walk slowly during the convention of the first two words the delta gamma combinatorial code coordinates the feature bundling of the atomic data structures hosted by all the men so all the men have various features and minimum disinvolves posterior superior temple circus low-framed activity coupled with neighboring posterior temple cortex but also cross cortical sites responsible for the specific feature types in question so I've already mentioned places like ATL and IFG and then I also mentioned a theta gamma coupling this maintains in short-term memory I mentioned the lexical memory book before a theta gamma coupling maintains in short-term memory the relevant units via high-frequency activity and obviously oh with the gamma in a linear sequence so this is worth by David Popell to the theta gamma coupling is a linear feature clocking mechanism that tells you these are the set of features features one to five with one to seven and you're going to access them and trigger them and read them out in a given sequence so that's a kind of linear lexical memory store it doesn't give you the hierarchical syntactic relational component that is independent of order that comes later okay and at the transition between men and walk so with two and with three the superordinate delta theta code maintains the categorical identity of the object so you know old men is obviously men that are old right as I mentioned it's not an old quality that happens to be men so that's a noun phrase when you get to old men walk you relabel the object and it's no longer a noun phrase right you have to recategorize the syntax and it's not a verb phrase it's a old men doing something it's an event structure so things like event structures and you know quantification structures and all the rest of it they involve very specific unique semantics and that are informed and provided by and configured by the syntactic category so that's why I mentioned that you know there's no escape from syntax right the syntactic category feeds the semantic information okay so when you get to the third word walk the superordinate delta theta code maintains the categorical identity of the object in this case the negotiation between a multi-unit noun phrase and a more complex verb phrase hosting old men so during the same period the lexical memory code again if you remember the lexical memory code is the theta gamma the lexical memory code increases its number of theta nested chunks due to the occurrence of walk right so old men walk is in terms of the theta gamma code all the theta gamma code sees is another word when you get to walk all the theta gamma code sees is a third word that's it it just sees okay another way it's coming another word another word and it just simply chunks those features together using the feature cock up clocking mechanism that I mentioned but that's not what the delta complex is right that seems something else so the superordinate delta theta code maintains the identity and and in this negotiation between and in this case the negotiation between a multi-unit noun phrase and a more complex verb phrase during the same period the initial lexical memory code increases number of feet and nested chunks due to the case walk the same transition occurs from walk and slowly with the exception that while the lexical memory code still increases in conflict strength that's the measurement of pack relations between the feature and gamma right high pack relations the hierarchical memory code would decrease closer to but not identical it's pre-verb baseline due to the adjunction relation not demanding a revision of the hierarchical memory representation so what the hell does that mean and so when you get to old men walk you change the syntactic category it's a verb phrase so that involves a activity at the delta theta level when you go old men walk slowly the word slowly does not change the syntax it's still a verb phrase so old men walk slowly is a verb phrase old men walk is also a verb phrase so slowly changes the lexical semantic information so you keep getting theta gamma pack increase but you get either a steady state or a decrease again I predict a decrease of the delta theta pack relationship because all you're doing is maintaining the same syntactic identity you don't need to change the syntax same same syntax again I talked about syntax feeding semantics if you think about it for a second and old men walk slowly is still involves old men it could be old men walk quickly old men walk you know methodically whatever it's still the same thing so the fact that old men walk slowly doesn't change the fact that it's still old men whereas with old men walk that does change the fact about old men so when we get to old men we have certain facts we know about old men when we get to old men walk that changes things we know that it's old men walking when we get to old men walk slowly that's a modificational structure it's called an adjunct the adjunct structure or adverb or prepositional phrase anything like that simply it doesn't it adjoins so that only involves basic set of formation it doesn't involve a re-labeling of the category itself because the category is still the same so this gets into some theoretical syntax ideas that are not important here but the basic idea is that you just get an enhancement of lexical semantic information but the syntax is identical so that's why you get this kind of wave of old men walk and then slowly I mentioned some of the delta peaks before this kind of stimuli old men walk slowly is what has allowed researchers to figure out that you get those low frequency peaks at the third wave old men walk slowly because that's the period where you really change the syntax and it turns it into a kind of sentence I guess a full sentential structure with a prepositional identity truth evaluability it can be true it can be false you can afford all sorts of different epistemic judgments to it that's the whole kind of you kind of get a whole complex of instructions being sent to conceptual systems at that stage okay so the transfer of relevant lexical information from categorization or labeling that I mentioned would take place for interactions between these two neural codes right so separable codes I speculate here potentially with theta and phase phase coupling or phase locking of the gamma workspace and the delta and theta driven dynamics effectively constitute the handoff of information after lexicality has been established by lower level R and O processes transitioning from encoding lexical memory to multi object memory and direct testing of these dynamics specifically with respect to syntactic workspace instruction has currently not been undertaken although much work has been carried out demonstrating increased theta gamma coupling in human hippocampus during memory formation as well as enhanced frontal theta to posterior gamma coupling alongside the recent discovery of a rapid neocortical beta network mechanism for flexible information encoding and familiar operations from left corner memory list grammars can be appropriately pass slated onto these R, O and S levels that's a bunch of jargon but the Lingus will appreciate what I'm talking about there I hope but future work should still treat this issue of precise passing models with more care intact than I have demonstrated here in this initial architectural proposal so what I'm trying to say there is that I mentioned Mars computational, implementational and algorithmic level in the middle I've talked a lot about computation and implementation I've only spoken a little about the algorithm the thing connecting the wetware the kind of biological stuff you can touch and see with the more abstract algorithm in between the computation and that's the domain of what's called cycle linguistics so the field of cycle linguistics has a bunch of different passing architectures for how humans pass sentences there's rival models lots of them have different predictions for how human beings pass different types of weird sentences in real time I have my own favorite candidate I've mentioned left corner minibus passes that's kind of my own favorite account for all sorts of empirical and theoretical reasons but it's an open question and if I later I kind of discuss the possibility that different parts of the brain might implement slightly different passing models so the posterior temporal cortex might be a good place to find a minimalist grammar but the inferior parietal cortex or the inferior frontal gyrus might be an excellent place to find a different kind of passing model and I cite a bunch of examples in the paper somewhere in here I think but that's again a bit of a question but it's also kind of a more I think it's a more pluralistic way of kind of viewing language in the brain it's not just like minimalist passing grammars explain everything the entire brain is in concordance with the predictions of minimalist grammars it might be a kind of more a different kind of consolation of passes that we need but still nevertheless with the basic primitive computational architecture being infacted by moments of merge and labelling in kind of more traditional basic assumptions from genitive grammar okay this moment just yeah this period just turns a little bit more towards traveling waves and a little bit more to do with fMRI research, kind of building the paper more to what we already know from the extra cranial world this section talks about syntactic memory as being a phase synchronization over successive cycles so this invokes a bunch of interesting work and consult some models from the world of mineral biology oscillations more generally in terms of phase dynamics and all the rest of it just to kind of situate what the model might look like in a more kind of mathematically rigorous feature space and the section on symbolic computation turns a little bit more to what we already know about infact I think I should review this very quickly because this again comes back to the issue of domain general so russian directions and the neurobiology of navigation and memory are relevant to the conception of e so hippocampal cortical sequence replay and encoding is not constrained to simply repeat past experience rather this process is informed by an internal model of the world generating representations of inferred entities not necessarily encountered physically again this is one of the reasons why I work with people like Carl Friston because I think this whole notion of building a general model of one's environment the interplay of action and perception cycles and so on that's really a fundamental you know, negotiating a building block of what language gives us of what the role of language really is in the human mind it's really about interplaying these two different divides of the human brain so this active generative capacity motivates the authors to propose that replay in the brain instantiates a form of compositional computation so a given replay sequence constitutes a set of entities strung together into a compound whereby each entity is bound to a representation of its compound role determining its function as part of a whole this establishes a clear separation with respect to composability between entity and role or what's called syntax and semantics right the entity and the role so while roles encoded by hippocampal cortical interactions can certainly be spatial they can also be non-spatial and even non-spatial and non-euclidean potentially involving arbitrary roles such as a verb or what we would call some kind of event anchor so the entity role bindings currently explored empirically in humans are limited to things like which position and which sequence but if other roles like if then else can be encoded in a similar way then replay may form a viable candidate for a neurophysiological mechanism implementing symbolic computation so this compositional nature of replay is implemented via our friend data gamma coupling ensuring closely the present assumptions of lexical feature sequencing and basic semantic compositionality being implemented via the same dynamics and high gamma activity and there's a nice paper recently by Nina Kazanina and David Popel that explore very similar notions they kind of argue that a lot of the neural mechanisms already known to be found in hippocampus could be used for a symbolic language of thought in language language of thought meaning the classical so a lot of what's already known outside of language about the neurobiological infrastructure things like navigation could be utilized and exploited by the language system in order to achieve its goals of composable functions compositionality and basic structure building so that's again once again it's an empirical question these are all empirical questions but I think the purpose of this paper which is a purely theoretical paper is to look at this terrain and see which candidates are more feasible more likely etc and I think this candidate in particular I'm not the only one who thinks this is a very viable candidate for grounding some of the initial again the stress is initial for me it's initial it's just that initial phase of the early phase of it it's the basic set formation process the feed to gamma stage potentially not the best methods that involve other mechanisms that I mentioned but at least in some aspects of language lexical search, lexical access morphological complex word binding and so on it's likely that these things are potential to use as well okay and then in terms of so memory transfer this is an important concept that's been raised recently by Brinkett et al and Elle Miller this section is very briefly reviews how we might transfer memories across cortex which might look a little bit like transferring syntactic structures into workspace or transferring stored lexical items into a separate kind of consolidation or monitoring workspace there's some kind of potential for research in this domain and then turning to alpha beta dynamics this is an interesting topic but I mentioned briefly earlier to do with how the role of these oscillations that are outside the main code space that I already mentioned I've already mentioned I've delimited my code space what about the role of alpha and beta I mentioned it briefly but it's kind of a quite an important role given that they are quite dominant they're very prominently found in intracranial and extracranial research into language so just focusing on syntactic memory there's a recent paper Garak et al which supports a role for beta in syntactic identity just being sensitive to the identity of the other sentence the type of category that you afford it these authors investigated speech memory representations using intracranial recordings in the left carousel of cortex during delayed sentence reproduction in patients undergoing awake tumor surgery based on the memory performance of patients they found that the phase of frontotemporal beta represents sentence identity in working memory the notion of sentential identity presupposes a labeled or categorized structure like a verb phrase seemingly represented partially by frontotemporal beta and converging with other literature beta may represent aspects of the global cognitive set going beyond syntax specific information through include conceptual and statistical information again I mentioned the role of beta in syntactic anticipation syntactic prediction the kind of statistics of language and I think that has an important role to play here again and intracranial recordings by yours truly of auditory language comprehension also implicate frontal alpha and beta power in phrase prediction and anticipation they kind of mark the moment of potential anticipation of a licensed phrase another paper found that both peter and gamma are sensitive to syllable rate but only beta power is modulated by comprehension rates so comprehension obviously implies you know semantics rather than just syllables and over the past decade in fact a little bit longer than that over the past 15 years or so there's been a debate about whether beta power affects during sense comprehension reflect syntactic computations or instead reflect maintenance or set updating and to briefly summarize the most current results it appears that the latter maintenance hypothesis is most well supported and so I'm therefore going to put further discussion of this on one side but the basic role of it in maintenance in terms of just continuing aiding with the assisting in some way of maintaining the set of generated and sustained cognitive representations not necessarily being sensitive to the properties of this set not being sensitive to the particular syntactic identity or the category of the syntax or even the category of the semantics or the features of the semantics but at least being involved in coordinating neural resources available neural resources to help with the sustaining of that memory in a current space again this is very domain general not specific to language which is why I think it's you know you don't you find fewer of the language specific syntax semantics sensitivities and even though it's well comprehension as I mentioned comprehension means everything it's just like you know proceeding a sentence means loads of things it doesn't mean specifically syntax specifically language specific stuff it just means you know when you when you when you process a sentence you do all sorts of things and so I think it's most likely that it's involved in these kind of more domain general maintenance processes and then in parietal cortex alpha enhancement seems to index syntactic working memory demands much like the regulation of gamma by alpha in control and attention mechanisms that I mentioned earlier parietal cortex may play an important role in memory and complexity such that low frequency rhythms originating in lateral parietal cortex regulate the activity of gamma encoded operations in lateral temporal cortex and single unit or assembly encoded are in medial temporal or inferior from the cortex that's a more kind of specific reversal and another likely site of syntactic working memory is in prefrontal sulcus for reasons cited in this paragraph it's very heavily involved in all sorts of interesting semantic integration processes and then yes as I mentioned the kind of bringing this model to a kind of conclusion and these assumptions take place all within the context of the traveling wave frame right so lower frequencies migrating across the cortex particularly being relevant to cases of maintenance and storage so traditional standing waves lead to periods when all neurons in the network are turned off whereas traveling waves can ensure that sub portions of a network remain consistently active directly compatible with the mosaic like architecture of posterior temporal cortex in semantic integration I won't explain that further but it's basically the idea that within these small regions of cortex like posterior temporal cortex there's a very interesting patchwork of activity of little satellites of sensitivity to different aspects of language even though they're all generally sensitive to language within them there's kind of more specialized regions which is not unlike what you find in ventral temporal cortex for things like face and place and other kinds of shape and perception kind of ventral visual acceptable cortex being having a kind of tapestry of you know cell specific sensitivity to different representational features and so where exactly the waves travel to during phrase composition is again my favorite phrase is a purely unempirical question but some kind of regions have been suggested about right that's kind of where we stand at the moment that's where the field is at the very moment and so figure three here is the basic processes of the rose model and this kind of just gives a general outline of how rose is hypothesized to be implemented across the frontal temporal language network and at the bottom there's a representation of the various frequency interactions proposed to implement very flexible in so you have representations category specific information being coded in spikes operations you have broadband gamma activity initially I hypothesize in PSTS based on the intracranial work that we've done but it could be anywhere it could be in PN2G but also be in portions of IFG depending on the category potentially of the phrase the type of structure you're generating that's an open question very much open in terms of structure we have the low frequency neural program driven by temporal cortex and then for encoding we have these traveling waves that are basically my great structure so this I also think that given what we know about the speed of traveling waves I think it's possible that a lot of this stuff transfers up out of your door sort of ventral pathways across posterior to anterior temporal cortex and from posterior to anterior frontal cortex given the term scales of sensitivity that we know exist there okay this final section of the paper talks about causal evidence which is increasingly becoming very important these days in your science as we've exhaustively mapped correlational activity of loads of different linguistic processes but we haven't really got a good sense of which parts of the brain are actually essential for language so when you get regions X, Y and Z being activated region X might be causally involved whereas regions Y and Z might just be activated or recruited or implicated but not actually essential for the function so this section is kind of charts out some of the ways of exploring that empirically through cortical stimulation mapping in the OR or in the AMU using TMS for example or joint TMS meg studies that's a possibility there's also a way you can think of kind of providing different portions of causal evidence for these models and these paragraphs here just kind of just review mostly non-linguistic but also some language research into how people have used these methods to disrupt language in particular parts of the brain there's some more interesting research disrupting particular phase codes trying to modulate either hyper or hyper-excite particular acetylodynamics or packed dynamics in different parts of the brain now that's a really good way to directly test the causal involvement of a phase code more generally because if you can hyper-excite a theta rhythm or hyper-excite a gamma rhythm then that can really screw up your facilitation of the coordination of these electrical features and your attention and working memory and all the rest of it so that's a great way to kind of causally test not just a brain region's involvement but also a particular brick like how a brain region is involved this section is much more philosophical if you guys play Elden Ring you might get the reference in this section but the point of this section is to kind of evaluate the Rose model to kind of situate it in a large context to relate it to existing models in the inter-cranial space and extra-cranial space too so some of the important periods in this paper so I've already mentioned this paragraph I think to do with how how different parts of the brain might be sensitive to different passes so I'll skip that bit this paragraph here just outlines how the very basic philosophy in Rose is that mesoscale neural organization can be useful for brain function and the hyper-excite gamma oscillations report mechanisms and underlying communication channels of neural computation so neural oscillations organize cortical activity to produce computation and Rose builds an architecture for syntax that presupposes this some exciting prospects for testing Rose come in the form of multi-channel recordings with broad cortical access using binary micro-electrode arrays etc etc and I just cite some more technical examples there for people in the field and then let's see what else do we have here yep I've already mentioned the inside out outside it well actually I haven't mentioned that but just very briefly I think my model is incompatible with a bunch of recent ideas to do with encouraging people to explore the brain from inside out so I'll just read this paragraph and then maybe try and digest it a little bit because it's probably the most important philosophical kind of grounding here so brain models informed by computational concerns will continue to be needed in neurosciences yes that is true in particular as we approach the advent of widespread availability of single unit recordings and again one of the anxieties I have with this paper is that as linguists get their hands on sophisticated neural recording measures we really need to be careful and quite conservative in knowing what we can say with each of these measures so if you give every linguist on the planet access to single unit recordings in the human brain you're going to get a lot of papers making claims about as I satirized here Neo-Davidsonian existential closed neurons or specific very niche kind of cell types responsible for all sorts of baroque and complicated and not very cognitively plausible linguistic constructs so I really wanted to in this paper boil it down decompose language into its primitive components decompose at the computational level and decompose at the representational level that kind of goes back to the 1980s kind of classical framework of the computational representation of theory of mind where you have a series of fixed representations units and fixed computations or things that you can do with those units this kind of actions and objects right that's a basic metaphysical human distinction between things and processes actions objects and all the rest of it right nouns and verbs and computation representations and so I really wanted to make it clear that and you know next time you find yourself with access to a single unit dataset you need to kind of have sufficiently decomposed these levels to make it kind of neurochemically and neurobiologically plausible rather than talking about the kind of more highfalutin stuff that I mentioned earlier so it's kind of more it's more of a cautionary tale I guess because I you know other fields have gone through this before right and so for example you know grid cells right it's it's over the last decade grid cells in entorhinal and paracampal cortex were initially implicated in spatial navigation then it turns out they're involved in order to a navigation and then they're involved in conceptual navigation in a certain of words they end up just being involved in navigation full stop just all types of navigation and I think a similar kind of transition might happen with when we get an increasing number of publications about language in single units you'll get a lot of people initially saying oh well you know these units these specific cell types are involved in X then it turns out they're also involved in Y and Z and we'll ultimately end up being forced to propose a more generic account for these things which is why in my paper I try and you know focus it I try and appeal to as much generosity as I can there's always going to be some domain specificity but hopefully language has domain specific representations and apparently domain specific computations so we will need some notion of domain specificity on some level but the only question is like where are you going to find that okay I'm rambling a little bit so I'll just I'll continue and so this outside in perspective has been critiqued recently and and and and it's out of perspective on building neural models of cognition and considers the classical marine framework really outside in perspective yet more himself stressed that the three levels should be investigated in parallel not necessarily prioritizing any given level so this is a balancing act though right too much outside in and you're going to get these near-Davidsonia neurons too much inside out and you're going to be told that linguistics departments need to shut down right so regardless of your own philosophical bent I think single unit and other types of intercranial recordings are plainly the most direct and reliable test in refined rows in particular given recent independent assessments and critiques of both activity but you know the basic message here is that and a purely inside that perspective is when people will say well what's called data-driven science or data-driven neuroscience right where you say well let's just do a bunch of sophisticated statistical analyses on our single unit data and see what kind of a measures and let's just see that let's just hope that language will just arise somehow without any ideological presuppositions or any kind of biases first of all it's impossible I think for any human being to do that even if they tell themselves they're doing it you're always going to be delivering some biases you're always going to be bringing some philosophy of science or philosophy of mind to the table whether or not you like it or not and on the other hand you have the completely outside in perspective which I'm also very much against so the outside in perspective would say let's you know everything that everything that linguists say is bible and all these theories of double rates movement and pipe piping and all the rest of it they are literally what the brain cares about and we have to find that in the brain we have to keep our fixed theories of higher order cognition and just look at the brain and expect the brain to care about everything we care about we expect the brain to be sensitive to all the stuff that we as linguists care about and the problem of science is that you know there's always a negotiation going on there's always a kind of deal that needs to be made so you know one of our former presidents talked about the art of the deal in this case the art of the deal in this case is figuring out how much linguistics you need to hold on to and stay true to and how much you need to let go of and how much you need to say the brain doesn't care about this you know it only cares about this but not that so my own feeling is that the brain excuse me the brain does genuinely care about some aspects of language domain specific otherwise namely the representational feature types and also this business of marriage and labeling that I mentioned the basic computations I think the brain does genuinely care about that it probably doesn't care about all the other stuff that linguists talk about but it can be characterized at some kind of more epiphenomenal level I guess at some kind of emerging phenomenon we can talk about the philosophical details you know in a kind of frameworks if you like but that's the kind of I think way to go about I think it's much too harsh and arrogant to just completely abandon all of the insights of contemporary linguistics just because we have sophisticated neuroscience and I think some holding on to some insights from linguistics will be essential to direct and test and you know refine our data driven science too and so I think what Brizaki talks about in his book is I think misleading and kind of a you know unhelpful division I think it's not I don't like to talk about inside out or outside in I think it's kind of just you have a balancing act between both right it's you have to just negotiate when it's appropriate to use computational levels and when it's appropriate to use implementation level theories and okay so that takes care of that and I'll skip over this bit here and yes so this this this gets into what I mentioned I mentioned this maybe a little bit briefly and but it's very common to object to these kinds of proposals of the kind I mentioned in rose as a kind of crude form of direct mapping right or isomorphism and I'm sure some people listening to me right now might think well you're kind of a hypocrite because you're doing exactly what you're critiquing Brizaki we're doing right and complaining that Brizaki is just I'm sorry I'm complaining that these outside in people are just imposing their linguistic theory on the brain and I'm saying you shouldn't do that at the same time though the whole point of the rose model is to directly map on what different types of linguistic structure onto the brain so a complete nihilist might say well the brain really doesn't care about any of these structures the brain really doesn't care about atomic features lexical items, feature bundles, phrases you know sets of sets of sets and all the rest of it now that might indeed be true that's totally possible I think it's unlikely given that you know the specific responses that we've seen at different levels of linguistic complexity the kind of dissociable scales of sensitive that you can see at different neuro complexity scales that I mentioned here but so you know the rose model is essentially invoking a direct mapping in some way a kind of one to one correspondence between a scale of linguistic complexity X and a neural mechanism Y and you kind of you know you follow that all the way up until you get to the ultimate the highest level of nesting that you get with the delta theta complex all the way down to the spike at the peak I mentioned earlier right okay and so some people might say that's a problem rose continues this crude tradition and by establishing an even more explicit isomorphism between system level complexity and syntactic complexity so for example cells and syntactic features versus global coherence and phrase structure and alternative accounts that build syntactic operations into neural systems of diverging levels of structural complexity have not been forthcoming by the way so if they do both come then we can test them as well and what would also naturally be required to establish any kind of adversarial collaborations and experimental testing of the models and so through this it would be possible to pick different candidate neural mechanisms for syntax against one another and compare effect sizes across studies to see which mechanism either from rose or some of the model best explains real time direct political effects of language processing so that's a nice way to directly test the rose model but I also want to emphasize here that as far as I know the rose model is the only model of syntax non-neurobiological model of syntax that takes seriously this whole notion of the fact that the brain genuinely exhibits distinct levels of neural complexity organization and scales of complexity and also takes seriously the fact that not all neural recording measures just measure I think right, single units up to utars, up to segs up to ecog, up to MEG up to scalpy eG these are very different beasts and they record very different things it's easy and convenient and maybe comforting just to say well the kind of signal that we see in scalpy rp responses in delta waves are probably just some kind of summated phasory set of whatever is going on at the spike level and on some level of description that's true however it doesn't help with building a model of syntax in the brain that's just kind of dismissal so my appeal here is to kind of ground it more concretely in different levels of analysis and so far you know all the other models of syntax in the brain as I've said are localizationist as opposed to kind of algorithmic or that's why the subtitle of this paper is a neurocomputational architecture because it's a neurocomputational architecture and it is very much an architecture within the Rose architecture you can imagine three or four or five very different specific theories of or models from the architecture so you know it's very much an architecture and within this paper I propose my own specific theories at each level but the scale of the architecture scaling from the spike spike rp coupling up to pack up to trapping waves that's the architecture but so within the architecture you could maybe reframe some of these things as long as the mechanistic relations between r to o to s to e are maintained that's the kind of general framework here and then going back to metaphysics for a moment Rose is also sympathetic to recent moves in philosophy of biology to view a range of biological constructs as processes rather than objects Rose is built entirely from neurocomputational mechanisms that are already known to subserve clusters of generic perceptual and cognitive operations I'm not invoking any new neural mechanisms I'm not proposing a new kind of microtubule quantum computational mechanism to explain all this I'm being very very conservative I'm just using literally stuff that's been published in the most high impact serious scientific journals which has been afforded textbook level treatments at every level so all of these mechanisms have been given serious textbook treatment the only difference here is that I'm migrating some of these concerns over to the linguistic domain with various different motivations across the levels okay and then turning slightly further so I have a nice point here made in the Nature Neuro paper by Mina Kazanina and Alessandro Tabano current theory is that model hierarchical structure building via low frequency dynamics correlates some neural measure with attributes of a hierarchical syntactic structure and thus can send the outcome of syntactic structure construction so this is a critique of those MEG and EEG studies that show this low frequency delta entrainment or peak response at moments of syntax it's a very legitimate critique but by initiating certain syntactic structure building of the cellular R level and accounting for how these output low frequency responses at the S&E levels the rose architecture goes beyond these other accounts that are more closely tied to the output of structure building so and again even in the rose architecture this is correct that these low frequency delta peaks are indeed the output of the structure building I'm not denying that I'm not saying that the input they're not some kind of intermediate response Kazanina and Tabano are completely correct they do reflect some kind of readout at the final stage of the syntactic structure building process and to kind of wrap up or even monitoring stage and for me it's part of the structure to encoding phase but at least the early stages of syntax of the R and O levels are definitely at this kind of spike level and high gamma activity level okay and yes and then again open questions remain about how to ground this in more neurochemically explicit models I haven't talked much about neurochemistry some of my earlier papers in 2015 2016 tried to take neurochemical frameworks much more seriously and I have more of a mathematically rigorous framework into negotiating particular low frequency dynamics but in this paper I'm just kind of leaning that to one side for that because the paper is already way too long and I'm just focusing on the kind of architectural components that's something to be focused on at a later stage okay um yes yes yes okay so turning to the conclusion um you know I've mentioned a few times that this the Rose architecture really does come in response to the absence of phase coding models in language there's been a lot of phase coding responses or phase weighted you know entrainment and low frequency responses empirically documented they've been empirically documented but in terms of the actual theoretical and lagging so that's my main motivation um and many researchers who have kept the frameworks emerging purely from extraneous you know meg derived event related fields or scalp EG rate of potentials or FMRI responses etc these guys have begun to lose hope they've begun to lose hope in the prospect of finding syntax in the brain so for example Lena Pulkinan who's probably you know the world's most respected neurobiologist of syntax she's written that the neuroscience of language field has long assumed that our brains build syntactic structure during language processing today it's reasonable to question this assumption and she's you know she's totally right and motivated to say that because she's summarizing research from extra mostly extracranial stuff like meg and FMRI and she's definitely right that if you just focus on that research it seems pretty hopeless and I think the lack of spatial temporal the lack of high spatial temporal resolution in these methods explains the absence and that's kind of one of the implications of this paper and this paper is essentially indirectly critiquing that field so Pulkinan you know reasonably speculates that based on the evidence reviewed in her article something like merge might not be an obvious neural operation or alternatively it might not be found in presently explored neural signals in fact she has this nice distinction between syntax as knowledge and semantics as process where in neural recordings across all methodologies finding signatures of semantics is very easy because semantics is very energetically costly the number of words and possible concepts you can combine at any given moment is really kind of astonishing the number of different feature spaces you can talk about green horses made from licorice that ride to Mars next Tuesday you can talk about all sorts of surreal stuff right so semantics is very unpredictable it calls upon a wide range of cross cortical representational features what about syntax syntax is very different because syntax is syntax is just merge that's it you just build you put two things together and then you label it so that's it there's no novelty there every time you you build a phrase you do it in the same way you build a phrase of architecture you label it you stack it into a workspace you label that you ship it off to an interpretation procedure and that's it that's all of syntax and you do it again and again you don't have a different merge process on Tuesday as you did on Friday but semantics is like I said there's a big difference here between the space of semantics space of syntax is just this merge this one operation so it's no surprise that it's been very difficult for scientists to find neural signatures of syntax syntax specific responses because it's like we do it so reflexively and so trivially and we've been doing it since infancy maybe even before infancy depending on which language acquisition theories do you believe in whereas semantics were constantly changing it you know it takes years and years and years for us to really build our full repertoire of developed semantics with syntax once you've built a phrase you've built a phrase that's it you've got all of syntax the only thing that you can go beyond that is you know kind of performance aspects like how complex a sentence you can construct right can you read Shakespeare can you read Joyce you know can you read David Foster Wallace that's a very different question but that's a question about performance it's a question about like working memory attention you know all those sorts of things when it comes the actual basic component of syntax is just mage so I think Pilkin is right that you know maybe she's not I don't think she's right that you know mage is knowledge and semantics is process I think they're both process and they're also both knowledge too but in terms of why it's been difficult people to find a signal reliably that it's because that's all there is to syntax whereas of semantics you're going to get a very reliable very serious neural energetic cost involved in processing sentences so and in fact in our lab in our intercranial recordings for epilepsy patients all the most reliable responses you see are clearly with you know meaningful sentences as opposed to you know semantically impoverished sentences that have a good syntax because it's just difficult to really disentangle those things but that leads to some of the questions that I won't get into that'll take us to a different topic and but so you know I guess I've tried to what I've tried to do with this paper is that with rows I think I've at least provided some candidate neural signals and they do exist there's plenty of candidate neural signals some very good ones and that can you can be readily investigated in this manner so and yeah I think that kind of sells that issue at least for me and so under rows there's basically no sense to be made out of claims that syntax lies in one specific brain region I talked about PSTS before being the real site of the initial phase of merge but in order to get to structural inferences and then transferring that to a workspace that's a global phenomenon it involves most likely you know posterior temporal and fear frontal coordination there's a nice paper in PNES by Oscar Walno who's in our lab at postdoc in our lab who shows very convincingly that building a coherent sentence structure involves the coordination of overlapping spatio-temporally overlapping but distinct clusters in both posterior temporal and inferior frontal cortex so it's not just you know IFG versus PT it's a kind of like okay they're both involved but to what extent some issues are a bit tricky to discuss because you have to kind of relate this to the lesion literature which kind of more reliably implicates posterior temporal cortex strongly and for my money if I were just to contradict myself just a little bit I think the language region is essentially posterior superior temporal so I think PSTS is basically the language region with other things coming online later due to either coupling or functional coupling in coordination with PSTS PSTS will always be involved it's always the hub and these other regions like IFG and ATL are involved and called upon for performance reasons or reasons to do with a downstage readout and interpretation well PSTS is the language region for me at least so that's kind of the seat of the Rose localizationist approach at least but then the rest of the neural code spreads far beyond that as I mentioned I only read one sentence of a paragraph before waffling but I think I pretty much already said that in many ways and then we can think about testability in all sorts of different ways too to do with separate out the category of the phrase versus the symmetrical phrase that's something to think about okay and then in terms of the general roles in shaping information processing the currently identified frequency specific mechanisms seem to align with an emerging consensus in the field that slow frequencies control input sampling alpha and beta gate information flow and high frequency activity is controlled by slow rhythm so that's a very general neuro computational framework that's been explored in the coca neurosciences and then moving beyond this a newly emerging focus on brain criticality and how this concept might relate to brain rhythms provides exciting your avenues for future work in theoretical linguistic so that's a cool concept too and in fact there's some really good way exploring how in some of my earlier papers I talked about what's called a globularity which is this concept of the fact that the human brain is uniquely almost spherical relative to other primates not completely spherical obviously but a lot of cognitive abnormalities come with a brain shape reorganization and the fact that the human brain is kind of more globular than our ancestors has been used to motivate a lot by a lot of serious anthropologists and also neuro biologists this idea that the human brain is kind of uniquely efficiently wired in some way to facilitate a cross cortical and also sub cortical cortical information integration so I think there's something to be said there to do with an evolutionary change in brain shape globular brain shape would have naturally facilitated different paths with these travelling waves to move across different patches of cortex can communicate with each other and in fact we know from work by people like Liz Spelke that the basic computational kind of contribution of language is to provide a kind of universal currency or different conceptual domains to talk to each other excuse me so a language is basically giving you your number sense your sense of morality your sense of intuitive physics intuitive geometry intuitive botany what is all sorts of different domains concerning how human beings have intuitive modules for understanding reality and constructing a gender model of the world language seems to uniquely allow us to conduct transactions between these domains whereas in non-human primates they may not be encapsulated exactly but they are much more kind of atomised the modules non-human primates have most of these modules for sure no doubt maybe all of them but they don't have a way of conducting transactions between them whereas with humans we can talk about a large number of funny green colourful heavy balloons we can combine different conceptual categories together and number sense, morality different geometrical constructions emotional constructions we can use language to access these different cognitive constructions now even more interestingly if you look at the syntax of natural language there's been some really interesting work by people at Queen Mary University and other universities showing that if you take the list of human semantic concepts and the list of human electrical items and preachers there's a lot of overlap in terms of event structure and agent processes and features but there's also some non-overlap the certain notions like worry or concern for example that are not morphologically marked in any language whereas things like trustfulness or belief and things like that are marked by way of language so there are certain conceptual features that are very readily found across the world's languages and have certain morphological inflections or features for but there are also certain epistemic notions or notions pertaining to knowledge and truth and so on that are not readily morphologically marked so that suggests that there are certain parts of the brain set in cortical modules that the language system does connect with and does functionally speak to and interact with but there are other cognitive or perceptual modules that it does not speak to but it's more isolated so that's kind of a potential way to kind of think about ways to relate the kind of neurobiological infrastructure with the kind of behavioral linguistic kind of phenomenon okay so yeah and then just to conclude the more kind of philosophical foundations of Rose are also in line with research documenting a rich array of innate capacities utilized during language acquisition in the Unix which can distinguish distinct phonemes that were boundaries, led words and so on I'm kind of really invoking a very rich kind of array of innate capacities so currently due to the lack of consensus regarding how to ground syntactic combinatorics and exhaled in the brain researchers enjoy many degrees of freedom when selecting from their preferred linguistic theory, processing theory, the algorithm and your biological framework the implantational framework so I hope that Rose basically allows us to you know in a landscape to narrow the space of likely candidate neuro-mechanisms for syntactic frequency building it's really all about figuring out what are the most likely mechanisms that we're going to find natural and rude syntax in the brain, which parts of the brain are going to be sensitive to it and which really index that process on some level and so yeah I think I guess I should probably leave it there and conclude it but yep that's pretty much the whole model thank you for the massively informative overview and the research agenda do you want to stay? happy to stay for a bit longer for sure yeah yeah I'm happy to chat alright well there's so many places to join for people who are watching live maybe we'll have a few minutes to ask a question wow yeah if you have any questions or comments or thoughts I can sit here all day and just talk forever about this stuff I probably shouldn't alright I have a handful let's do some short answers and just some general because on the evidence and on the specific brain regions and specific time scales you've made that abundantly clear so I'm going to ask some alternate questions so how do we study these densely woven cognitive phenomena without how do we hold that tension without just being simply only always to wholism or reductionism you kind of talked about that with the inside out and outside in views but how can we approach a different cognitive system maybe not the human brain and language where even the relationships amongst the different phenomena might be themselves not known okay I see you mean so if there's you mean a human specific cognitive process that is not language yeah if you want to pull out a different thing or study a different system what kind of account are we looking for how are we going to know we're on the path that's not being just massively misleading and so on no no that's a good question so I think so the Rose architecture I think is is partially applicable to lots of different cognitive domains like tension and working memory you know face perception they all involve something at the R to O level the coordination of spike timing and the realization of some kind of sensory motor transformations at the broadband gamma level the thing where we take over is really at the S level where you get the particular type of low frequency coordination that seems to be specific for language language is not the only thing that recruits low frequency oscillatory phase codes to conduct the orchestra of representations that need to be externalized or interpreted so I think it's transferable in many ways the real testable issue will be to do with whether or not these whether or not I'm correct that these different levels of linguistic structure are found only at these levels or maybe they're found across all of them so for example if effects of syntax specific responses can be found truly at the R the O the S and the E scales that I've mentioned that muddies the waters because it means that the brain at every scale across all areas cares about the most niche specific issues in syntax we might have to turn ultimately to state space architectures which I suspect will be used to sort of meant rows at least at the the S and E levels I think in terms of you'll have like different abstract states being coordinated across network nodes rather than the kind of which is similar to what I mentioned with traveling waves just a more accessible interactional process but for a very different mathematical grounding it'll have a very different mathematical grounding so but I think I don't know if I'm not sure if I'm answering a question well but I think I think in terms of transferring it to other domains that could be explored and it's really about you know trying to find trying to find a prediction of like I need to use a particular methodology I need to use a particular experimental paradigm to find effects that should only be found at this particular neural resolution and in this respect it's no different from the rest of the cognitive sciences because in working memory and attention and navigation that's exactly what these guys have done for decades linguistics as usual is you know the last person to the party and we are linguistically usually the last field in cognitive neuroscience to kind of pick up the developing tools in the field and apply them so and that's fine you know there's nothing wrong with that I think the reason is because human language is the most complex cognitive process that we have it requires a lot of the weeds to be sorted out before we have to figure out how working memory works, how attention works how vision works before we can even worry about language because all of them kind of presuppose and require that right at some level so I think it's natural for language to be saved until last right save the best to last for sure no doubt but I think it's also it makes it it's also the case that neuro linguists have not always taken the lessons that previous fields have learned as seriously and we often end up making the same mistakes so when we got scout SCALPEG for the first time or fMRI we made a lot of the same mistakes that the memory guys did or the navigation guys did and I think my own fear in writing this paper is that the reason why I wrote this is mainly because you know I see it in the last few months I see it in the last year or so cognitive neuroscientists getting their hands on very advanced recording techniques but not really knowing what to make of it again this very aggressive very but also very contemporary and very progressive kind of so called progressive viewpoint of data driven science where you just kind of let the data speak for itself and allow the statistics of your you know also do some kind of machine learning on your kind of on your own data and expect that to kind of feed into an explanation for language a lot of those techniques and also discourage this kind of theoretical reflection and if you look at people like Carl Friston you know he spends most of his time he's the world's most influential neuroscientist and he spends most of his time just doing theoretical neuroscience because it's that's really where we're at these days in all of these fields it's about consolidation and interpretation a lot of the I mean I was surprised when I started writing this paper I was surprised that a lot of the clues and answers to this puzzle of how syntax is inputting in the brain are kind of already out there they just haven't been very you know the dots haven't been connected the principled relations and logical relations between different findings and different you know recording methodologies and the kind of more principled relations between them haven't been established yet or haven't been figured out and I think there's often a push and a desire to just you know do an over experiment you know get funding to do more and more experiments collect more data and do more analysis that's as awfully essential I do that as my day job well outside of my day job I think it's also important to spend some time you know thinking about how to thinking more carefully about this so that you don't need to do any more experiments for a while so at least so you know that you know when you do your experiments you know the landscape better and you don't end up making the same mistakes that other people do but really it goes back to this issue of kind of cost cost benefit like you know how do you how do you kind of weigh up the pros of collecting more data with the you know the foreseeable benefits the whole notion of foreseeable benefits empirical foreseeable benefits really presupposes and needs this framework of what the benefits are likely to be if the benefits are just I'm going to just let the data speak for itself and get a bunch of different you know machine learning or LLM architecture to kind of figure out the story for me you're probably going to be disappointed whereas if you take a step back and think about a more principled architectural approach think about the brain the way an engineer would and figure out like which level is going to be best served for a different a particular cognitive process that's the basic mindset behind behind this model it's about figuring out like what you know which scale of neural operation is going to be most helpful for you great well on that kind of research program scale the compass rose it's a great model I was curious about a few other settings for speech like listening to a video or recording at a different speed than 1x or watching subtitles or just the idea of reading and whether there was some kind of a clean handoff between one sentence is heard then one is read then one is heard is it beyond the sensory modality by what stage or what elements that are extra lexemic like prosody and timing and so on play into for example speech as opposed to writing where there's formatting and italics and so on yeah no that's a great question so there's an unanswered question at the moment which is to do with the bottleneck of reading or speech processing not so much but definitely for reading we don't really know that much about the bottleneck like how fast you can read so the rose model talks about delta rhythms, it talks about theta rhythms these are very slow waves if you ever tried speed reading it becomes very clear straight away that human beings can read very quickly so if you you know code in matlab or python some kind of rsvp style a textbook where you can just like have one word flashing on the screen at a time and set it to different rates if you have 500 milliseconds away no problem it turns out you can go way faster up to like 15 20 words a second and you can still get interpretability you'd be surprised if you try it you'll be very surprised by how fast you can actually read so you might think that poses a problem for the rose architecture I don't think it does I can get to that after I've answered your question it does pose a problem at all but so that's a good question so we know that there are certain parts of the brain that do in the ventral temporal cortex which is the lower side of the temporal lobe they are responsible for transforming visual information into orthographic information the work of oscar wolno in my lab he's done the most impressive work on this by far very exquisite like intercranial recordings of how human beings in fact his whole research is on reading so you should probably definitely read his work basically about how the human brain takes visual information and then trans converts it into an orthographic space a constrained orthographic space a space in which your mind I guess will be sensitive to particular statistical configurations of shapes but you know this one's a letter this one's not a letter so that takes place in the same part of the ventral temporal cortex and then maybe somewhere around there that will due to endogenous processing and limitations probably not to do with vision probably due to some kind of next stage process to do with interpretability and because as you know we can read we can see based on such the card rates and the number of objects and things we can see in a given second is very large indeed the human brain mostly filters out most of its sensory information every second just to kind of focus on the stuff that's key to its general model it's kind of minimizing the free energy of its general model and keeping it kind of sustained not changing the status quo etc but on the reading level we know that at that stage that's when you get what I call the centrum of the transformations in high gamma activity so that's where you get that high gamma activity being responsible and sensitive to the mapping from that visual information to the linguistic information because often somehow you see words on the screen you see squiggles and lines somehow you've got to convert that into a more abstract space so that will take place in the ventral temporal cortex and then you get some interesting connectivity on top down information from being sent from frontal cortex back to ventral temporal to constrain and kind of coordinate that bottleneck at least that's one of the main theories that exists I'm not an expert on reading per se I'm not a vision scientist but because I understand that's the kind of the general framework is that there is a part of the brain that's sensitive to the statistics but also the symbology of orthographic information and we know that it takes place in very short time scales when it comes you asked a very interesting question about the the kind of the kind of dual modality like when you're watching a film you're listening it and you're also reading subtitles at the same time that's a very tricky question because on the central motor side we know that you'll get oratory cortex involvement and orthographic kind of occipital temporal involvement engagement at the same time but somehow they both have to converge on a single interpretation because we know that you can't comprehend things in parallel you can't read two books at once or at least most people can't so that means there's got to be a single kind of interpretation procedure and a single kind of semantic integration process that is being instructed in a form by different modalities but when you think about it that might sound difficult but it's actually no different from when I use hand gestures right now right so when I use hand gestures I'm providing some kind of extra modal information to you it's obviously easier because this is more hand gestures are used kind of pragmatically informative rather than explicitly propositional of course unless you speak sign language but it's still just a case of different sensory motor transformations taking place so that would still be at the old level that would still be at the old level it's still high gamut transformations that are then fed into a more uniform discrete and abstract syntactic inference being made there's only one syntactic inference being made about the meaning of the sentence but it's being informed by multiple sites now you do raise an interesting question it might be the case that this low frequency delta complex therefore has to entrain and functionally connect with multiple cortical sites it might have to speak to auditory cortex but also ventral temporal cortex if you're gleaning distinct types of if the lower level lexical representations you're getting are being sourced from auditory then orthographic so for example if I give you a sentence and I get you to hear the first word and then read the second word and then hear, read, hear, read or listen, listen, read, listen, read if you go between listening and reading but you're still integrating a single unified structure we know you can do that, human beings can do that so that tells you straight away that it can be done but the question is how it's done like I said I suspect it's through this dual kind of trade off between sensory motor instructions being sent from auditory cortex to the same unified low frequency program and then the visual information too so that's actually a kind of a nice way to separate out the distinction between sensory motor stuff from the more abstract unified kind of syntactic information that's being built and in fact you've just given me a good idea for an experiment so thank you yeah because language is so linearized it's like being pulled through a needle and then in the saccade so-called speed reading but it's not simply moving the linearization faster speed reading can leverage formatting and all these other strategies like almost thinking through other minds ways of reading and also having an off-center effect in visual system and the way that that kind of involves the orthography versus in listening you basically always want to be listening more clearly because the moment is the only time you're going to hear it the active listening component it's like you can't let it fly by but reading you can take saccade strategies and cognitive strategies that you just can't take and that's kind of this like all at once versus one at a time element there's actually there's a framework in Psychic Wisics called the now or never bottleneck so the now or never bottleneck refers to what you're saying right in reading there's no now or never you can just take it down with listening you're totally right it's now or never you have to pay attention now and integrate or your working memory is going to dissipate or your attention is going to or your phonological loop is going to decay and repeat themselves again again it's a very different performance system and it's also very different from Braille when you're using touch and you're reading Braille when you're gesturing and using style language language especially syntax truly is a modal it's completely independent of any modality the neuroscience of speech is very different from the neuroscience of language the neuroscience of gesture language is a more kind of a modal abstract system that calls upon these very various sensory motor processes which is what most people intuitively trying ground language in a sensory motor context because that's natural of course that's how we access it that's how we trigger it most of our daily language use is probably just thinking to ourselves using language maybe subconsciously using the computational capacity that language affords us in non-linguistic ways meaning to construct hierarchical structures of thought and planning and all the rest of it in some kind of subconscious way rather than explicitly saying I am saying a sentence or even I am thinking a sentence in my head because even when we think a sentence in our head that's still just an internalised form of externalisation so when you think about in your head that's still externalisation which is a different type of externalisation you're not externalising it with your mouth but you're still putting it out into your own phonological loop but that's still externalisation internalisation is truly the subconscious process of generating the syntactic inference which is done rapidly, reflexively intuitively often without much thought often without enough meaning attention often without much attention being directed to it and any kind of conceptual implications being made so most of language use is outside of the sensory motor space maybe I'm wrong about that but this is a point that Noam Chomsky has made a lot and I think he's right about this a lot of the daily use of the language infrastructure is like I said I mentioned the universal currency metaphor across these different conceptual spaces and it's potentially represented very predominantly across sparse neural codes as I mentioned in the Rose architecture such that it survives brain damage, it survives all sorts of different things but it's still crucial and you get a lot of people like Rosemary Barley and Federico showing work using imaging research showing that the language network is not activated during XYZ non-linguistic cognitive tasks or damage to the language network does not damage XYZ non-linguistic tasks that's all fine but that's a separate issue from whether or not the more abstract syntactic neural code that I've presented in the paper is recruited on some level by non-linguistic resources which I suspect is again, another empirical question for you but I think that's a reasonable framework to kind of assume at least one I guess reflection on that then one last question you described language human language as being a kind of psychotechnology with supporting architecture and enculturation all these different features that are totally relevant influence it that allows us to articulate like the sense making in action like when we look at figure 4.3 in the active inference textbook and we have a Bayesian graph that factorizes out through the sparsity of just how things are related like how the measurement of the thermometer is related to the temperature in the room how the temperature in the room changes fundamental concepts of place and time where preferences fit into that is about the semantics of this variable whether it's kind of like a central tendency or whether it's a variance estimator like confidence which is something that you brought up as a strongly linguistic utility so it's kind of like it has an element of just being the geometry and topology of thought and thinking and generative modeling that has ultra, mundane everyday uses because this generalized apparatus really does have utility, pragmatic value in communication so it can be used for the most referential just one bump above pointing at something and pointing at something and grunting and pointing at something and say look or look it's like you can keep going but often what's only needed is the first little bit, especially given a context and yet there's this open-ended arbitrary state space navigation element as well yet it's only I just, I think it's incredible how it's laid out and how what you described with a heavy emphasis on the neurophysiological and the computational comes into play with active inference and the generalized state spaces and all these concepts so that's just super exciting I guess my sort of closing or we can keep talking or whatever but how do you see this relating to language learning or first language and for not first language learning? Yeah, I totally agree with what you said about the active inference framework I think linguistics, semantics and well syntactic structures are a unique kind of inference that the human brain has to make I see a lot of the properties of lexical semantics in human language as being a unique type a unique contributor to the active inference framework and I think it's a surprise to me that more linguists haven't turned to active inference to think about certain properties of semantics because it seems so obvious, it seems so compatible I've tried to do it in some work and I think maybe some other people have too potentially but it's an obvious kind of well I mean I've cited the Peter Hegel paper he cites that work too a little bit but no it's a very promising way to picture it. In terms of language learning and acquisition that's a very, very good question so I mentioned that at the beginning some of the low frequency delta responses to syntax, all of that research is in adults, well all of it was until some research in the Cambridge UK a few years ago started to think okay well maybe is this how children process language too or maybe is this how is this where they process language in the same regions. It turns out if you look at teenagers and children and you do the same you know scout BG or maybe even MEG research I'm not sure I can't remember, I cited it in my book the low frequency responses change over development in terms of which ranges are sensitive to the periods of syntax so it's not necessarily you know 1.3 hertz precisely that entrains the syntax, of course not this gets back to the point I made earlier about speed reading right, it's all about the flexible coordination and packaging of representations into a given complex, again I mentioned the issue of trade off between number of representations you can chunk and the fidelity of the representations, you can have a few and what represented in gamma or a large number and only a small number of spikes per representation that's just kind of that's a very general framework that's been supported in the attention literature too, it's not just my proposal so that poses a question if children and teenagers show a different kind of low frequency phrase response how do we deal with that I don't have a clear answer to that question I think the Rose architecture can very readily be modified to that because it's not as if they'll still have the same architecture they'll still have the same R to O to X to E the particular frequency band of interest might shift over development and in fact we know from working memory and attention dynamics and alpha dynamics definitely and beta dynamics do change cross-quartically over development and these are behaviorally relevant to things like development of working memory and attentional resources that's been very clearly studied and very well studied too so it's more like there's a more abstract code that simply shifts its spatial temporal profile as the brain gets older and then changes more when we get even older so I have a paper that's going to come out fairly soon I hope to do with aging what happens when we when our brains decay and we get old there's some nascent kind of research looking at the language processing not at the behavioral level but in terms of the neuro level the neurobiology of language processing in older people and how it might also relate to breakdown of semantic information access and things like that and what are the potential correlates of that so you can imagine very clearly if you take the Rose architecture which components will be disrupted or modified based on if it's a syntactic or a conceptual or a lexical kind of deficit the isomorphism is as I said very crude so you can predict that very clearly and in fact there's some research in that space that I think is concordant of this which is what I discussed in this new paper so like I said it's a relevant architecture it's a very difficult question about how the whole brain matures over time because the certain aspects of knowledge that may not be represented in ways that we currently know how to you know detect or record and we have the four or five measures, main measures that I mentioned in the paper they're what we have right now maybe there's a particular form of neural encoding that just goes way beyond what we can physically provide in the year 2023 I don't think there's any reason to assume that we've reached the limit of the codability of neural signals and their complexity absolutely not so the learning process is a very mysterious one so yeah I think it's also a really good way to falsify a lot of this stuff by the way it's a really good way just to completely falsify and test a lot of these proposals to do with the architecture itself and then developing code over time as well so yeah it's a very relevant concern the best way to do it is to get inter-craner recordings in children that's very difficult because so there's a in Austin, Texas there's a lab there that has these recordings and uses access to children with epilepsy and they get these children to watch trailers and listen to stories and things like that and they do some interesting language research in these kids that's exceptionally rare data and very very very useful data for answering these questions I don't have access to that data and most of that research has not been forthcoming yet there's been a couple of inter-cranial papers in children and language but it's fairly sparse fairly low patient numbers and the types of analyses have been too kind of remote and departed from this framework to really test it that would be the way to test it electrodes directly inside the brains of children and potentially even younger I think the youngest I've seen is about four but even by four years old it's maybe too late to chart the actual full development of acquisition of language because by that stage they've already acquired syntax right if you could get I don't want to sound like a mad scientist but if it's possible ethical to one day get access to inter-cranial recordings in very young children then it would be possible to chart this developmental process especially if the children are in the hospital for protracted periods and in fact get some kind of I mentioned BCI implants if they have some kind of implanted device over time that would be the most direct way to test this stuff not just test my theory but just you know just just completely chart out the full developmental profile and the evolution and the development of the neurophysiological basis of speech processing and language processing so that's kind of as far as I understand that's kind of like the current landscape of where the field is right now well you can have the last word with any last thoughts but thank you for this amazing presentation and I hope that if people are interested that they can of course read the paper and learn and reach out and maybe we can have 64.2 when the time is right but anything you'd like to kind of leave it with yeah no thanks so much for having me on man it's been great you've been great I've been enjoyed it's been enjoyable just kind of outlining my own theory you read your own paper over and over again and you realise all the things that you kind of misinterpreted or misread on your face read through in terms of like phrasing and all the rest of it so it's always good to refresh these things but no absolutely if people are interested in this stuff I'm always open you can contact me via email on Twitter absolutely so yeah thank you for listening thank you for having me thank you peace peace mate