 Hello and welcome everyone. This is active inference guest stream number 55.1. It's August 30th, 2023 Today, we are going to have a presentation followed by a discussion on geometric constraints on human brain function by James Pang and Alex Fornita Thank you again for joining Really looking forward to the presentation and the discussion. So please off to YouTube to introduce yourself and continue however you like Thank you Well, thanks very much Daniel My name is Alex Fornita. I'm based it in the School of Psychological Sciences and Intern Institute for Brain and Mental Health at Monash University and Here James Pang who is a research fellow working in the team and so The idea today was that we were going to talk a little bit about this work. I'll begin by providing a bit of a historical and theoretical background and then James will Take over and jump into the specifics of what we did and what we found We're assuming some basic understanding of neuroscience and some elements of human brain imaging And we'll go through a little bit of maths, but hopefully it'll be enough to Give a picture and if there's any questions about it afterwards, we'll be happy to try and answer them So a key motivating factor for this work is really trying to understand how The structure or anatomy of the brain constrains its function and if you look at a diverse range of systems found throughout nature We know that the structure of the system can strain its dynamics. So for example the shape of a Racing car will influence its aerodynamics and how fast it can go the folding patterns of a protein will Influence the other proteins that it can interact with and quite have quite large effects. So here, for example, we've got an example of the Hemoglobin protein in a healthy person and someone with sickle cell anemia and this change in the geometry of the protein leads to a 20-year difference in life expectancy and here we've got the Molecular structures of diamond and graphite. So both of these Materials are made of the same atoms. They're all purely carbon atoms It's just the bonds between them that are different and that difference is sufficient to create quite a Variation in the properties of materials. So diamond being one of the hardest substances on earth And valued at somewhere between 12 to 90 million dollars per kilo Whereas graphite is one of the most brittle and only cost you about 20 dollars per kilo And so this is essentially the reason why we don't propose to people with H2B pencil We use diamond rings instead Now the brain is no different. We think an enemy constraints function But there has been a long-standing debate about the specific Properties that are the most relevant and how we should best conceptualize them and this can be traced back to The days of Ramonik Kahal and Camilla Golgi Two people considered to be some of the founding fathers of modern neuroscience Golgi developed a technique for staining neurons that allowed you to visualize their structure under the microscope And Kahal really perfected that technique and studied the nervous system in extensive detail and Really laid down the foundations for what we think about the anatomy of the brain to this day Now both of these men were recognized for their efforts in 1906 with a Nobel Prize But even at the award ceremony when giving their lectures, they were in bitter disagreement So if you actually look at the recordings of their lectures Kahal says we applied Golgi's method in our observations revealed in my opinion the terminal arrangement of the nerve fibers nerve elements possess reciprocal relationships in Consecuity but not in continuity so here is saying that cells are fundamentally discreet elements separated in physical space and that there's a Gap between them at the synapse the synaptic cleft and that's and signaling occurs through this gap Golgi on the other hand didn't agree He said while I admire the brilliancy of the doctrine which is a worthy product of the high intellect of my illustrious Spanish colleague I cannot agree with him on some points of an anatomical nature which are for the theory of fundamental importance I found myself unable to follow the current of opinion because I was confronted by one concrete anatomical fact This was the existence of the formation which I have called the diffuse nerve network And so for Golgi all the cells were continuous. They form one continuous network There was no point of separation between them and signaling occurred through this continuous network Now, ultimately The growing body of evidence and you can see it in Golgi's Statements was starting to favor Kahal's model But really it was only resolved in the 1940s with the advent of electron microscopy where people could actually resolve synapses and saw that they were spatially disjoint And this ultimately established the neuron doctrine as the fundamental theory about how the nervous system is organized at a cellular level And this basic idea that we can divide the brain up into discrete units Was then carried through to different scales most famously by corbinion broadman Who developed a parcelation of the cortex based on site of architectonic grounds? His idea was that different parts of the cortex have different cellular properties And so they define these discrete areas that are functionally specialized in in some way And certainly this has been a very influential idea in neuroscience ever since But around the same time in fact a couple of years early up the neurologist Alfred Campbell published his own site of architectonic map and He agreed that you could identify discrete boundaries largely in primary zones But then in the cortex in between it was a lot more difficult to find sharp Outlines or boundaries and he suggested that actually the changes were more gradual or continuous in nature Broadman's view became more popular and certainly became sort of the dominant theory in neuroscience even though Subsequent attempts to try to replicate it were different and so different anatomists would parcelate the brain in different ways using different boundaries and so on but nonetheless this kind of Congruence between the discrete model of kahal discrete model of broadman Eventually led to the idea that you know, essentially we can think about the brain as a multi-scale network Or at the cellular level we can identify Discrete cells that are connected by synapses Then we can scale up at the population level that a mesoscale resolution and identify how population of cells are linked by bundles of axons And then all the way through to the macro scales, which is what you sort of see when you look at one of broadman's maps You have these broad Roughly sort of architectonic areas connected by white matter bundles and in the last two or so decades we've had the measurement techniques for characterizing these networks on a vast scale and there's been a pretty well established pipeline for mapping these networks So the basic idea is that we have the brain we divide it up into these discrete areas and they act as our network nodes And then for every pair of nodes we measure some aspect of structural or functional connectivity between them Functional connectivity being some aspect of coupling and activity And then we can represent that in the form of a connectivity matrix So each row and column represents a different region and each element tells you about the type and strength of connectivity between areas That can equivalently be represented as a graph and then with that graph representation We can apply all sorts of different analytic tools to understand different aspects about how the network is is connected and organized And this is this basic approach has been used quite extensively to understand how the underlying anatomical properties of the network constrain function This is one of my favorite examples. This was a study of two different worms c elegans and p pacificus Both of their nervous systems in the head have been mapped at the level of each cell and synapse and they're actually identical So they've got the same type of cells the same number of cells It's just the connectivity between them. That's different. And that's what you see in this network projection here The topology of the network is different and this study they showed that changes in the connectivity of just two hub neurons were sufficient to Elicit a difference in the behavior of the animals such that p pacificus is a predator of c elegans And here you can see pacificus eating poor c elegans for lunch Now we can't lock humans up in a room and see who eats who and try to understand how their anatomy Constrains function in that way, but we can do things like Use mr Right to measure anatomical connectivity and functional connectivity and correlate the two and that's pretty consistent finding that we generally find There is a correlation between them We can run computational models where we have a structural network We simulate the activity then we apply say a lesion to that network and look at the effects on the activity And we can show for example that if you lesion a hub area You have quite a dramatic impact on brain function compared to if you lesion a more weakly connected non hub area Or we could use clinical studies. We could look at cases where there are naturally arising lesions through stroke or some other form of brain damage And we can understand the functional impairments that result. So for example in this case People with lesions to certain types of hub areas with diffuse connectivity showed much more generalized cognitive impairments than people with lesions to areas with more specific patterns of connectivity So this is great. This has been really informative and great for understanding the role of individual brain regions or connections in brain function, but it doesn't really give us a Coherent unified perspective about how dynamics emerge from the underlying structure. It's all a little bit piecemeal If you want to Develop such a coherent understanding we can draw some inspiration from different The physics and engineering where typically the structural Constraints on the dynamics of a system are studied by understanding the eigen modes of the system Now the eigen modes of a system Found by solving an eigen value problem such as this one So an eigen vector we can think of as something that we measure about a system That doesn't change its direction if it's subjected to some kind of transformation So if that's the case, then this equality holds. So the idea here is that Psi is our vector something we measure about the system subjected to some transformation l And if if psi is an eigen vector, then The transformed eigen vector is equal to the eigen vector just scaled by some back value. So We can the transformation can scale the vector, but it doesn't change its direction And so if that's the case then this scaling factor lambda is what we call an eigen value and psi is an eigen vector This kind of equation is applied to A lot of different systems a lot of different contexts But when we're studying the eigen modes of a system typically Our l term here is is an operator called the Laplacian operator Which essentially describes how smoothly some kind of structural property of the system varies Through an appropriate spatial embedding So I'll come back a little bit To what that means because we can make different choices for what we use in l and that Determines the specific anatomical properties that we prioritize but for now I just want to emphasize that a key property of these eigen vectors is that they're orthogonal and so that we then we can use them as an A basis set for reconstructing different activity patterns that we might see in the network So I guess the most famous example of a basis set that many people will be familiar with this is a floria set So for example here we have some one-dimensional signal and we can decompose that signal as a weighted sum of Sine and cosine functions of varying frequencies So here you've got the the different functions different frequencies and here you've got some weighting factors And we can reconstruct this signal as a weighted sum of these functions In a similar way we can extract our eigen modes of the brain Based on some particular anatomical property And we end up with these spatial patterns here that are organized Again in terms of spatial frequency or the wavelength of fluctuation So here you can see you've got a very low frequency fluctuation moving from back to front top to bottom Media lateral and then as you move up you get these more high frequency fluctuations And so then we can use these patterns to reconstruct any spatial map that we Observe on the brain. So it could be a map of brain activity recorded during a particular task It could be say a map of cortical thickness variations or gray matter volume variations It's sufficiently general to allow you to reconstruct any type of map And so we can do that just as a weighted sum So here y represents our map and then we represent that as a weighted sum of our modes with the a coefficients representing the weights And people oh, sorry I should emphasize that that explanation can be a little bit abstract for people who aren't familiar with eigen modes And it is always a bit of a challenge To get an intuition and so you can do what one of the people in the team did and and ask chat gbt to explain Eigen modes to a 10 year old and it turns out that it does a pretty good job I was quite happy with this explanation. So I thought I'd share So our chat gbt says of course, let's imagine we have a wobbly jiggly ball of jelly when you poke it at wiggles and shakes, right Now imagine if you poke the jelly in different spots or with different amounts of force Each time you poke it differently the jelly wiggles and shakes in a different way Eigen modes are like those different ways the jelly wiggles and shakes when you poke it So essentially seeing this video here the jelly is poked and it wobbles in a particular way If you poke the jelly in a different spot or with a different amount of force, you'll trigger a different pattern of vibration and so commonly eigen modes are used in the context of systems that vibrate And the idea is these eigen modes correspond to preferred patterns of vibration when the system is perturbed And those preferred patterns are just dictated by the structural properties of that system Or we can we can extend a more general in Think about it more as how a system responds to any type of excitation and how an excitation spreads through some Connected system and the eigen modes correspond to the preferred patterns of excitation again determined by the physical properties of the system so Naturally given everything we've been talking about And these ideas that the brain is a multi-scale network And we need to make a choice about How we define our Laplace in what specific anatomical properties are we going to extract our eigen modes from And so a natural choice would be to extract them from brain connectivity All right, and we can do this quite in a reasonably straightforward way We get some map of the connections in the brain We map connectivity between every pair of nodes after we apply some discretization We define how to scrape nodes we map every pair of connections We represent that in the form of a matrix and then we extract this quantity here. It's called a graph Laplacian It's the details are not so important But that's essentially our L operator here and it describes how smoothly connectivity varies as we move from node to node in the network And when we do that we can Several studies have shown that you can use the eigen modes extracted from this graph Laplacian to reconstruct kind of classic functional networks that you record with functional MRI We can reconstruct different patterns of functional connectivity between pairs of areas You can use it to reconstruct different e g signals and people have even shown that you can make predictions about the pattern of neurodegeneration that you see in different forms of dementia So this has been really fantastic important work and really important proof of principle That this eigen mode framework can be used to understand how the structure of the system constrains its dynamics And in each case the interpretation is that these dynamical patterns that we see with f m r i or e g or so on Are reflecting excitations of these underlying eigen modes or the resonant modes of the anatomy of the brain This view very much aligns with what we would think of as a classical view of a brain organization that I've just been discussing So following from Kahal and broadman's work the idea is that at lots of different scales We can divide the brain up into discrete areas They're connected in some way and it's the signaling along these connections between these discrete areas that drives activity But there are reasons to think that particularly at a course of scale So we know at the fundamental cellular Scale the network is discrete, but at course of scales perhaps a different Set of approximations might be useful So if you look at dense reconstructions of neuronal tissues, so this is say with electron microscopy you find that Cortical tissue is extremely dense. It's like a really rich thick forest of cells and their processes And as we've talked about The there's been a lot of debate about where you can actually draw boundaries between discrete areas And how valid that and reliable that is outside some kind of classic examples saying in primary cortex And if you look More deeply at the cellular organization of specific regions here on the left We've got the cellular organization and on the right. We've got a myelin stain You see you get these deep penetrating fibers That are coming in from the white matter This is what this kind of classic graph approach will pick up because we divide we define our nodes in the cortex And then we're mapping the connections that pass through the white matter It can also say that there's a lot of connectivity That's moving horizontally or tangential to the geometry of the cortex And so these horizontal connections allow for the continuous spread of activity So rather than activity jumping between discrete nodes, it can actually spread as a as a propagating way And you know just to emphasize the some of the limitations of the discrete model Not long after broadman's work just a couple of decades the renowned physiologist laurenta deno was pretty emphatic about His concerns with the approach He said the only really good cytoharchitectonic pictures are those of cambal who let me put in capital letters As the being the only cytoharchitectonist who has described facts and only facts The german cytoharchitectonist has mixed facts and theory in such a manner that nobody can tell where facts ends and theories begin So Again, this is just emphasizing the idea that while we might be able to define discrete areas at a mesoscopic or macroscopic resolutions in some cases Especially around the borders of primary sensory or motor areas For large swathes of cortex those distinctions become a little bit blurry So an alternative approach has emerged out of neural field theory So neural field theory has been around for several decades now And this is a different approach to thinking about cortical activity So the basic idea is that instead of dividing the brain into discrete specialized areas or nodes We can treat the cortex as a continuous sheet and we're actually Uninterested in trying to capture our patterns of brain activity propagate through that continuous sheet And so this is something that you can see in diverse types of electrophysiological recording. So this for example, this video is of The dynamics of voltage sensitive dyes recorded in in the mouse brain and you can see that you observe these Propagating waves of excitation that move continuously through the cortex rather than jumping from region to region as would be implied by a kind of Strongly discrete model So a big emphasis in neural field theory is on describing these wave-like propagation patterns And it's been shown in recent theoretical work that if we accept the wave-like propagation drives neuronal activity Then really the eigen modes that we should be Extracting or prioritizing Are those that are derived from the geometry of the brain? So our eigen value becomes this problem here where this Operator here is called the Laplace-Baltrami operator And it describes how smoothly the geometry or the shape or the curvature of the brain varies through space And just to briefly explain why that's the case here we've got the Wave equation that's used in some forms of neural field theory list of the versions that we've been focusing on And so the basic idea is that the firing rate of a given population Of neurons b at location r and t is given by The afferent field of incoming populations are other populations a so we've got other populations providing this afferent field of activity That determines the firing rate of b And obviously we then need to describe What this afferent field looks like and that's given by this part of the equation here So on the left here, this set of terms describes how the field evolves through time. We're not going to focus on that today Over here, which is what we're interested in where we've got how the field Is evolving through space And so here we can see our friend the Laplace-Baltrami operator Which describes the geometry and here we've got the length scale of axonal connectivity So in this model, we make a very simple assumption about connectivity We assume each point is connected to its neighbors in a distance dependent and isotropic way And that distance dependence is roughly exponential So the idea is that areas nearby to a given location r will have a strong influence And that influence declines roughly exponentially as a function of distance And uh that the speed of that decay is given by this r term here But you can see that geometry is firmly embedded in this equation Now I should point out this isn't something that's just been plucked out of thin air This is a damped wave equation that has been used for centuries to describe Wave dynamics in all sorts of different media And so from that physics, we know very well that the geometry of the system constrains The spatial properties of the wave propagation And you can start from first principles of neuronal biophysics and derive this equation And given all that work, we know that the eigen modes Of the Laplace-Baltrami operator actually correspond to the standing waves of the dynamics So what are standing waves? So in this simple like example, we've got two traveling waves here a blue and a green And as they interact with each other, you get these patterns of constructive and destructive interference Represented by this red line here that cause these very large amplitude fluctuations that are fixed in space All right, so they're not actually traveling like the original carrier waves. They're fixed in space So that's why they're called standing waves And then a nice example of resonance So these uh fluctuations in the red In the red trace are much larger in amplitude than the original carrier waves So it's an example of resonance where we're getting a larger response in the system than the energy that we put in And just to convince you that this is something real Here, for example, we have a pool of water a classic wave system If you apply vibrations at a specific point in in the pool You see these waves that travel out They hit the opposite wall reflect back and start to interact And then you see these the emergence of these standing wave patterns that are fixed in space but show these very large-scale amplitude fluctuations Uh another classic example is the cladney plate So if you arrange dust particles on a uh plate and then you apply some vibration at a particular frequency The dust particles will settle into these complex patterns Uh, the pattern is determined by the specific frequency that you apply and specifically where you apply the vibration And the pattern the patterns correspond to the eigen modes of the plate And again those patterns are determined by the physical properties of the plate things like its stiffness its geometry and so on This is a little bit gratuitous, but you can extend the idea to three dimensions And you can actually cause the particles to levitate again in this case They're trapped within the nodal lines of sound waves that are being projected in three dimensions Um And this does have real world consequences So this is a video of the tachoma narrows bridge in the 1940s The engineers didn't pay enough attention to eigen modes And so on this particular day crosswinds hit the bridge in just the right spots and at just the right frequencies To excite the eigen modes of the bridge So you triggered these resonant responses and created these large-scale amplitude fluctuations That ultimately led to the collapse of the bridge So uh Hopefully that background has kind of illustrated that There's sort of two different views of thinking about Which anatomical Properties of the brain might be more more important In constraining dynamics So on the one hand, we have the classical kind of connectome centric view In which we view the cortex as comprising discrete areas And dynamics are dominated by node to node transmission Uh, the connectivity between areas is topologically complex and not homogeneous and we very much think that that's very important And so we prioritize measuring that We don't really directly account for the spatial embedding of the network or its geometry We can look at it secondarily, but it's not baked into the model Uh, and because we're interested in productivity, that means we need to measure it So from a practical perspective, at least if you're dealing with neuroimaging data You need a t1 weighted image to measure brain anatomy, but also diffusion MRI image to be able to measure brain connectivity The other perspective, uh, which is a geometrically informed perspective that arises out of neural field theory Views the cortex as a continuous sheet. We're not making distinctions between different areas of the cortex We think dynamics are dominated by propagating waves of excitation, especially these mesoscopic and macroscopic scales We approximate connectivity between different points of the cortex as being being homogeneous and distance dependent So we know that there is a strong distance dependent in connectivity. It doesn't explain everything So some things are missed, but we know it's certainly an important feature So then the question is how far we can get with this approximation And a nice feature of this approach is that we are directly accounting for spatial and geometric effects the physical properties of the brain Which are overlooked by the the connectome approach And a nice practical benefit of this approach is that Because we approximate connectivity in a simple way. We don't actually have to measure it We can derive our basis set simply by looking at the geometry of the brain And so we only need a t1 weighted scan, which is very standard in MRI experiments So hopefully that's set the scene for um thinking about different anatomical constraints on the brain and which ones we might want to prioritize I'll now hand over to James who'll talk about how we went about testing these different ideas and um try to understand which Uh constraints might be more important for dynamics Thank you, Alex Uh I'm gonna focus more on the things that we found in the Paper that we recently published. I think a lot of you have the link for it But there's no consensus message if you want To we want us to forward the link. So basically we start with We're trying to answer and test the theory on which modes of the brain. Uh, I could best Uh describe activity measured in fmi. So we'll begin by how do we actually measure these modes? so, um geometric modes are based on the t1 weighted image, which is a standard imaging tool that people use in the field And you can extract a model of the cortical surface here on the right And just there are robust pipelines to do so and you can do that on a population average sense. Basically you get a group Template of the surface or you can do that on an individual basis uh On the other hand connector modes need Diffusion MRI and a structure of MRI, but you need to kind of align the two and then we run a Pipeline basically finding connections between regions in the brain uh And then from there you can construct a vertex level or even region level connect connector matrix basically It's a matrix where each element represents the connection of one region or one vertex to another region or another vertex and You feed these cortical surface or these connector matrix in the eigenvalue problem equations that alex has mentioned in the previous slides And then how do we test the performance of these geometric and connector modes? Basically, we focus a lot on fmi data so we There are several open access datasets in That are available. Uh, for example, the popular human connector project Which is uh, which roughly has about 1200 Participants, but we focus on 255 participants who are completely unrelated. There are no twins and no siblings and within the human connector project data set we you can For each subject, uh, they have what we call task evoke data. Basically these are uh functional activity whilst That was recorded whilst each participant was performing a particular task and hcp has a suite of seven different tasks and within each task several several different, uh Contrast within it And then we also analyze resting state data basically resting state data is where The the participant is just inside the scanner and just not doing anything and you just scan Brain activity through time and you can construct a functional connectivity as well where each element of this matrix Represents how correlated the activity is between two regions We also Analyze data from what we call nervel. It says it said it's an open access repository of thousands and thousands of different maps that people have contributed and each map was acquired with distinct experimental protocols Uh, and we talk about 10,000 of those from nervel uh, and again it's a the the level of heterogeneity in the different tasks Is quite remarkable in the neurovolts. So we wanted to we wanted to Do our analyses on on quite different tasks to make sure we we are capturing something really meaningful So just to remind yourselves of what we're going to do next so basically Alex already mentioned this You extract for example the spatial map say that's the task evoke map And we decompose that into different sets of eigen modes So these patterns could be the geometric modes or they could be the connective modes And it will be weighted by a coefficient a and the weights represent how Each mode is expressed For that particular map And you can truncate the number of modes that you can that you want to use And then you can reconstruct back the The the data by just taking a linear sum of The coefficient multiplied by the geometric or the connective modes So what did we find by doing these analyses? So first i'm going to show you this How the spatial maps look like in terms of reconstruction So on the left i'm showing you the different task evoke spatial maps And these are group average maps For the human connector project data set and again, these are the seven tasks for the data set And then we ask okay. How if we use about 10 modes? How would this spatial map look like and for this result we're focusing on the geometric modes? So the reconstruction looks like this From afar you They look quite They look quite different, but if you actually look a bit closely It already captures the kind of the large scale Core scale patterns that you can see from the data. For example, this relational map You can clearly see these the emergence of these big pattern Now what happens if we incorporate more modes into it? So we increased the number of modes to 100 and now you can see that the reconstruction almost Looks like the empirical data, but there are still subtle differences Especially in the more localized Activity for example here in the arrows And if you increase it further you can see that we can better capture those isolated localized patterns Just by increasing it to 200 modes So you can see that the modes are actually Really an inefficient basis set for tasks give up data We do the same for resting state data. We increase the number of modes in here And compare the functional connectivity of based from the empirical data and the reconstruction And we see the same thing that as you increase the number of modes to 200 We can kind of we can capture the empirical data And you can do this in a more systematic way By calculating how well can we reconstruct the empirical data just by You know taking the correlation of the empirical and the reconstruction In reconstructed maps as a function of an increasing number of modes And we see that indeed the reconstruction accuracy increases As we increase the number of modes and up to 200 we can get about you know 85 to 90% accuracy which is quite efficient given that We do the analysis on a on a cortical surface We about 30,000 vertices So mathematically the maximum number of modes available to us is about 30,000 and Using 200 hours but out of the 30,000 we can already reconstruct the data quite Efficiently and then we now So we've shown that the modes Quite powerful and efficient in representing the data. We now test the Hypothesis or the the the questions that Alex's laid a while ago that Are the geometric modes more parsimal use than the other anatomical modes? And when I say anatomical modes again, we're focusing on the connector modes, which is based on The connectivity matrix, but we also test another anatomical mode based on An ideal rule. So an ideal rule is the exponential distance rule which as Alex has mentioned a while ago, which is that Because the assumption of near full theory is that The connections are very localized and also Organized in a distance-dependent way Because specifically The connection white changes exponentially as a function of distance. So we want to test that Hypothesis on how dominant that exponential distance effect is so for the the Details we just created a synthetic network based on a Simple probability of connection following an exponential distance rule With a skating factor or an exponent alpha which we matched From empirical diffusion MRI data and again There's a lot of Evidence of the existence of this exponential distance rule. So it's not just a A hypothetical a theoretical Kind of assumption we see the effect in macaques if you follow the weight As a function of distance from track tracing data and this is in logarithmic scale you see these linear curve, which represents an exponential in Linear, you know coordinates and we see the same with human Diffusion MRI data that the weight changes as a function of distance in an exponential distance weight And even in in other species like mouse rats Mamosets and so on so forth. So this is a universal property found in a lot of different species. So this exponential distance rule connectivity would be It is actually Empirically valid so By using these three different Anatomical properties, we can calculate the modes using the equations that Alexis mentioned a while ago You can get these geometric modes on the left connect the modes in middle and the modes from the exponential distance rule connectivity on the right and Again, the modes are arranged in quite a unique way In that the first couple of modes will represent the lowest spatial frequency Or patterns with very long spatial wavelengths And if you go further down you can get these high spatial frequency or short spatial wavelength patterns, so it's a very So the basis set that the modes provide Can capture different scales of the data so How do these and anatomical modes compare when we now reconstruct the different fmi data that we have So for resting state functional connectivity We compare the three modes again by measuring the accuracy as a function of the number of modes And you can see that the geometric modes are performed both of the Connectome and the edr modes, but you can see that the edr modes are quite Close to the geometric modes, but so again suggesting that the the Fundamental effect that the edr connectivity That's in in empirical data, so they I can say it highlights the importance of the simple but Important exponential distance rule connectivity, and we can see the same thing for By reconstructing the different task activation maps that we have again I'm showing you here the difference. So it's the geometric minus the connector and geometric minus the the reconstruction accuracies and red means the geometric modes are outperforming the the other anatomical modes, so In all the datasets that we have the results are quite consistent So again, you'd suggest that the geometric modes Are more passive menus Than the other anatomical modes that we have And then we go a bit further on how do we actually use the concept of these modes in analyzing FMI Activation maps So I just kind of just want to explain a little bit of what the traditional approach we have in brain map mapping research So for example, you do an experiment you capture the activation map shown here for a particular task and I'm showing you here the kind of the 1d version of that So the value the activation Versus the the vertex in this cortical surface So what traditionally what we do is to Because we because we want to highlight we want people ask, okay If a person does a particular task Which regions of the brain are kind of driving those? the the The the behavioral performance that we have was performing a particular task. So what people usually do is to threshold these maps And you can set threshold Barely arbitrary And then you can get these clusters of activity and we We then make a conclusion that these clusters of activity The ones kind of driving the behavioral Uh Effects that we see whilst performing a task and people can even go further and you know Hone down exactly really really highly localized regions if you increase the threshold But if you go further back, uh, we started from this whole brain activation map You can clearly see that by thresholding you're actually removing a lot of important, you know activity patterns That are sub threshold, but they but but they do exist. So it's kind of like The analogy is, you know, it's By doing this these thresholding approach you just looking just at the tip of the iceberg You're ignoring a lot of the important properties of the iceberg that are not seen by the threshold that you set so What we can do is to use the mode approach to actually look at the and Kind of look at the properties of all the Of activation maps and here on the left. I'm showing you here the power Of each mode. So basically the whites that we Obtain from the reconstructions that we had in a while ago and these the Figure on the left shows you the results that we have On average for the human connector project that I said and on the right is for the 10,000 maps from mureville So what you can see here actually is most of the power is already is Hanged within about zero, you know about 50 modes and if you remember the modes represent And each mode represents a different wavelength and they are arranged And that the first couple of modes represent these long spatial wavelength Spatial structure. So what this means is that actually in in fmi activation maps most of the power Is is within those long wavelengths kind of suggesting that Indeed by even by performing a particular task We we are actually activating this almost whole brain Yeah, yeah, so you can see this almost whole brain activity That again, you will miss out if you do some thresholding effect and to further emphasize the importance of these long wavelength patterns We did a simple experiment where we tried to remove Stay the first couple Long wavelength modes and that's the solid curve that you see in here And by doing so we start removing those long wavelength modes We the reconstruction actually completely drops Even by just removing 25 of them. But if we just remove the You know the short wavelength modes. So starting from the right The effect on the reconstruction accuracy is not that Significant so again alluding to the fact that these long wavelength modes are really important And they describe The important structure found in the activation maps from fmi data So again, so we've shown how the modes can explain and and represent spatial activation patterns what happens with the dynamics and The way we can do this is we go back to the crux of the neuro field theory Which were to be the geometric argument modes were based from And your field theory is very generalized You can you know construct different populations of excite tree and imagery neurons and you can even connect the files and so on and so forth But the important bits to remember is that your field theory relies on the idea that activation propagates through space as waves and simplify the the Experiment that we want to do because we want to model the activity as a function of time We just focus on the cortex And we simulate the equation based from the wave model and we tune the parameter But if you look at the equation, there's there are only two parameters and we fixed one of them. So basically the model is Only relying on tuning one free parameter and What we did was to compare that with A more sophisticated mirror mass model. So there are mass models rely on the connect time. So basically you have regions In the brain that they are connected and the connections are based from diffusion MRI And each region will have its own dynamical properties. So they evolve through full-time So it relies a lot of this on this connection and and again local dynamics But it's important to highlight that these highly sophisticated model has about 19 parameters and we Fixed about 15 of those but we still need to kind of tune about four So in comparison to the wave model again Wave model only has one And what we did was to how well can it recap recapitulate functional connectivity data So here i'm showing you an example or The empirical functional connectivity that we have and the resulting model based functional connectivity for both the wave and the mirror mass models and we can kind of Quantify how well the the model based functional connectivity results Stack up with respect to the empirical data enough in a variety of ways One can look at how well That's the edge fc. So basically each element of these matrix Compares between the empirical and the model based data. You can see that both models are you know performing quite similarly We can also look at the average connectivity Within each region. So basically you're taking the average of a row or a column And you can see that the wave model now upperforms the mirror mass model by a bit And you can again further because these properties the first two properties are based on static fc Basically, you're kind of averaging the effects across time But we also measured how well two models compare in capturing Dynamical properties through time and you see that the wave model actually upperforms the mirror mass model And so in summary for this is just to show that the simplicity of the wave model Can capture a lot of these properties of functional data Again highlighting the the idea that these wave Evolving although traveling in in space and time in the brain is probably a good approximation We also simulated the effect If we stimulate say the visual cortex So we just ping the visual cortex and allow it to the activity to propagate Through space and time and this will be the propagation starts from the visual cortex and then it goes to the frontal Cortex and I think I want to highlight one observation that we found is that We can see that if when we started from the visual cortex the activity starts to separate into two And in the psychology and neuroscience literature People have have actually found this effect and they call it the separation into the dorsal and ventral stream and each stream has important cognitive implications but people have Try to explain this separation by capturing the complex layer specific activities In the visual cortex, but what this simple experiment that we have done has shown that maybe The geometry and the idea of waves propagating could be sufficient in Obtaining these kind of physiological effect And then we also tried to compare it with Our world as it recapitulate the visual hierarchy that people have found in primates for for example this work by in the last Couple of years where they Also did the stimulation Where they stimulated the visual cortex and what they found was if you follow regions along the visual hierarchy in the maca cortex, you see the Activity patterns are kind of delayed in a hierarchical way. So they follow the same kind of uh Relationship as the visual hierarchy So we did the same thing. We compared the activity the amplitude of the activity By matching the regions that people found in the hierarchy of the maca cortex and we we tried to find the regions in the human brain And then we see the same effect and then we when we compared it to a typical measure of our hierarchy in the neuroimaging space By using the inter cortical malin map, which is just the ratio of t1 and t2 We see the effected the time to peak of these activation patterns Well captures the Rack of the inter cortical malin again, highlighting that indeed what we're getting is capturing this this hierarchical kind of processing uh Starting from the visual cortex And think the last result that I want to highlight is the uh The sub cortex uh because Again, the brain is just not it's not just the cortex So what we did was to also calculate the modes from sub cortical regions and what we did was to mask the sub cortical regions and perform the same uh and and um Some of the same equation using the whole plaspid primal operator And you can get these argument modes for the thalamus triadum in the hippocampus And we compared that to uh, what we call in the field as functional ingredients. Basically, they are representations Uh low dimensional representation of resting state fmi data uh so What we did was to Compare the geometric modes for the different uh sub cortical structures and the different functional gradients based on fmi data and you can see that the modes uh are Almost perfect in perfect correspondence with the functional gradients And we see the same effect for the different structures again probably highlighting the idea that The geometric effects that we solve in the cortex are much more dominant uh in the sub cortex That geometry really constraints function um Just to uh tie things together. Um What are the advantages of this approach that we found? First it's a it's a more physical understanding of the relationship between structure and function based on ideas in physics engineering That has been used in the last century The direction modes are only based on t1 rated images. Uh, so it's very simple Perform and the analysis can be done by anyone with a structural t1 rated image And hence you can apply it in quite different experimental contexts in both humans other species health and disease And finally to summarize things I hopefully we've shown that geometric modes are indeed more parsimonious than other anatomical modes And this is confirming a lot of the predictions from theoretical uh calculations of near field theory and highlights the importance of treating the brain as this uh really physical system uh in correspondence with what we people are doing in physics and engineering Uh, but it also highlights the idea that the effects that we see in FMI might be dominated by the effect of the simple local and exponential distance for connectivity however, uh the the The contributions of long range connections are still there, but the what we're highlighting is maybe the uh Their influence may not be observable with the resolution Uh that we currently have with FMI data So it would be interesting to see and perform these experiments in a much more highly resolved say track tracing diars from other species Uh, that we found that evoked activity Uh is dominated by these long wavelength modes, which is you know challenges a lot of the traditional approach We have in the neuroimaging field where we focus a lot on these focal activations and that traveling waves Shaped by geometry are sufficient to explain a lot of a lot of physiological phenomena by you know, that they just emerge from the model and we didn't beg all these physiological phenomena into the model Uh, and finally that geometric constraints seem to be universally Exist in both cortex and sub cortex and that we can only we can use one single approach to study the you know, the whole brain quite nicely and I just wanted Both Alex and I would like to thank all the other co-authors of the work. He has been a truly multi-institute collaboration all the open access data set that we have used and if people are interested in Doing our analyses we have written an open access uh software package uh repository in github you can access that in here so people can you know Analyze modes in the brain in quite different contexts that they want to do Uh, and I think that's where we want to end. Thank you Awesome. Well, thank you for the great presentation And a lot of places to go first all pass to dean um, thank you gentlemen. I I uh I first got wind of this paper through uh another individual in the active inference community And I think I was the one that may have out of the table that We might want to do a uh a guest stream on this and It's ticked the paper and really brought it to life. So Thank you very much. I really appreciate it. Especially some of the uh dynamic images I think it really helps in terms of explaining what you guys were trying to um Trying to do your work with and your and your subject matter on so I'll just start with that I have some questions later on but just first of all, thank you for For a fantastic, um explanation Thank you When you present this in a neuroscience setting, what's a common response or reaction from the introduction I could really tell how careful you were to speak to the totality and the Different threads in neuroscience. So after all that setup and all the statistical analysis That's presented Where does that put people in terms of their own understanding of the way that they've been doing Complementary methodologies or orienting their research questions that way Yeah, I think um And generally people are I mean at least we found that people are quite interested. Um, I guess the approach Is a bit Different to how a neuroscientist would typically think about the brain But kind of aligns with how say, you know, a physicist might not only easily start approaching Thinking about and modeling the brain and so the work was really trying to bring those two kind of perspectives together um You know, I think yeah, the reaction is is usually kind of yeah, this is this is really interesting. Um, you know, then How do I how do I make sense of it or what I do practically? Um, You know, there has been I guess there is Some people might kind of split Split it artificially into this comparison of odd geometry versus connectivity. Um, Which is probably not the right way to think about it. Um It's you know, the way we like to think about it is that You know, we're sort of comparing two different models and each of those models makes certain approximations and prioritizes different features um The you know, the approach we put forward in this paper Emphasizes the role of geometry and it does have connectivity embedded in that model It's just a kind of simplified approximation of connectivity. This kind of homogeneous distance dependent channel Um, whereas, you know, typical connective centric view We'll try to capture all the complex connectivity, but we'll ignore Also the horizontal connectivity That really is what drives activity spreading through the brain. That's completely ignored. And so really, you know, the idea behind this work was to kind of Test in a way between these two approaches and you know City which ones approximations were kind of more useful in a sense Dean want to add something or I can make a comment there No finish finish this because then I'm going to ask my questions about the cello Yeah, well the statistics Was very clear and and it was very interesting how with two different data sets There were similar curves with with also a very different number of samples So it was almost like the the some of the statistical distributions were latched on to relatively early and then that like captured the big continent and um A lot of the connections are local. This is and this is of course a topic to explore about the molecular and the Mechanistic electromagnetic basis And whether this to what extent it represents traveling propagating electrochemical waves and or loading heavily on simply Diffused local connections and and even the the blurry line between those two So it is really like a question about what is being captured because even the edr with the spread Captures a variance component related to local abundance direct fast-acting connections In in some regions and distance connections. Maybe over the days distal connections Really do matter as as the the Jell-O sets and things change But at the exact time scale that some of these patterns are happening at Your work is showing new Information and comparison with different ways it's been looked at Yeah, um, I mean I think one of the Things that it highlighted or I even told you just yeah in my head is we've been focusing a lot on Trying to capture the effects of this complex connectivity, right and we ignore All these lateral horizontal local, you know diffusing connections But what the work has now kind of shown it highlighted the kind of maybe the issues that we have with fmi, right, you know Because truly the both of these roles happen Probably something simultaneously, right But just the effects that we're seeing in fmi Is probably not the most efficient way to capture both worlds Or it cannot hung down the specifics of one world over the other In the results that we're getting is that the effects of these, you know, very localized connectivity Probably dominates A lot of what we see there fmi but Time will tell if you know, we improve the ways we capture brain activity That we can really kind of particularly focus on the different aspects Or the approximations To me that's the kind of the beauty of the work it now highlighted the these effects Into the broader of our audience Yeah, I think that the fmi, I mean, you know, everything that we've presented is really addressing Things that we were at measure with fmi. So on pretty coarse spatial and temporal scales There's some theoretical work suggesting that kind of You know, the more complex aspects of connectivity that can't be explained by in edr A plain important role in Supporting dynamic transitions between different brain states But that would happen on a very rapid scale that Could be missed with fmi or at least a typically Organized fmi experiment And so one thing we're interested in in looking at further is trying to pick that apart a little bit and trying to Isolate the functional contributions of those more complicated connections Oh dean And I can ask about the metaphor of the the spoon tapping on the on the cello there Metaphorically speaking that seems like it like an external perturbation on a on something that can respond to that Now He'll appreciate I'm I'm asking from a very naive point of view I'm uncomfortable with it Is there any difference then in in terms of say Something that happens Interceptively like something that's simply coming from within system and does that Does that show up in the same way as when there's a response to an external stimuli? Yeah, so You know the actual underlying model a key component that we haven't really talked about is that you have thalamocortical loops and there's a driving input from the thalamus that influences the wave activity in the cortex and You know, they could be sensory thalamic nuclei or they could be Interceptive inputs as well And they would be expected to drive cortical activity in different and you know, spatially non-uniform ways You know really distinguishing between those different types of inputs as far as I'm aware hasn't been modeled yet It's all been kind of a Simple level way you treat the cortex homogeneously and you have a single thalamic nucleus. I mean some people have started to split them up but You know that would be that the underlying principle and the model could be extended In such a way that you could potentially distinguish between inter-receptive and extra-receptive inputs Can I ask him on that? So yeah, I just want to add for example the visual cortex experiment that we perform It's a really simple perturbation, right? But it could represent Different types of perturbation. We just can't you know particularly capture at the moment But it's like an overlying If you perturb it in a certain way, this will be the effect that we see But yeah, you might also know what I was going to ask next thing because that was the direction I was going with that visual cortex example so So from the active inputs community one of the things that we're always dealing with is surprise Say versus sensory attenuation and then on the third vertices the Seeming unawareness prior to the surprise or the attenuated state But still available and so I'm kind of curious When you move away from the visual Can you still have these sort of same waves going through the brain because other Sensory uptake is causing that same You get a surprising feeling like a sense of weightlessness perhaps right or you're you're an astronaut and That's now something that you become attenuated to what Is there I know it kind of stuck with the f mri, but it just seems so The possibilities in terms of what you guys are opening up Just seems really interesting. I have is that something Like the interceptive thing where you can see this now having Opening up new new areas of inquiry I mean, yeah, so I think there's some really interesting work where People have looked at recursive neural nets Structured in such a way that you can Get wave-like propagation between the elements of the network And have shown that the systems that have that wave-like propagation It actually facilitates their ability to predict what's happening in say a movie So it improves their predictive performance Physiologically people have shown that you know just with simple visual stimulation You get these back to front propagations, but there's also front to back propagations Which are presumably encoding the predictions and so then the question is How do the Top-down predictions in the bottom-up sensor input get reconciled between the interactions of these waves You know, I think these are kind of Really interesting ways forward that we're still coming to terms with and we're not we haven't really quite Made sense of what the What the ways mean for computation I know that peter robinson is on the slide here has written a couple of papers Um talking about how these fields can relate to predictions. Um But yeah, I think this is kind of a Very new exciting area that um is both exploring Yeah One kind of connection there is like the standing wave That you showed in the physical water pool And so now when we're looking at a solid color visually There's some and there's some stable Non-equilibrium steady state or stable representation or a stable population code or all these different things which which may not be um Exhaustive or incompatible like at all, but there's some standing wave now. Maybe that is Component number 99 just like principal component 99 its amplitude might be small But there's like some mode That is that's ramping up and it's associated positively or it's ramping down. It's associated negatively And and there's some standing pattern That amidst the pelting of new sensory observations in like a Bayesian updating setting It is remaining the same now the more inertia you have on the distribution the more stable it's going to be Which is like equivalent to saying the less attention you pay to sensory stimuli The more stable your your priors are going to be it could be a prior about perception It could be a prior about action and There are descending motor Patterns as well and all these different modes are being played like In an apartment building and there's like different songs, but they're happening in different frequencies spatial temporal and frequency bands so that we can tune in to different songs like different radio stations But with more spatial richness It's a total expansion of the palette that Recapitulates certain aspects of topology and helps us even identify residuals that Are are most apt to be explained that way But it also opens up to like these big waves That get filtered out like by low pass potentially filtering in different data sets So that shorter term Relationships can become more apparent But that can be the ripple on the wave And so end up throwing out a huge amount just of the sheer information content of rich dynamical data sets Yeah, we're really interesting concept Trying to wrap my hair around it Yes, it's it's really cool because you know again We still don't know what Because what we found is not for example in a different task data that we have Not all modes are equally excited certain modes are excited in quite different ways and We don't know yet what These combinations mean truly behaviorally cognitively and But yeah, we'll we'll keep working on it as far as and see what happens Yeah That's one more quick question. So so I appreciate the geometry part of it So does any of this Speak also to The the sort of the tunneling effects of the sort of that eigen value or eigen mode eigen value Quantum piece where It's you know, you can be writing the surface of this and appreciating that the that the the parsimonious speaking Speak to that surface effect Is there anything about this that also helps us understand better how something goes You know from through the wave like quite literally across from one one slope Past the crown and on to the other side or is that again something that you guys weren't really spending a lot of time on But I did notice I I pulled up the paper and I noticed on Page 11 you were talking about some of the ways that you were analyzing the volumes So that's the context that I was Curious about that sort of tunneling piece to go along with how we sort of follow the The patterns on the surface No, I'm not quite sure I follow you asking how But let me try to let me try to see where it comes as I see it Dean's a quantum tunneling on a sewer Post even and uh with the basis set Of the sine wave It's like imposing this kind of Oscillatory pattern and then you're you're up waiting or down waiting But is it possible to use a basis set or is it possible to detect modes in your analysis? Where there's like an effect locally and then there's like a Then there's like a tunnel and there's like a region that the wave doesn't appear to pass through By the way, thank you, Daniel We're stumped, uh, I guess the main zone What you mean by tunneling, right? So like each mode like if you look at the high frequency modes They are quite patchy and so there'll be sort of disjointed Areas of high amplitude and low amplitude But they fluctuate in time right like they're standing waves and so they'll kind of alternate Up and down so in a sense it looks like there's coupling over long distances Um But um, that's just due to the standing wave dynamics, which is just driven by the interference pattern of the traveling waves um So You know, I'm not sure if that's kind of what you were getting at but I guess Yeah, from a perspective of just if you were just looking at the standing wave dynamics It would look like Things in spatially disjoint locations were kind of Uh coupled in some way It's like one jello It it stays put where you hit it and it stays put one mile away And it oscillates between like a jump rope Um, or it could be hit somewhere that's also very variable then it'd be like the opposite of a jump rope with one one mode And that's what's captured in the set of functions that are then parameterized So then when you assemble them or when you um superimpose them you you could get like Kind of digitized looking outcome patterns In terms of the real empirical correlations And it's map not territory and it's tempting to interpret these very literally as with many statistical tools When I think as we've discussed like there's there's multiple possible mechanisms and in your paper and in your work you probably thought of many many mechanisms that lend themselves to like unique empirical predictions but basis set decomposition is used like in spm which is kind of like the One of the precedents of active inference and and the work by carl frist and others. It's using basis sets as well so Yeah, they used uh they used it everywhere All right Do you have any closing Or or near closing like thoughts or where does the work go? How do people what data sets are salient or questions or cognitive phenomena? Or like what are you excited about for your lab and for the fields? It's your lab I mean there's there's a few different uh why is that we're interested in in progressing it. Um So one is I guess trying to reconcile this purely geometric approach that makes a very simple approximation with connectivity with the actual connectivity of the brain and trying to understand if The contributions of these kind of complex connections are not so easily revealed by a classic fmri experiment What are their contributions to dynamics? and is there a way of kind of Bringing some of that complex connectivity into the into the wave model um and trying to get some traction on on that question um Another aspect is You know, we've done the simplest thing we treat all areas of the cortex as homogeneous But we know that's not the case Areas vary in terms of the cell density degree of myelination level of synaptic complexity and density and so on And so that in a sense could be imagined you could imagine that is changing the effective geometry of the cortex and so we're playing around with How do you introduce that heterogeneity and how does that change the mode structure? and Does that have an impact on your ability to reconstruct dynamics? um We're then also interested in sort of taking this forward to Leverage these approaches for thinking about different ways in which we might be able to map brain changes in different disorders or variations across species and so on and so kind of like um You know classical spm analyses, but um making inference at the modal level and seeing if that can provide some useful insights So there's a few different strategies Any other comments or thoughts Yeah, dean where where do you go after this dean in your own neuro imaging career? No, stop it. I just think it was a very clear picture and I really again, I really appreciated the paper itself But today's presentation sort of brought it to life and made it a lot I know that you guys were really trying to focus on making sure that you got all of your your All your formalities in in in a row, but just the way you presented it today made it a lot made it even more come alive in terms of that the uh The mechanisms that you use to sort of come up with with uh a clearer picture But that's what I came in with way away with so again, thank you very much Thanks for the new day. Thank you. Yeah, we'll look forward to seeing The continuation of this trajectory. You're always welcome back. So thank you both Thank you. Thanks guys. See ya. Bye