 here and So I want to talk about what I call cognitive neuro informatics, and I'll start with a quote which I think probably Describes the experience that a lot of us have in doing science right? We're drowning in information, but starving for knowledge the question is how do we take this mass of information? That hits our retinas every day and actually feel like we know something and it's you know I do functional imaging and it's particularly an issue there where if you look at the number of publications that mention fMRI it's now somewhere between three and four thousand a year that are coming out and you would hope that you know We would feel like we would get some sort of Knowledge that would accrue from all of these activation maps But often you end up sort of feeling like it's just a very techie version of a Frenological map without really getting us more towards understanding basic function So what I want to do today is first kind of ask the question is how is it that we go about trying to map mental function onto the brain? Because that's sort of my Fundamental interest is understanding how does the brain implement mental functions? And then I'll talk about several different projects that we've done that have kind of tried to address this in different ways First being the cognitive Atlas project, which is really building an ontology for cognition second is topic mapping so basically taking that and trying to figure out how can we map it on to To imaging data and finally talking about data sharing So how is it that we map mental function onto the brain? Well, usually we use a an approach that That Rick Henson first called forward inference, right? Which is basically we try to wiggle something in the mind and see what in the brain wiggles We do that so we manipulate some mental process like working memory maintenance I tell you hold this phone number in your head and we look at what brain areas Turn up in activity when we do that And then we infer that those brain areas must have something to do with whatever mental process It was that I was wiggling and so We see for example that a region like the interior singulate shows up when we Turns on when we have people engage in maintenance, but If you look across lots of different studies, you see that lots of different psychological manipulations End up turning on very similar looking patterns of activity, right? So this is patterns of activity. These are actually estimated Via meta-analysis so these are actually patterns of activity that are associated with the presence of these particular words in papers They're not actual activation maps, but if you look at actual studies You see a very similar story that the same areas light up across many different types of task manipulations So the question is what do we make of that? What's what does that tell us? Well, there's a lot of different alternatives, right? One is that there's some confound that drives all of those such that you know whenever we engage in any of these tasks It turns up our autonomic nervous system and that's causing activity in the intersingulate because it monitors the autonomic nervous system It could be that there's you know different cortical columns within the intersingulate that are doing all these different things But that they're really distinct right and if we had the right resolution we could see that it could be that The ACC does all those different things, but it does it in different neural context So when it's when it's communicating with the left posterior parietal it does one thing when it's communicating with Cauteate Nucleus it does another thing Or it could be that we're just chopping up the mind the wrong way that these are all not really sort of fundamental cognitive functions, but But that that you know we're kind of misled about the structure of the mind and a thought experiment that I like to suggest is What would have happened if the phrenologist had had functional MRI, right? It's doubtful that they would have decided that phylo progenitiveness and whatever their other faculties were that those weren't real, right? They would have found blobs that map onto those things because presumably all those The faculties that the phrenologist had are sort of correlated with real psychological functions and such so something would have lit up And they would have taken that as evidence for the reality of those things, right? Which it's pretty clear that that would not be the right that would not be an appropriate strategy so So this has kind of led us and others to ask the question of you know What is it that what is this mental stuff that we're actually imaging and I think many of you may know this book From about ten years ago now called the new phrenology in general. It's kind of a kooky book Because he doesn't really know much about imaging, but but he actually makes some really good points regarding Kind of what it is that how it is that imaging works and he makes this point that That basically we haven't done a very good job of formalizing we as cognitive neuroscientists people who use neuroimaging Haven't done a very good job of formalizing What it is that we're actually mapping on to the brain so what he calls the butterflies of our mind We haven't sort of laid out the the really the ontology of mental processes that we then want to map on to the brain So if we ask the question, you know, what are the atoms of the mind? What are the fundamental parts of the mind? You know one answer is this right the the phrenologist gave us an answer that was derived from early faculty psychology that said that things like I can't even read these like you know acquisitiveness and adamantiveness and Individuality and mirth are all fundamental psychological functions Most of us who study the brain now don't really believe that those are the way that we should chop it up Instead we think it's probably something more like this. We have things like perception and attention and memory And those might have some sub parts And you know one person might think that they're related in one particular way But you know other psychology is not a place where in anybody agrees on much of anything So other people might think for example that you know Memory is that for example working memory is not really part of memory that it's part of attention Other people might think that working memory doesn't exist at all So for any of these concepts you can pretty easily find somebody who doesn't think it's a real concept that it's really just a Kind of an artifact of some other function so the the first The sort of foray into Informatics that we made was in trying to develop trying to sort of make some sense out of this ontological mess and try to develop a formal ontology or at least something down the road towards a formal ontology of Psychological function, and so we developed this project called the cognitive atlas. It's online at cognitive atlas org it's been up for several years now and Basically the cognitive atlas is meant to be a collaborative Knowledge building platform, so we know that people in psychology don't agree on anything We want to be able to not just say Russ Poldrack and his lab think that the mind is structured this way We want to be able to say how does the field think the mind is structured and be able to capture the disagreement around that this sort of grew out of Out of talking to people who think about you know JB mentioned that I was in part Affiliated with psychiatry at UCLA and I spent a lot of time talking to people who study Psychiatric disorders and in the end want to know about you know, what are the genetics of psychiatric disorders and The intuition that that world has sort of come up with is that if you want to understand the link between Genetics and psychiatric disorders the way to do that is via what they call endophenotypes Which are basically things that sit in between in the causal pathway between genes and those diagnoses of schizophrenia or whatever other disorder you're talking about and particularly interesting endophenotypes are Psychological functions and brain systems and so we want to we wanted to start thinking about how can we better? Formalize this relationship if in the end you want to have sort of you know automated reasoning that can go from genetic association Data or knowledge about Genetic function all the way up to psychiatric disorders You have to make all of those links and the link that we are particularly interested in making was this one between brain systems and psychological functions and The intuition that we had is that how do we get between those two levels? you know, we can't see psychological functions physically right the only thing we can do is poke people's brains using psychological tasks and see what happens in the brain And so it's that link between cognitive function and brain systems is kind of mediated by tasks that we really wanted to capture So here's a sort of a schematic of how the the knowledge is laid out in the cognitive atlas We break things into two different types of Entities one is what we call mental concepts and these are kind of conceptual entities things that are in the head like working memory or Pain or you know sort of any any kind of psychological concept that people might talk about Then we have mental tasks, which are actually the things that are used to measure Those mental concepts and we came up with a novel relationship that is instantiated in this knowledge base That we call the measured by relationship and the idea is that any particular Cognitive concepts who in this case we might have a concept like Response suppression and we have some ontological relationships between different mental concepts So for example, we might think response suppression is part of response inhibition but in the end this particular thing is defined by its measurement and In this case it's measured by something we call SSRT But there could be multiple different things measuring it The point is that if we want to in the end link real data up to these psychological concepts we have to do that by linking to the the measurement relation or by by Defining the measurement relationships that links those concepts to two in the end to real data So let me just show you what a couple of examples look like within the within the cognitive atlas This is one of the pages the one for working memory and you can see that we have a little definition We have some ontological relationships, so we have you know, what are the kinds of working memory? What are the parts? What is it a kind or part of so it's kind of standard ontological relationships? And then if you go down on that page one thing you see is what are some of the tasks that we think measure? Working memory Okay, so we can define task and we actually define We don't say that working memory is measured by a particular task in general because any particular task has many different measurements That are taken so it in general it's going to depend on a particular measurement within that task that defines that particular concept That's these contrast measurements then we have a we also use some kind of a quick and dirty text mining Using a project called pub brain that I wasn't going to say much about but I can say more about it There's time at the end which basically mines PubMed abstracts for the presence of Particularly basically we have a lexicon of anatomical terms We mined PubMed abstracts for any particular search term and create a sort of a pseudo activation map That basically says what parts of the brain are people talking about in the context of that particular search term So we have one of those maps. We also have also links out to Neurosynth, which I'm going to tell you about in a bit And then we have discussion a la Wikipedia Which in the end is meant to kind of allow people to go back and forth and kind of capture some of the Disagreement that we think is going to exist on some of these We also have links then to tasks. So we for any particular task we define. What are the conditions? What are the contrasts? What are the indicators of the things that get measured there? And one of the interesting concepts that my colleague Bob builder from UCLA came up with this is idea of a Task phylogeny the idea being that you know many many psychological tasks derive from other tasks often going back to you Know the 1920s So in this case we could say for example, you know that this particular task the operation span is a descendant of an earlier Task called the digit span so we can capture kind of the where tasks come from And also again capture discussion. So what's what are our goals in this project? Well, we didn't you know when we first started out We looked at things like protege and we decided that we wanted to build something that was very different in that It would allow Scientists to come on and do something interesting quickly. So they could come on within five minutes. They could actually You know do something useful and interesting We can't require them to be ontologists because that's just not going to happen And so we decided to try to take advantage of what's been learned about social collaborative knowledge building One of my colleagues Nikki Couture is he's actually worked on analyses of Wikipedia and other kinds of social collaborative knowledge building And and it's thought a lot about how we can do that So we try to sort of place it somewhere, you know on these two dimensions of ease of use and structure somewhere in between Wikipedia and something like protege Where it's it, you know, it can be used by a non ontologist But it it has a little more structure than than something like Wikipedia would So we you know we as I said people don't agree on anything in this field And so we've tried to build ways to we want to involve the community in specifying the knowledge and so far the community has largely been My lab and a few other labs who sort of work together on it But it's open to anybody to come on and and contribute And we also try to capture disagreement one of the ways we do that is through the discussion page that I showed you earlier another is through what we call concept forking which is sort of like the the Disambiguation page idea from Wikipedia where if we have a term like behavioral inhibition that people really use in multiple different ways We can say this term has multiple senses One is the cognitive sense one is the temperament sense and then those get treated really as different concepts So that's one of the ways that we can allow People to sort of disagree as if in the end they really can't agree on what a term means they can fork it and it can have different senses We've added some personalization features So now when you log in you have a dashboard that can sort of tell you there's interesting stuff That's relevant to what you're what you've been working on or relevant to terms that you flagged you can follow particular concepts We have a semantic web representation, so we have a sparkle endpoint you can you can grab all of the data via that endpoint as RDF and Just in terms of where we are with the project right now we have about 650 psychological constructs about 393 tasks and That's that's pretty much leveled off and it actually we did a big sort of culling recently where we got rid of a number of them That didn't really make sense But that's that's about the right number. I think for where we want to be for the for the short term at least We made the decision early on not to build the project around Formal ontology language we built it using a relational database that gave us a lot of flexibility and the kinds of knowledge We can capture and then we what we did was on the back end Basically shoehorn that into a formal ontology So we now have an an owl version of the knowledge base It's available by a bioportal or and we had actually the code that we used to generate it is up on github as well and we For sort of for for giggles our software developers built an Android and iOS app So if you want to know what working memory is when you're walking down the street, you can bring that up on your phone So we've tried to integrate as much as we can with other projects, so we now are included in Neurolex All the the terms from our from our ontology We've worked with the cognitive paradigm ontology who are working to build sort of more detailed Descriptions of cognitive tasks that we've sort of aligned our information models I'm going to tell you about Neurosynth. We're deeply sort of engaged with them and I'll also tell you about open fMRI Where we're using this as the platform to describe tasks Okay, so One problem with the cognitive atlas in general is right now the database is pretty flat There's a lot of terms not a lot of relationships between them It's certainly not a lot of deep relationships, and so we wanted to see if there's any way we could use Data mining tools to try to pull some more interesting structure out of the the published literature And then see if it's sort of if that structure comports with what we expect out of the brain imaging literature So the approach that we decided to take was to use what have been come to call generative models of the published literatures The idea is that for any particular document you treat that document as a mixture of Topics and every word in the document is is a sample from from one of those different topics So one of the most popular Approaches for topic modeling is called latent directly allocation from Dave wine colleagues That's the technique that we ended up using And basically the idea is that you have this set of latent Topics and you have to say how many topics there are and then those topics generate Text and what you do is you use Bayesian inference to basically infer the topics from the text So here's the idea that for any particular paper there might be so you know papers in the fMRI world There's gonna be some pay either one topic is sort of decision-making right one topic is things having to do with fMRI One topic is things having to do with basal ganglia each of those topics is associated with words that could show up in the document And so you you start out only having the words across a bunch of documents and you infer what are those topics and so this has been used before for example in in on abstracts from PNAS Grevith and cybers did some early work showing that you can pull out different parts of PNAS abstracts Or different parts of the structure of PNAS papers by just doing this topic modeling stuff on the abstract So it's really sensitive to Different the use of different terms together It's actually it's an amazingly powerful tool when you start working with it Okay, so how did we do this so we first we have full text on about 5,800 articles from the fMRI literature and what we do is we We take all those papers and first we have to decide how many topics do we use and we do this by a cross validation where we split the Split the papers into sets run the topic models at a bunch of different Dimensionalities and then you can you can actually get from the topic model the empirical likelihood of the left out papers and You what you do is you basically find the topic dimensionality that maximizes that empirical likelihood of the left out papers So we started out just doing it on full text We we left out a few things that we thought were important one is standard English stop words To is author names because we found when if you do this You don't leave out author names most of the topics end up having a couple people's names in them And we didn't want that so basically we got rid of any name that That showed up on any paper as an author in the in the database except if your name is pain or something like that That happens to also be a cognitive term There is someone whose last name is PA in and then we also left out brain structures Because we didn't want the topic modeling to be driven by sort of the brain structures that were popping up We really wanted to be sensitive to the conceptual stuff around cognitive processes So here's what some of the topics look like and so the topics are not ordered right? We there's a hundred and thirty top are sorry 160 topics these are just a few examples and The idea is that each of these are sets of words that define a particular topic that that different ones of these papers are about And so you can see that it finds some that are just about very general stuff like that you would say in writing any paper Right figure abstract those kind of things Study analysis. This is one of my favorites Which is basically? Spellings that British people use So but you also see you can get some very very detailed ones here's one about Alzheimer's disease MCI One about aging here's one about sort of gambling neuroeconomics one about memory So it does a pretty good job of picking out sets of words that seem to be you know We can all look at these and say oh, yeah, that's a sensible topic, right? That's about X But there's a lot of stuff here that that isn't really cognitive right and we were really interested more in getting at the cognitive structure, so We took a next step which was to basically Only use the terms from the cognitive atlas So basically we take that the words and phrases that are in cognitive atlas and we model the phrases just as like a single token And then we throw out all the other words So we have in this case when we did it we had 545 terms in the atlas And here we get about 130 or we get 130 topics. That's our dimensionality and So here's some examples so now we get topics that are much more focused on cognitive stuff, right? Because that's the only terms we've let in and The underscores here just mean that that's a that's a compounds term from the atlas So you I've sort of focused on memory here, right? You get three different memory topics that are all about different aspects Here's one about working memory one about episodic memory one about semantic memory Here's one about response inhibition one about cognitive control And you can see that each of them maps across a decent number of documents in the corpus, but certainly not all the documents, right? It's relatively, you know, usually less than 10% of the documents Load on any particular cognitive topic Okay, so This tells us that there's interesting structure that we can pull out using these topic modeling tools now we want to ask does can we map this back on to brain activity and We don't have the raw imaging data for each article, right, but It turns out as you know that brain activity is usually reported in a somewhat standardized Format in these sort of tables in papers where it's in this XYZ M&I or Talorak coordinate system and The the five thousand eight hundred papers that we have are chosen for particular reason which is that Tal Yarkoni was able to develop a tool called automated coordinate extraction they can actually pull out the activation coordinates from all of those papers and In a paper in Nature Methods last year we showed that that we had if you compare it to David Van Essen sums DB manually annotated database. We get pretty good precision and pretty good recall Against that database so we seem to be doing a pretty good job of actually getting these coordinates accurately and so what we can do is you know for each of those papers we have the full text and we have the the activation coordinates and In first time and I sort of try to convince you that this automated coordinate extraction thing actually works So we can create meta analytic maps that for example, this is just using single terms yet So we're not up to using the the topics, but this is Looking at the association of activity at each voxel in the brain with the present with the Sort of over-presence of a particular term So I forget what the threshold is but this is basically when I say that the paper is over-represented in the term visual it means that it occurs more often in that paper than it does in most other papers and You see that you get the regions that you would expect for visual auditory and sensory maps It also works in sort of more complex stuff So for sort of working memory executive control you see kind of as you expect overlapping maps for lots of these different concepts Here's maps for language and again you see this sort of left hemisphere activation that you would expect and Finally emotion reward social processing again just making the point that we can this is just looking for the words in the paper Now single words in the paper We can do a pretty good job of pulling out kind of you know what we already know about About the localization of those particular functions A more the more interesting question is whether you can actually classify Whether a paper is going to over-represent a particular word based on its Its activation pattern so JB mentioned the reverse inference question that I've sort of been talking about for quite a while This is one way to try to get at that how well can you tell what a paper is about given the the brain activation? And so in this work, we used a naive based classifier and asked at a couple of different levels you know can we predict for example whether a paper is about working memory or emotion or pain where about means that it over-represents that particular term and We did cross validation and in this case we did it across 25 high frequency terms And so first I'll show you the pairwise Classification data, which is basically can we decide whether a paper is about one or the other of these things and then we actually move to multi-class So here's the pairwise data and basically if it's green That means that we can do an okay job of classification. We're in sort of the 70% range well above chance The blue ones are the ones where we're not doing very well. So for example, let's see that's imagery and attention are sort of pretty hard to pull apart and So What this starts to maybe tell us is that some of these concepts that are hard to pull apart might not really be Either they're being used in in sort of imprecise multiple ways in the literature or they're really not different concepts I'll come back to that This is what we get if we try to do multi-class classification across many different terms and again the the accuracy is is not great But it's well above chance the the sort of dotted line down here is chance. This is our accuracy up to Classifying which of ten different terms is going to be present So it tells you that despite the incredible sparsity of that sort of you know three three dimensional M&I space representation You can actually do a pretty good job of pulling out the maps that can drive this kind of classification All right, so now What what can we do in terms of the topics that we've pulled out? So I've shown you that it works for individual words. How does it work for topics? So so I mentioned that you know for every document we know it's loading on each topic and Each document loads on about six point five topics on average So we have the the activation coordinates for all those papers And then at every voxel we do a chi-square test to basically say is the activation of that voxel across papers associated with the loading on that topic across papers and we and then we do a whole ring correction for each of those and So we can get an association map between a particular topic and activation. So here's an example This is showing threshold correlation thresholded correlations. So everything that shows up here is FDR corrected for this whole brain Blue means negative correlation red means positive correlation this is the map for memory working memory maintenance visual working memory and All the maps I'm going to show you the the upshot of them is yeah, they kind of make sense It's actually a challenge to know where we go beyond kind of saying. Yeah, this already tells us what we already know and That's a I think that's an interesting point for discussion But but certainly we see very clearly that this is able to pull out probably in an even cleaner way than the single word Analysis so for example, here's episodic memory recall verbal memory and you get you know nice bilateral medial temporal lobe and retrospinial cortex Here's our semantic memory one where you get this Left is right here. So you get all this left prefrontal temporal stuff Here's that Response inhibition one where you get the right prefrontal stuff that you expect along with basal ganglia Here's the cognitive control one again. You get prefrontal sorts of things that one expects from the cognitive control literature There's a motor one so you get exactly what you would expect motor cortex And you can see that there's also some that so here's for example one that loaded on a relative Sorry, this is the number of papers it loads on the previous ones We're all loading on you know several hundred papers this one loads on a relatively small about 1% of the papers Is that yeah about 1% of the papers in the in the corpus? And it's about regret surprise reasoning arousal and you find this sort of fairly focused stuff down in the amygdala So it says that it can actually find relatively subtle things in the literature that you wouldn't necessarily see if you were just you know There probably are not enough papers that use the term regret to actually find You know a robust map if you just look for that term But using the topic mapping because we can sort of integrate across terms. We may be able to get more power to actually find that So we next wanted to ask whether we can use this to find out something about mental disorders So remember that you know, I mentioned the idea of the endophenotype The idea that we want to be able to sort of figure out what's sitting in between, you know, the disorder and the and the genes And so we started by basically doing the same thing that we had done with the cognitive atlas terms But now we do with the disorder term So we took a set of we basically came up with our own lexicon based on NIF Standard and on DSM which sort of spans across both neurological and psychiatric disorders Um, we that gave us 56 disorder terms and they're here We had to do some synonym mapping because there's a lot of different ways that people can use all of those terms So we map them all into this set of 56 items And here again, you get things that make a lot of sense So here's one for amnesia and Alzheimer's disease and Corsikov syndrome and vernacles and you get primarily bilateral medial temporal lobe and interestingly all of this kind of Lateral and medial parietal stuff, which I think might actually tell us something interesting as Hopefully it's clear that one of the challenges for us is really figuring out what's a discovery and what's noise out of this stuff And and I'm actually really interested to hear if people have ideas about how to address that challenge Here's one for gambling drug abuse impulse control disorder and you get basal ganglia and Ventromedial prefrontal cortex Here's one for dyslexia specific language impairment and you get these left hemisphere regions that are usually associated with that And here's a final one for anxiety disorder panic disorder again. You get bilateral amygdala So again, we can we can pull out Pretty robustly the maps of regions that we know to be associated with these terms We can also ask about kind of the higher order structure there's a lot of interest these days in Sort of how our different Psychiatric disorders related to one another with the idea that they might be actually Describing sort of underlying dimensions rather than so for example schizophrenia and bipolar disorder Many people think are not really separate disorders that they're kind of falling on some sort of on a couple of different dimensions So what we did was we took those topic maps for each of the different topics and clustered them in in Brain space to basically say how using hierarchical clustering how do the The sort of neural representations of those different topics relate to one another and interestingly you get Several clusters that make a lot of sense. Obviously. This is arbitrary in terms of where I put my My my tree cut here, but it was a pretty obvious Place to put it over here and sorry about all these these crazy acronyms But basically these are this is aphasia and dyslexia. So these are our two language disorders off on their own over here these are amnesia Alzheimer's disease and then several autism Things so those are kind of off on their own over here. These are all the sort of schizophrenia ADD conduct disorder basically what are called externalizing disorders in the literature and Then these are all of your depression anxiety gambling Phobia eating things like that what are called largely what are called internalizing disorders? So it shows that not only can we pick out kind of you know individual mappings of brain systems to Two particular disorders, but we can pick out the larger clusters of those things based solely on the brain maps that come out of this This meta analysis So in the last question we wanted to ask can we actually discover anything new can we discover any new endophenotypes? so that would be groupings of disorders and Cognitive processes that group together via sort of common representation in these brain maps and to do this We use the sparse canonical correlation analysis where we basically come up with you know weightings across mental functions and disorders that basically end up projecting into a common brain space and You know again here we find some things that sort of make sense So for example and sorry about the tiny text Well, we found a bunch of stuff related to you know mood Induction reward reward anticipation and anxiety disorder depression gambling obesity There's another one about emotion Valence and it maps to amnesia schizophrenia autism So you start getting kind of Clusterings across disorders now This is really a case where it's hard to know what's discovery and what's noise and so we You know we haven't pushed this part of it very hard yet And I think it's it's going to be interesting to see kind of where we can go with it if we can actually drive some new Hypotheses for example that one might be able to find relationships between particular disorders it based on particular brain systems using this sort of approach okay, so So let's say that you know we so clearly the the topic mapping stuff gives us some information about the structure in the you know in the literature and and Kind of what how these terms relate to one another but it doesn't give us sort of strong enough structure that we want to really start Building ontological links, so we're still going to need people We're still going to need domain experts obviously to come in and do the annotation on on the ontology on the cognitive atlas But let's say that you know in in at some point in the future We actually have such a such an ontology built One idea then what what can we do with it right? We want to we would hope that it's actually useful and the reason that I as a scientist got involved in this the intuition that I have is that Once we have these ontologies we can use them to actually tell us which parts are right and which parts are wrong So my intuition about you know all of this psychological stuff is the a lot of it You know a lot of it hasn't really changed since William James wrote his 1890 principles of psychology and And probably has to be incorrect right a lot of it is what we call folk psychology is kind of how we intuit our minds to be structured Which may well not be a very good? Way to actually figure out how the mind works So what we want to do is ask you know once we have the ontology asked are there Are there parts of the ontology that seem to comport with how the brain breaks things up? Are there other parts that don't? So one way that we're thinking about doing that and we haven't actually we don't have the data to do this yet But it's kind of a a promissory note on how we can think about doing this in the future is Thinking about this in terms of meta-analytic testing of cognitive theory So let's say in this case. I'm just going to talk about behavioral data so let's say I have four behavioral measures on different tasks and One and I have an ontology that says well these two particular measures Rely on a concept called inhibition and these two particular measures rely on a concept called updating all right, but somebody else and sorry and and those that Claim about the ontology has a particular implication regarding the covariance Structure amongst those data right it suggests that these things that rely on the same concept should be more correlated with one another Than they should be with things that rely on the other concept. So that's our little Covariant predicted covariance matrix up there Somebody else would claim that oh, they're all just Related to some very general thing called executive function that can't be broken up and That predicts another covariance matrix. That's different So if we have data across all these measures We we have observed covariance. We can actually use Techniques like meta-analytic structural equation modeling to ask which of those actually fits better with the observed data Right, so then we can start to basically say how do how do particular? Ontological claims fit with the data and that's part of why we want to be able to capture multiple different senses of Are multiple kinds of ontological claims within the cognitive analysts so we want to do this with real data and Our intuition is that the data that we get from Neurosynth probably is not sufficient to do that In part because I think that just you know finding terms in the text is not a strong enough method of annotation that we really need Something that's that's probably more human centered Or at least something that's human verified Such that we have annotations linking the ontology to each of to any document that we're looking at The second thing is we need a really broad range of tasks that tap so we basically need We can't really we don't want correlations between What particular task is being used and what particular concept is being measured each concept needs to be measured by multiple tasks? And we need to have voxel-wise data So instead of the the coordinate-based data that that I've shown you we really need data at the voxel Because many of these things they might not be distinct at the kind of very broad level But they might be distinct to the level of pat of finer-grained patterns of activity That's become clear in the literature and so in thinking about this we got excited about trying to To actually implement data sharing for fMRI and most of you probably know the history of data sharing for fMRI which started Back around 2000 with the fMRI data center Which generated a lot of controversy and in the end ended up sharing about a hundred data sets, but Stopped accepting data about seven years ago and has not shared any new data in quite a while And as far as I know doesn't plan to share any new data So so sensing the need for open data sharing We developed this project called the open fMRI project. It said open fMRI dot org It basically started with me when I moved from UCLA to UT Just saying I'm gonna take all the data. We have to anonymize them any way to move from UCLA to UT We're gonna take all these data and make them available online openly for for download and We've we've tried to come up with structures such that they can be easily analyzed We've come up with a very Tight format that all the data sets are formatted by so once you know that format You can easily just go analyze any of them And there actually exists a night pipe workflow to do that full analysis that that Sautra created We right now have 14 data sets with about 250 subjects and we have more coming in We got a grant from the National Science Foundation that supports us to sort of run the site and put our own data in And also supports a number of other sites with a part-time data manager Whose job is to basically take data from that lab and feed it into this to this project And we've also had volunteers from other sites who've Who've given us data as well and we're hoping that that will increase One thing that we do is we integrate the description of the tasks in open fMRI With the cognitive Atlas project So this is the open fMRI page for a particular data set It's rhyme judgment data set and you see that it says there's one particular task here with two conditions And when you click on that it links out to