 Okay thank you. So firstly thank you for INCF for inviting me here to give this talk. I will talk about brain atlasing and data-basing in the BrainMinds project. So what I actually want to do is focus on brain atlasing and how the atlas that I've been working on has is used as a common space and a way to organize all of the data that's going into the database in this project. So for those who don't know the BrainMinds project started around five years ago so it's in the it's halfway through and the goal of this project is to map the structure and function of the common marmoset at the micro meso and macroscopic scales and apply these results to the diagnosis and treatment of human disease. So it's one of Japan's large-scale brain research programs and it consists of a large number of independent labs and in the past is around 65. So this is this is both a good thing and a bad thing. The independence allows people to pursue their research interest but it makes it more difficult to organize data if you want to put this into a database. So in the past five years in the neuroinformatics unit that I was a member of the infrastructure and basic systems to register store and share data with metadata was developed and so this is a diagram of the system that was developed it's basically an internal database with 1.2 petabytes of storage and an open database and data can be registered into the internal database by this research portal interface. There's also an high-performance computing cluster that's connected to this storage and then the open data is accessible by the data portal. So currently there's a large amount of raw trace and MRI data that's in this internal database and the next step is really to compile and a large amount of data that can be shared openly with the public and also develop the website that goes along with this. So this is the current website if you have a look at it it still needs a lot of work and that's the that's the plan in the coming five years. So what data do we want to focus on for the this open database? So actually we will focus on the meso and macro scale structural data for the open database. Part of the reason is we have in terms of the neuroinformatics activities we have a strong research collaboration with the groups that are involved in this data and all of this data will be organized around the brain mines 3D brain at this. So computationally there's a transform that will take you from the individual data to this common space. So here is kind of a summary of the types of data that we have been collecting and are organizing. So we have a large amount of trace injection data. The plan for that is an example of the data sizes. So this trace injection studies is using the tissue site system. So the goal is to have around 200 brains after 10 years and each brain the raw data is 4 terabytes so you can see that it's taking up a lot of space. In the next five years we plan to get adult brain ISH for around 1000 genes and high resolution MRI with a 9.4 Tesla MRI system along with the division weighted imaging for all of the brains that have trace injections. Cytoarchitectonic maps for a number of brains resting state fMRI and the functional data will be we hope to integrate that into the open database but it's not yet decided which data that will be. So I want to talk about the brain mines 3D mama set brain at this that is being developed. So I worked on this and I published on it and this is based on Dr. Hashikawa's Cytoarchitectonic annotations. This was also compared the nomenclature is compatible with the Paxenos Atlas so that's if you if you are here yesterday Piotra uses this this Atlas so this is one step towards interoperability and if you see here the the key point is that the nissle sections have been 3D reconstructed and aligned with the MRI T2 contrast and we have the annotation the parcelations here and we also have the divisions between the the brain boundary the grey matter white matter and an estimate of the mid cortical surface and the best thing is that it's in nifty format so it's very easy to use in your software. So we've also been upgrading this so now we have a population average 25 based on population average MRI T2 weighted image contrast based on 25 image scans and this has been mapped with the Atlas so now we have a very nice kind of MRI contrast that is being from the same scanner being used in the project. We also created a flat map and there's this this parcelation is 116 regions which corresponded corresponds to the original Atlas divisions and there's also an aggregation of this so some of the MRI studies want to use a Corsair grain division. So how is this Atlas being used? I want to spend a bit of time describing a use case as to how this Atlas is being used and it also give you an idea of what the type of data what type of data is being collected in the project until now so if you remember you might remember some this this mama set brain figure from Piotr's talk and his work they're doing retrograde trace injection studies and in our work we're doing anterograde trace injection studies so we inject here at the source the anterograde tracer infect cells and causes them to express fluorescent proteins that travel down to target sites so the idea of this trace injection connectomics is we do this for a number of brains we computationally process this data and reconstruct it map into a common space and then we can generate a connectome of the mama set brain so what does the tracer signal look like? Here's an example so this is generated from a nanosomal batch slide scanner it's one of the one of the techniques that has been used in the project so this bright signal is the tracer signal this is the white matter so this is a coronal section of the mama set brain if you zoom in you can see that these are the axons and where you see the blurry blurry regions this is the out of the focal plane so what is the actual procedure that you take to to go from raw data into some matrix of connectivity so you have the raw data you want to 3d reconstruct this raw data map it to a common space with an atlas you also want to segment out the tracer signal and you also want to identify the injection site and using this information you can add one row into your connector connector matrix so in this case for this for this example for this use case the experimental data that we that we acquired so we have injected the brain the brain is sectioned he's in using a micro time and at the same time block face images are taken so if you assemble these block face images they're all aligned and you can get a stable reference of the brain shape and two traces are injected once so you have green and red so there's a correspondence between the the fluorescence images captured by the nano zoomer and the block face so the idea is to map these fluorescence images to the block face so you can get the 3d reconstruction so just in terms of steps this is the first step you want to isolate the block face from the the dry I said it's surrounded and then you have a 3d reconstruction of the of the ex viewer brain and then the idea here is to register the population average MRI to the block face so this is in the individual brain space its top line is the horizontal sagittal and coronal views of the individual brain and the idea is to once you've registered this you can slice the MRI registered into the individual space and instead of using the block face which has low contrast you're registering the the the average MRI signal so you're registering the the fluorescence images to the average MRI signal which is shown here and in this procedure I'm using the ants normalization tools both linear nonlinear registration so once you've done that you can recover the shape and you can also since there's no interslice consistency in the reconstruction you can also use this approach which basically removes high frequency distortions in the registration and you can effectively align and reconstruct the full brain shape in a more reliable manner so if you if you look at the cerebellum here then you've got a more smooth a smoother reconstruction so unfortunate so in this nanosumus slide scanner they wanted the experimenters wanted to capture the very faint tracer signal so unfortunately the sometimes saturated the injection site so what they did was they took separate images of the injection site at lower arm exposure and so this is just showing that we can take these separate images isolate the cell bodies and map this back into the 3d reconstruction then into the common space and we can recover the injection site volume so that's it there and you can also see where it was the brain was punctured so what does the reconstruction look like at the end so if you just took all the slices the sections and from the taken from the nanosumus slide scanner and stacked them by the centroid you'd get this kind of distorted brain as you can see here but if you use this reference brain and you use computational techniques to align everything you can recover the true shape and actually this bulge here is this this brains a bit deformed so that's not a error so you can see that one problem is how do you so the next problem is how do you actually segment out this tracer signal so the images from this nanosumus slide scanner not stable they the intensity varies and there's nonlinear illumination distortion so removing the linear variation is not too difficult but as an example of the nonlinear kind of low frequency distortions and intensity if you look at this region of the coronal section where there's strong tracer if you just threshold this out of this some set value you can you can isolate this strong tracer signal but if you're also interested in capturing this weak signal you you need to reduce your the level of your threshold but at the same time you're getting some of background fluorescence of the brain so how do we deal with that well artificial intelligence has become so popular now we trained a unit architecture on manually created binary masks so we sliced the large fluorescence image into small regions manually drew masks took a very long time and tried to carry tried to use this neural network to characterize the different kind of features that appear for the tracer and also ignore the background which can be bright as well so as a result we could successfully do that so here's a here's a close-up of the the occipital lobe and if you zoom in further you can see this is the tracer signal for a dense and we can also segment it so this is after binarization and this is a video of the result as well so this is a zoom in of the one chrono section and how we can use AI to segment out the tracer signal and this image is very big so it's three gigabytes 16 bit image at five fifty thousand by fifty thousand pixels so in order to process this image we had to divide it up into smaller regions and then apply the neural network and what does this result look like once we map everything into the common brain space so here's some reconstructions this is for one injection into MT but this shows the first row is the green channel of the one tracer and the second row the other red tracer so if you look closely you can see this is the injection site here and the injection site here and what's interesting is even though these are both in MT the green inject I think let me see the red injection was injected more superior and the green injection slightly lower and you can see how the projection varies even though the injection was done into the same region so this is also why it's interesting to study the and I think intrinsic connectivity or the connectivity within regions and not just make a summary region to region and in these type of studies so to normalize the data further what was done was the neuro anatomists actually annotated each of these brains being used in this some trace injection work so now I made a pipeline to convert the their annotations into 3d so now we have individual cortical atlases for each brain and we can use this information to further normalize this data when you map map into the common brain space so this this also helps us validate our results we have we have we don't have to rely on a single parcelation scheme we have actual some psycho psycho-architectonic evidence for the the brain boundaries and how we use this information we actually try to align the data in the cortical flat map space so 2d to 2d flat map space is a bit easier to align than then full 3d and this is the final result of two inject two injections also for two injections per brain and two cases so you can see the injection site here this is the cortical flat map anterior posterior and superior inferior and yet so very small injection here and actually this is this shows that this is just the the way the flat map of the individual brain was warped into the common flat map space so the next step after you've got this far you want to summarize all of the connectivity from a large number of injections so this is just showing that I'm going from this kind of data this this kind of image image data you can quantify it further and look at some summaries of the connectivities within and across hemispheres so moving forward as I mentioned this was from a nanozema batch slide scanner and we were we're going to move to well it's already been in process so in the previous five years there's a number of trace injection activity so one of them was using the batch slide scanner another was using the tissue site system and now the project has consolidated the efforts into this tissue site system for superior data quality because with the system you are sectioning the brain and imaging it all together so with that you can reconstruct the a very nice some 3d reconstruction you don't have this alignment problem when you're working with the data and just as an example so now this is the tracer signal and mapped into a common space and then also to characterize different trace in different regions the cortex and some subcortical structures have been delineated so in summary I've talked about brain atlasing and atlasing and how it's been applied widely to the brain mind studies so the plan for this atlas is that we want to upgrade this we want to upgrade the atlas by using information from multiple brains as you saw before we have so to our site architecture maps for individual brains and we also have been collecting diffusion-mated imaging so all of this data can be used to improve the atlas in terms of data basing so the brain minds database infrastructure has been established it's it has been it's being used for sharing of the raw tissue site data and raw MRA data within the project but the the open database has not been developed to a sufficient degree so that's that's what we want to focus on in the next five years how can we share a lot of this valuable mama set experimental data and another thing that is quite obvious is from this is that we can pursue and interoperability with other mama set data sets data sets specifically via the atlas that we have built so if you if you're here yesterday then you would have seen Piotr's talk on the mama set brain connectivity atlas all of their data is retrograde tracer data and we're collecting anterior grade data so it it's a very complimentary data set and it'd be very nice to look look at ways to connect these two and study it so thank you for listening and please talk to me if you want to know anything more thank you very much indeed I think the marmoset now is really quite an exciting project I never saw it developed so nicely when I first time heard about that I thought it's just a curiosity of some sort but no it's really very important so few questions please yes very nice talk thank you congratulations for the project and I have a question is about the choosing injections like this is more for the like starting of the sorry could you choosing injections I was part of the cortex are you guys kind of evenly dividing cortex in the two dimension or two dimension array based on the flat map and just unbiased and inject everywhere or you choose some strategic reason to focus a little bit so in the first five years of the project there are three tracer teams one tracer team is focusing on the prefrontal cortex another on the auditory temporal region and another was trying to do some sort of general coverage and that would that was how that was working and in the next five years the plan is to have more general coverage yeah okay next question hi that was a fantastic talk thank you and I was curious because you talked about ants quite early on which is the the an elastic registration yes so because you've gone to the effort the brilliant effort of individual atlasing you've got all these anatomical components delineated have you attempted to do 3d to 3d mapping using ants because what I've always been interested how well does it register and you could actually answer that question with these coefficients and measures oh yeah so I haven't quantified it based on the data that I showed you today okay but there is very I mean you can see the variation appears more in the harder to register areas so especially for example the the mama's it brain the cortex is relatively smooth but you usually have the most trouble in the lateral fissure because in many small regions there yeah but it's actually with the mama said I found ants it works that you can normalize data but the approach you take is the more information you can use to normalize even even if that's coming from expert annotations that the better I think if you want to get something perfect you can't avoid that yeah okay thank you okay any more questions at this stage you know everything there's one thing that that that came to my attention I saw this injection in empty two injections one slightly slightly more superior and we got this very distinct difference in connectivity pattern could you do you know I can you can recall what's large are the injection sites what are a lot what's the volume of the halo and then on the ejection which leads me to the follow-up question whether it would be actually indeed possible to study studying transit connectivity with these injections so for the second part of the question yes definitely I think I think this data will be great for studying this especially with this one challenge is how how were we going to segment the the the kind of from the actual interior chaser and what we found with the AI technique is it's very good to pick up these various what faint axonal projections that you might miss if you're just trying to search through a very very huge image and for the size I don't know but I can share that with you later and there's also a publication on the sexual data set that's that's specifically focusing on this so okay cool things yeah