 Let's try, I'll try to keep this talk short and try to cover the pipelines we use to generate these virtual brain models, individual virtual brain models, mainly related to the question you had in this morning, whether you can use different templates for TVB, like different partulations, different connect homes, and the answer was yes, and now we'll cover some more, go more into detail. Maybe some of you have some background in your imaging, so some of the steps we are doing won't be new to you. I'm basically showing what steps are necessary to create a connect home, the structural connect home, a functional connect home, and maybe even a little bit more. And then I'll also show some solution, some like plug-and-play solutions if you have your own data and you want to generate a data and TVB format to play with it. Okay, so I have to press the button here. So what do we actually want to do? We want to create a brain network model from structural data, right? So we scan our patient or subject, object of interest in the MRI scanner, and we get structural data, T1, T2 weighted, we get diffusion weighted images to construct these tractograms in the end. We use different kind of modalities to construct a postulation of the cortex and also the sub-cortical structures, and in the end what we do is we group these tracts, the fibers in our tractogram, we group them together to create an abstract representation of the brain, right? So we want to add up with this graph where each node indicates a brain region and the edges between them are the connection strengths, not the connections in itself. So there's a whole bunch of software out there, different pipelines, people try to automate everything and depending on what kind of data, what kind of data quality you have, you can use different tools and different methods to correct for artifacts and so on of course. So I cannot cover everything here and just showing you a bunch of different things that we are using and we also would like to recommend. So from these different pipelines out there, I want to mention three. One is the minimail pre-processing pipeline of the human connectome project, which we consider as our gold standard. They have, let's say, very, very high quality data acquired in their skinners and also very, very good pipelines adjusted specifically to these high quality data. So this is what I mean is the availability of your own data, not you don't have always the resources to scan and to use the same scanning parameters or maybe you have data acquired years ago, but you still think are important, then sometimes it's hard to adjust an existing pipeline to the data you actually have. Okay. So two other pipelines are the one from Shona and our lab, TVB empirical data pipeline and the other one from our colleagues in Marseille. Also basically pipelines to construct brain networks model and they already give you the output and TVB format, which you can actually ready to use then in your simulations. So all these scripts, basically these are either Python or bash scripts. They are available always via GitHub. Sometimes there's also a container. If all of them three, there's a container, a Docker image, which you can like Docker pull and then execute on your own computer on your own supercomputer. Especially TVB empirical data pipeline is also installed on the neuroscience gateway and also on human brain project collaboratory. I'll show in the end how to get there. If you have access by now, then you can log in and also get links and there's tutorial videos and Jupyter notebooks, which explain how to use this pipeline. So the overview of the pipeline here taken a scheme from Shona is basically we have some kind of data in the beginning and we want to clean it from artifacts, a lot of distortions, artifacts in your time series, which you don't want to include because they might mess up your analysis in the end. So one big point is artifact removal. Another big point is to generate accurate cortical and sub cortical parcelation. So you want a parcelation of your brain, where regions really correspond to functional units represented here. And in the end, you want to extract your connect home. So you have maybe this is the tractor ground from your diffusion weighted imaging and then you lay the cortical parcelation on top of it and this is what you get. This is your structural connectivity either represented here in 3D as a graph or as a adjacency matrix. You can also get a functional connectivity, which in this case, for example, here is just pairwise correlations of region time series. And also if you're interested in EEG, MEG or SEG, then there's several methods to compute lead field matrices, which gives you the mapping of any dipole in, for example, the cortical surface, how this is mapped onto the sensors on the brain, on the head scalp. This then here. But really, really necessary, like the fundamental for TVB to work is a structural connectivity. This is always then used to compare your simulated data to empirical data. But this is really fundamental. So that's why I briefly will cover some artifact removal stuff, some aspects of to get a parcelation to get a connect home. And in the end, I'll show you the pipeline from the HPP Collaboratory. So artifact cleaning, I'm sure you probably all know a lot of them. There's spatial distortions. There's inhomogeneity artifacts. And right, so I introduced these three pipelines. I want to what I want to mention here is always that there will be a short table in the left or right lower corner naming the pipeline and then having a red cross if it's not available, this particular step in the pipeline or having a green tick if this step is done. And here I have two dots for the HPP pipeline because they really take care of a lot of these artifacts in a really good way. But again, this is dependent on the data quality they have. So gradient nonlinearity distortions are basically caused that the magnetic field doesn't decay linearly. That's a huge problem. And these HPP, they're special scanners. So they have to really take care of this. And other Siemens like the usual three Tesla Siemens scanners now this shouldn't be a huge issue. But it can be corrected for another big issue is this B1 bias field. So this is a applied magnetic field, which causes these inhomogeneity drifts. So actually, white matter should always have the same intensity wherever you are in the brain. So white matter down here should be as white as up there. But as you can see there's a huge shift. It is like a shadow up here, which makes all the voxels up there lower in intensity. And you can actually there different statistical methods to compute these inhomogeneity fields like here. And then you can write for this and get a huge image, a nice image with correct intensities and all the voxels. This is basically important for segmentation. So if you apply then segmentation algorithm, which tries to classify each voxel into let's say gray matter, white matter, cortical spinal fluid, then these intensities are very, very important. Otherwise, these algorithms may fail. Okay, so B1 bias field correction, the next steps are, for example, in very important in FMI and diffusion weighted imaging are the spatial distortions caused by the interaction of the static magnetic field with the properties of the tissue, very pronounced in frontal regions and also temporal regions. So the tissue itself interacts with the magnetic field and causes these spatial distortions, which lead to these stretches or compressing in different areas. But you can estimate these distortions within the field map and then unstretch or uncompress the FMI image. Also very important in diffusion weighted imaging what another technique instead of the field maps to correct for this is you can apply, you can scan the whole sequence, the best way is to scan the whole sequence twice. One with posterior anterior phase encoding, one parameter of your FMI and diffusion weighted imaging. And the other one is anterior posterior phase encoding. And what is the result of this is basically once you have stretching and compressing in one direction, as you can see here, the frontal lobe is stretched to the front and here it's actually compressed. And out of these two images you can estimate these distortions and finally you get a corrected image over here on the right. But again this requires more resources to scan different images with different parameters all the time. So another step in artifact cleaning also very important in diffusion weighted and FMI images is motion re-alignment. So if you scan resting state or even task in FMI for a couple of minutes, subjects might move during the tasks as for example plotted here. So guess how many task blocks they are aware of this design. Each sharp peak over here is the reset or the resting block. So in each resting block the subject fell back with his head or relaxed and you suddenly see these spikes. So you have like rotations in all directions and also translations of your head. And that's why there's six time series. So you correct for them like you, yes? It doesn't show up in the next slide. Right, right. So in those pipelines which have FMI processing, yes. So the HCP pipeline does it, the Shona pipeline does it, the Timothy pipeline, the TBV rec and the third one actually doesn't process resting state of FMI. It's more concerned on the structural pre-processing. But that is usually, those are basic steps on FMI pre-processing. So you realign this image and sometimes this is not enough to account for all the artifacts in your time series. So you use motion regression, basically you use a linear model or approximation depth to use these regressors to filter out the effects of the movement in your FMI time series. And another other factor is the eddy currents basically due to this fast switching of the magnetic fields. You can induce electric currents and they themselves also cause distortions here and there is like FSL has a nice toolbox to correct for these distortions. Yeah, right here is back the table and the HCP does it. These other two pipelines don't do it yet. What I'm talking about here is ICA denoising. So your FMI signal, you can think of being composed of several sources. So there are several sources contributing to your neural signal, your FMI signal. There are neural sources which are good, which are really the ones you want to measure and there are not neural sources which are basically artifacts which you want to remove. And these artifacts could be signal originating from white matter, from motion, cardiac, respiratory pulsations in your cerebral spinal fluid in your sinuses. So there are different kind of sources. You also measure with your FMI and ICA tries to, there is a statistical technique to decompose your measured signal into all these separate sources. And in the end you get a spatial as well as a temporal pattern for each source and then you can decide, okay, judging by the spatial distribution by this temporal pattern, I declare this is neural source. This is some signal arising from neural activity in a grey matter or this is not neural. It's basically an artifact and I want to have this removed. And if this works perfectly, it in the end removes all the artifact component and you remain with the neural signal. So here are some examples. Sometimes, for example, you might find a spatial pattern here within the white matter and this is not what we want. We want activity in grey matter. Those are motion artifacts. This is cardiac artifacts here in the ventricles. So we said we had spatial patterns but we also have temporal patterns. Very fast oscillating. Rhythms are not really what we would expect in neural tissue and these smooth slow oscillations are what we want. So this is the time series here and this is the spectrogram. And based on different features like spatial and temporal features, you then classify your components into good or bad and you remove the bad ones and remain with the good ones. And there is actually an ICA fix is a classifier to do this for you. So it's a machine learning technique which is trained on a sample data set. So let's say you have a study of 50 subjects. You classify the first, let's say 20 subjects by hand. So you do the ICA on each FMI of the subject and then you classify the 100 components into good or bad. And then you give these classifications to ICA fix and he learns from your sample what's good, what's bad and does the other 30 subjects for you. There are also pre-trained classifiers for the HCP data set, for other data sets, but I would recommend or you would recommend to train your own, it's worth the effort. Okay, so next thing is then parcelations. What is the goal here? Actually we want to identify regions with the same functionality in the brain. So we want to classify a piece of the cortical sheet on the subcordial gray matter to be one functional unit which we want to model later on. And there are several ways to do this. The standard method some years ago was to define an atlas on some standard space like MNI, where you draw these lines between different regions and then you would use non-linear warping or non-linear registration to warp your subject brain onto this atlas and then there you have your postulation on the subject brain as well. However, there are several drawbacks to this. Non-linear registration in three dimensions is pretty hard and the anatomy is not always aligning well with the function and also smoothing for example in three dimensions the volume doesn't take into account the true distance or the functional distance between the brain areas because actually the cortical sheet is a two-dimensional sheet rather than really a three-dimensional object and this is why for example the Human Connecton Project does most of their processing on these cortical sheets. And for example here in the three dimensions the distance or the Nuclidean distance here between area one, area A and area B is very short rather than on a cortical surface this would be a rather long distance although if you represent your cortex in a sheet you have these advantage to unravel the true distance between areas. It's easier than spatial relationships are reserved and the file format for this is actually not more nifty which is like a volume representation of the brain nor neither gifty which is just a surface representation but it's the combination of both. It's called sifty and elements in there are called grey ordinates. So you have a surface representation and a volumetric representation of the brain in one file. The cortical sheet, left and right hemisphere are represented by around 30k or 30,000 vertices and then there's another 30,000 grey ordinates or voxels for the sub-cortical areas. And this is used by the Human Connecton Project Pipeline so they do all their registration or their parcelation on it and also fmi processing. And here maybe some examples the reason why one should take into account not only one feature of the subject for parcelation but several. For example, if we do non-linear registration we have these voxel intensities which get aligned and of the folding. Free-surfer for example does parcelation according to anatomical landmarks defined by the cortical folding as well. However, cortical folding can be very complex and varies across subjects. For example, twin brains and even there there's a huge difference in their cortical folding. Here for example microscopic studies or anatomy studies showing the extension of the one area, 17 and area 18 here in the brain for four different subjects and as you can see already the functional units don't align very well with cortical folding. So this is why the Human Connecton Project decided to follow a different approach where they use not only one feature but they use multi-modality to classify regions in the brain. So what do they use? They use myelin map, the myelin content in the cortical sheet which is calculated from T2 and T1 images. They use task as fmi activation patterns, functional connectivity and well in the first basic principle step they also use cortical folding for the first smooth alignment and then they basically compute the gradients the spatial gradients of these maps and they draw then they define areas where these gradients are very sharp meaning there's a sudden transition from one to another area and here for example is the cortical, the motor strip these are basically the four different modalities you can find there's the cortical folding and these are the names for different areas and what you basically see is that for different areas you really have different values for different features for example cortical signals are low in all of the four areas myelin is really differentiating here and cortical folding as well is higher or lower in different areas and this then the classification right in the end the four areas are separated by the different modalities and if you do this for the whole brain you come up with this cortical bar oscillation into 180 areas for each hemisphere so 180 plus 180 plus 19 subcortical areas makes up 379 regions in the brain here color coded by their respondents to different task FMI's so right I said in the beginning this is a necessary step we talked about functional MRI that this is basically used to compare then your simulated activity to something you have some empirical data to validate your simulation to but you definitely need structural connectomes and here it's just basically very short depicted how to do this or how this is done so this is not part of the HCP minimal preprocessing pipeline it's only taken care of in the other two HCP does diffusion preprocessing but they don't do tractography and connectome extraction in the end but you can easily build up on their preprocess data already so what do we measure with diffusion imaging exactly so we measure something like this signal which can be, so we measure the diffusion of water molecules and water molecules tend to diffuse along an exon more easily than through it so if you measure diffusion in a brain from different areas and from different directions different diffusion values and high diffusion in one direction is then indicating axons might flow along this direction in this particular voxel and sometimes you have crossing fibers so in one voxel there's not only an exon in one direction but another fiber which is cross it and this is depicted here so let's assume this is one fiber with its signal and this is the other one and if you add them together this is what you measure now the difficulty is now to derive some kind of to untangle them basically in the end therefore we use the spherical convolution so we have some function of a signal which we expect to be in those voxels where you only have one fiber so there's really obvious there's very directed this would be the function which we would attain if there's just one fiber in one voxel however this is our measured signal so we use this function to deconvolve the signal into something like that and this is then the probability of axons or fibers in this one voxel and here this is these ellipsoids we get we get them in every region of the brain and then we use integration so basically what you saw before in the face plane indicated by these arrows we also now have the brain starting at different positions and the direction of the steps are given by these functions we just measured in the fusion way that I mentioned and what we end up with is the traction ground these trees of different fibers and as I said we in the end use the porcelation to put this on top and generate the structural connectivity we are interested in okay so maybe this is then really quick of course this model we are using now we don't really measure any single fibers this is always a model we are trying to track these fibers with has some limitations and we try to improve them so this stepping through the brain happens probabilistic so in each voxel we like throw a dice according to according to this probability function we move in different direction in each voxel and it could happen that we end up in very different areas in the brain and some of the areas we end up with our track are implausible for example we could track fibers which end up in a ventricle or which go outside of the brain and this is then captured with this technique called anatomical constraint tractography so basically during tracking we give the algorithm information on which are the plausible tracks to keep and the other ones to discard so example if you enter grey matter don't exit it once you enter it keep the track it's good it's plausible but don't continue it outside of the grey matter this is implausible or so these are for example here to track the cortical column the cortical spinal column then keep the track as well if it leaves this area down here and so forth on some force there are different rules to keep with another drawback also from this model is that it doesn't take into account volumes so each track we are generating is basically assumed to have zero volume however this is not really the case in the brain right any every action even if it's very thin has some volume and there's only so much space in the brain so they have to share this volume with each other and here's like this filtering then applied afterwards consider this is the signal we are measuring with our scanner and then we construct a track program from this this is the from here to here through stepping through the brain and it's everything is green it's overlaid with all the different tracks we just recreated and if we would now imagine on this brain we would now perform a diffusion weighted imaging again we end up with this and as you can see the fiber density in each voxel is a lot higher than before this is actually the signal that we measured in the real plane and this would be the signal we measure in this model brain so something went wrong with the frequency some some axons are over-represented others under-represented and therefore we use different techniques like ZIFT and ZIFT2 to filter this track program and keep only the plausible tracks to make the structural connectivity more realistic in the end so the last step really before I show the pipeline on HPP is also forward modeling if you're interested in MEG or EEG there's different toolboxes out there one's mentioned here a brainstorm in M&E which use different modeling techniques for boundary models in open MEG but there are different ones as well so basically what they model is a dipole on the cortical sheet and how this creates an electromagnetic field and then can be measured through electrodes on the scalp so it creates a model for your EEG and this is what we use in TVB so for example this is our cortical surface and you see around the scalp of this person and these dots here are the locations of the EEG sensors and your software whichever you use or your model then predicts the impact each of these small dots here has on each of this on the sensors and this gives us in the end the lead field matrix it can be done for EEG as depicted here but also MEG or like electrodes stuck into the brain as you saw in the epilepsy case in the beginning and right and this is taken care of in this pipeline here it's automated with some problems actually you have to define the you have an individual cortical surface and also individual locations of your sensors but you have to define some fiducial points like left and right pre-oracle points and also nasium and to get accurate locations of your sensors right so last step ok this is summary of what we just saw before so we recommend using the XEP pipeline with limitations that you have this high quality data or you might have to adjust the pipeline to enter your needs it's very good in artifact correction FMI does surface based analysis but it doesn't do EEG modeling or forward solutions and also no connector mistraction but you could add this in the end with your own preferred software and pipelines TVB and Pivotal data and TVRecon also available installed different services here with the drawback that TVB Recon doesn't do FMRI but it does the connector mistraction and also TVB does, TVB Empirical Data Pipeline does EEG modeling in the end yes sorry they also provide a Docker image which we we're thinking about combining it maybe with TVB Empirical Data Pipeline so to merge different pipelines into one and then in the end build on top of this to create input for TVB but I mean FMI prep is basically also a Python package which interacts with all the tools it's good one always I think has to take into account how much you want to automate all of this I'll show now in the next slide that here this pipeline you can basically just with two commands you pull the Docker image and then you give it the input and it spits you out the connector and the TVB data formats but obviously if you if you want better to adjust it to some to your own data you need to provide it with some more flexibility and this is the like it's called trade-off there's a trade-off between flexibility and optimization in this case yeah so they do FMI cleaning with ICA fix they are not taking into account things like white matter regression or global signal regression I mean I didn't cover it here but I mean global signal regression also have these known drawbacks as introducing maybe spurious negative correlations it's debatable and I cannot cover all of this here yeah it's just one solution I fully agree there are many many different ways to tackle the problem okay so you have your login to HPP Collaboratory and once you logged in you can press this button up here platform and go to neuroinformatics and then not this page should appear but you should at least see this menu on the left and then click on list of tools and then several tools should be pop up should be popping up here and also toolbar where you then can search for the model of the interaction pipeline and this site should show up where is the introduction and also on how to use this pipeline there is links to a quick video tutorial on how to start this pipeline on your super computer there's an example Collaboratory like an example Jupyter notebook which shows you on how to actually start this pipeline from within the laboratory if you have access to one of their super computers if you don't then you can't but it gives you introduction on how to do this, it gives you instructions and also this you will find there and as I said it's made very very it can run very autonomous it doesn't require much you have to download the software with this command docapool from this repository and then you can either run this on your super computer here this job schedule was slurm I think which is this command srun here submits a job and on many super computers it's not docker installed but it's shifter which works with docker images without issues and to run it on your computer on your local machine you can type in this command and give it input as bits data format so you need T1, T2 and you need diffusion weighted images and this is then it will print out give you the results in TVB format and give you the structural connect home and the squadric surface the EEG forward solution which can then all be for example used in a graphical user interface this is the notebook and also linked to the video tutorial ok so this was the basic talk about pipelines