 Welcome everybody. We'll have a talk by Alexander Savio about brain imaging in Python. Please welcome Alexander. Thank you, thank you everyone for coming. Good morning. I'm Alexander Savio. I work in the clinic called HESDA ESA in Munich. The outline of the talk is going to be first I'm going to introduce the NIPI tools for neuroimaging. I'm going to give a brief introduction to most of them. Then I'm going to explain a bit of what is functional MRI. I'm going to explain a bit of the public database that's available to do this kind of experiments and I'm going to show a bit how you can preprocess fMRI with the NIPI tools. So NIPI is the name of a community. It's a community of people that do neuroimaging and neuroscience with Python. It's also a module. There's also a module called NIPI. If you go to NIPI.org you have a nice list of the modules available for neuroimaging. So the first one I would say is NIPI Babel which allows you to open different types of neuroimaging file formats. So you have IO functions for analyze, gifty, nifty. So these are these are different types of neuroimaging files. Then you have NIPI which is a set of tools to process anatomical and functional MRI images. Then there's NIPI which is a tool that allows you to define pipelines to process neuroimages. It provides a common interface to many other tools available elsewhere. So usually people do neuroimaging through command line or MATLAB and NIPI allows you to put all these tools together. So it's very handy. Then there's NITIME which has many algorithms for time series analysis. And DIPI which is to process a diffusion imaging. So when you see these images of fiber tracts of the brain, it's with a diffusion. So this is a nice model to process this kind of data. And then you have Nelearn which is I think the newest module which is done by the keynote speaker of this morning. So it provides fast and easy methods to do statistics with neuroimaging and it uses scikit-learn. So what is functional MRI? So functional MRI is uses the bold signal which is blood oxygenation level dependent and it takes so it uses the difference in magnetization of the blood flow when you have oxygen and not oxygen in the in the blood. So this allows you to more or less measure where the activity in the brain is going on because neuron activity needs oxygen to work. So yeah, this has some problems but it works. So the idea is that you acquire one whole volume. So the idea is that you acquire one whole volume each time and you have usually you take every two seconds, you take one whole volume of the brain and the functional image is a time series of volumes of brain volumes. And so well there is an international neuroimaging data sharing initiative which has many interesting databases with different sets of patients. So you have patients with schizophrenia, Alzheimer's, autism and well they are they not all of them have fMRI but most of them and you just you can download them and start playing with this kind of images. So I'm going to show a bit of how resting state of MRI is preprocessed. So here I have a nifty file, this is the extension of a zipped nifty file. So I use a nibubble to load this image. So I use the nib load function and this is this is the image data with some headers. So the header is necessary to put the brain in the correct coordinate system. Well here's a more clean list of the header. This is the long list. You have for example the type of each voxel, the first three dimensions, the temporal dimension so this is 240 volumes of this size and here it says that each voxel has three millimeters so it's three by three by three point six and so on. So this image object has a get data function which stores the this volume from the from the image from the from the file. So this this is a NumPy array and it has the shape that we saw before in the header file. So I learned apart from the statistics part it has a very nice has some very nice plotting tools that they work very well with the with the ipython notebook. So I import the plotting submodule. I use the index image function to to pick the first volume of the of the resting resting image and I use this plot anet function just to show you the for how is the first volume of the functional image. I tried to plot the whole time series in a loop but I couldn't do it with matplotlib. I heard I could use bokeh but well I'm using a tool that I work daily with and here this is the the loop of the functional image. As you can see there's a bit of movement there's a bit so there's a bit of noise so this image before doing any statistics with it I need to to correct for motion for slice timing and smooth it so I'm going to show a little bit about about these steps. So slice timing and motion correction is the first step for for this this type of images. Usually you need to know what was the the slice order of the acquisition so for each volume the scanner acquired it acquires one slice per moment so the slices have different times and since this is a a time related data you need to you need to shift these slices in the correct order to make this this image look look better and so this is dependent on the scanner so for example you can get for this type of image the slice order is like this so it's alternating so first it does the even the odd the odd slices and then the even ones and it alternates so why to to do this motion correction I I'm using here NYPY it has in this long sub module you have the space time realign class that allows you to to do motion correction and slice timing correction so here here I I create an instance of this object of this class I give the my functional image I say the repetition time which is also a scanning parameter so this means that is two seconds for each volume. Slice time is is a is the name of a function to to say what is the slice order to acquire to acquire each slice so the so this class is going to search for the the function and apply to to the image in order to to build the slice order the slice order list to perform the correction so I run this and well in this case it takes around 10 minutes I could use also job lip to cache these the result of this this is a very nice tool to job lip is is something very used in neuroscience to to store the the steps that this that are performed in an experiment so when I so when this this process finishes I get the a result which has the same shape as before but now it's it looks better so I save it I use a NYPY save image to to save this in in this path and well I could well let so first I'm going to I'm going to check the transformation that was performed so real liner is my feet the feet class where I that I used to to correct the image and I picked the trans the transformations and for each transformation I only I'm only picking the translation part of the transformation so it's an affine transformation yes so here I'm only using the I'm going to plot the translations so I have for each dimension XYZ I have the translation that was applied to each volume of the of the resting state image so this is the the affine function the affine matrix of the first transformation okay well and let's check how it looks like now so well it's not very easy to see the differences but if you check on the top top right there you'll see that there's much less movement than before well you have you have trust me well with fmri there are many paradigms in fact in the in the public data sets I showed you before there you have you have task fmri where you you put the subject in the scanner and you ask you ask this the subject to perform different tasks and you get different signals depending on the task and you can do statistics to see what areas are activating according to the to these tasks but also but also a very well so this is another paradigm is resting state fmri which you just leave the patient inside the scanner you ask them to not think in anything and and you filter the image to only measure a low frequency signal in order to see how the brain is working in rest so with this kind of of measures you can you can do some diagnosis it's a so it's a very interesting modality to to work with I'm going to talk a bit about connectivity later so the first thing you have to do with this kind of with this kind of images you have to do a band pass filter you have to filter it with free with a low pass filter of 0.08 Hertz and a high pass frequency of 0.01 I have this function here I I think I imported it from well I copied it from NiPype which is something I'm not showing how to use here so I just copied the the code of the function to show you but too complex for for now so I apply this function to my to my realigned motion corrected functional image I save it using the two file name of a function which comes with the with the with NiPype objects so this this function is returning me a nibable object object a nifty image and I use the two file name function to to save it it's quite quite handy and now let's look how it let's see how it looks now so it's a much messier but you see you see some uh low well it's a low frequency noise there uh no not noise it's activity I threw that the I could mask this uh the brain area to to get only the the voxels that are from the brain and well this is so you can see well you cannot see something happening here but this is how how these images look like when in the in the middle of the process so um Nylern allows you to to calculate brain masks and it has a this compute EPI mask to to calculate brain mask from the from fMRI and well it does quite a a good job if you have a clean a clean image you can also so here I use um to just to check I use so the plotting Nylern and to to add the the the brain mask as edge here in the over the first volume of the resting state image so another thing that we can do is to use the anatomical image so usually when you do functional MRI you also acquire an anatomical image it's also called T1 weighted or MP rage and it it's the MRI modality that gives you the best resolution and tissue contrast so it allows you to do anatomical segmentation from the brain and you can use that information over the functional image so for example I'm showing here um um an anatomical image from the Cobra data set and um I would have to register this image to the functional image to to um to work uh with the segmentation that I do here in the functional image and there are some registration algorithms um with NIPI NIPI tools but I use other other tools from well they run in Matlab but um another step important step is is smoothing since the these images are so noisy it's important to do to smooth smooth them and well Nylern gives you a very nice function it's called a smooth image and you can um so this you give the file so this is the motion corrected uh filtered image and you give the the full width half maximum so it's the size of the the kernel to to do the smoothing in millimeters and now let's see how the smoothing the smooth image looks like so it's black and well sometimes it happens I'm going to change the contrast and now it looks a little bit better and you can see uh the the function so it's a rest the the resting state part of the functional image so connectivity analysis so connectivity analysis is when you pick this filtered smooth functional image and you select different regions usually different anatomical images you extract the signal of each of these regions and you calculate the covariance between this these regions so a covariance of the resting state data between these regions and for that you usually need a brain atlas so an atlas gives you regions in the brain that are anatomical or functional um relevant so Nylern in the data sets uh super module has for example the fetch atlas hardware dock forward function which allows you to to get a nice a nice um atlas so down there i'm showing the regions of these atlas and you can use each of these regions to to perform connectivity analysis um so Nylern is a very nice it's very well documented tool um so so uh it uses scikit learn uh to perform the covariance measure and uh here here is um here's a connectivity matrix so in each in each uh pixel of the image you have a covariance measure between the different regions every other region of the of the brain of the atlas that you chose and uh it also has a a tool to plot the the connections between the regions so this is very used to to to compare different types of patients um subjects in different uh states so there's another very interesting thing about a resting state is that it was it has been discovered that uh the brain during rest uh it has connected regions that work uh when you're resting so these are these are the most the most uh famous ones um people are still uh doing research over this this kind of um this kind of uh knowledge and um so these are different areas of the the brain that are known to work together when you are resting and you can use this for example as well to to to check how well it works in patients with Alzheimer's uh and other neurodegenerative diseases so with Nylern the composition uh you can perform um ICA which is independent component analysis um they have a it's called canonical ICA and also dict learning and you can use this to extract these resting state uh components of the functional image so yeah this didn't work but um so for so how I do it I import from Nylern the composition dict learning I have to set the number of components I want to extract from the from the resting state image so basically this well um so these are the most significant uh uh components of the of the of the whole volume of the whole resting state volume and you can also so apply this to your data and you get the components uh so here there's a very nice example from Nylern so this is uh once I I extract this these components I plot um I plot them over a template and these are the areas where this component is present so I get different um these different resting state networks so well um I I'm using nipype a lot um I'm I'm I'm building a model that's called pipes which has many of these tools that I showed you showed you they are easy to use with a big data set so you can configure pipes to run over your data set and it has it has workflows to different types of images and uh yeah that's that's that's all if you have any question you need any help with this kind of analysis please come to me I'm happy to help thank you thank you excellent do you have any questions thank you for your talk I have a couple of questions um you showed at the middle of the talk some algorithms to smooth images and stuff like that I wanted to know if there's something specific about brain images that for instance can go to scikit image or other like more generic image processing libraries and the other one is that I've heard that matlab for instance is very used to do image processing and I would like to know how does all these tools compare to commercial alternatives okay so the first so there there is some of the specifics of this type of of images regarding other norm so for example brain doing brain image with scikit learn scikit image sorry so for example for a diffusion image you need very specific models to fit to your data for functional image you could so scikit image is not really prepared for for the I mean you can do it but for example you are working with chunks for so for example to do a smoothing if you run smoothing with scikit image for example a smoothing you need to know the voxel size because I'm working in millimeters for example yeah there are some specificities that with scikit image is not it's not it wouldn't be practical and so yeah matlab is very used there there are other tools that are uh command line based from different universities the thing is that if you want to work with this I mean basically you are doing research basically and you need to publish so so if you want to publish you have to use a tool that everybody knows otherwise they are going you are going to get a lot of critics from the reviewers yeah so for example there's a very nice tool from University College London which is called statistical parametric mapping which works totally on matlab it has a GUI so it's easy to use the good thing is that for example nypipe has an interface for most of the useful things that SPM offers and yeah there's there are many is another world there's a lot of tools available to do this most of them are public yeah other questions I was wondering how medical research software development and data algorithmic development go on kind of I mean who goes first what's the relation of the I mean I guess there are different of course different professionalities and teams but how do they interact who drive the things forward how everyone can do his own thing and push it to the other how does it work does it work does it work so yeah it's a multidisciplinary field where you have physicists mathematicians you have psychologists you have medical doctors you have computer scientists so it depends on the application I would say if you want to do if you are in a lab where they do psychology tests psychologist experiments mostly is a psycho as a could be a physiologist who is leading the the ideas or psychologists medical doctors are there to to check whether your your data is saying something to them because they are the ones that know about the diseases the brain of course in the end I have to I'm a computer scientist and I have to learn a bit of brain anatomy understand a bit the diseases as well but most I would say computer scientists are the are the in the bottom of thank you for the talk actually I was wondering why in these particular fields you need so many tools so in the beginning of the talk you showed us different tools what all of them are in python that's gorgeous but why so many so yeah so m the mr scanner is a very flexible tool you can set you can configure it to run different types of measures from the brain and each of these measures they are they are one whole field of research on its own so you have diffusion you have functional you have anatomical imaging so all these fields they are so all these modalities they are they have different types of analysis processes thanks for your talk difficult to talk about computer science and neuroscience in one slot I'm wondering if you guys have any training and if there is training if you guys have developed any training specifically for scientists to get involved in this rather than computer scientists who are trying to learn the science so to speak well I have been well I tried in May I'm doing a software carbon trees and I included a night pipe a night pipe tutorial one so half a day of night pipe and people enjoyed it so because mostly because they know the tools that are under it so the matlab tools the common line tools so I wanted to show them how to use night pipe because it's very useful for now that's it I mean I'm not paid to run a software carpentry or in fact I had to get some holidays but yeah it's something I enjoy doing if there are no more questions I have one you mentioned a necklace of connectivity of brain activity what does what do the the entries in this atlas represent exactly is it a special action that the body does so there is an atlas of brain connectivity brain activity yes if we take an entry in this atlas like it is it would be if I understand well some regions of the brain which would be active at the same moment yes so it's a yeah so it's how the different areas work together when when the so for example in connectivity in resting state connectivity you are measuring how synchronized are these regions okay and then what does that represent can you ensure that the body is doing a special movement or no it's resting so the subject is quiet in the scanner but you can you can see for example how how in Alzheimer's disease patients some areas they are not they are disconnected or the connection is different or lower yeah okay thank you thank you still one question so the last one have you had your brain scanned many times did it look familiar or it looks fine so I do have I do have it here so that is one of my brain scans so this this is me inside and outside thank you very much alexander