 Hi, good afternoon everybody, it's a pleasure to be here and So I'm a computer scientist my field is medical imaging processing, but we are a pathology lab and we works we work especially with dementia and We work especially with Alzheimer's disease. So I'm going to talk today about my role in this lab So why we are interested in Alzheimer's first? because it's one of the main reasons of death in the was and it's the only disease that has increased in frequency in the last decade and Currently, there's about five million people living with Alzheimer's and it costs more than 200 billion dollars a year in health care We estimate that by 2015 there will be 40 million people with me is like about one every eight people who have Alzheimer's I will be living with Alzheimer's and it's gonna cost about 1.1 trillion dollars And Alzheimer's can affect everybody. It's just not selective and It's a progressive disease. So It's characterized by Neuron death and towel-protein tangles and These two factors they are closely correlated to the cognitive decline that we see during the disease progress And it starts silently in small regions of the brain and it spreads as the disease progresses And it goes to the limbic system and finally to the whole cortex and The big problem is Alzheimer's is that a the biology underlying this is not really well understood For instance, we don't know if those towel tangles they really cause the there are not that there you see and We know that this is begins many years before the actual symptoms can kick in so the best time for treatment starting while the disease is still silent and In our lab we are interested in tiny regions of the brain the brainstem Called local serulis dorsal rough and substancia nigra cousin 2009 We show that these regions are the first affected by Alzheimer's When diseases to silent and also some of interesting things like unlike the hippocampus, which is We change even in normal age in the volume chains of normal age. I'll see doesn't change as people age So it could be a nice biomarker And here is the brainstem for those who are not used to brain anatomy like me and This is tiny regions. They're located inside this brainstem. They are some millimeters wide So they're very tiny in red here. We have the dorsal rough and blue here is the LC local serulis and green here with this substancia nigra problem is We work with pathology, which means post-mortem tissue. We are trying to understand the disease From this kind of material But we want to be able to translate this our findings to treatment of living people in the best way to image brain I'm leaving people that is not is MRI nowadays Problem is we can't really see these regions MRI because there are so tiny that current standard MRI Acquisitions don't resolve them. So What do we Want to do you want to understand how the disease is working is progressing In a microscopic level in the Stein regions and for these we need to quantify cells We need to quantify the amount of protein that we have in these regions of the tall protein We need to quantify the amount of cell death that have these regions But it would also to be able to see these regions spot this region in MRI so that you can translate these findings in Microscopy to MRI signal so can correlate these findings with more I signal and today I'm going to talk about two projects that are working parallel in the lab and one is image registration which is structural the structural microscopic view of the brain and the second one is using a machine learning for counting the cell detecting these cells in these images and So that we can bring up more productive analyzing microscopy so that you can have a more reproducible result and The first problem which the high resolution registration high resolution here I mean It's higher than the usual registration resolution that we see for registration work studies but it's not as high as microscopy and So how how what kind of material do we have? first we have a brain that we get from patients that enrolled in one of the projects in the lab and This brains is candid why it was to you in the patient's head and have a standard clinic MRI here and Then the spain brain is extracted it goes through a series of chemical process process to be fixated Sounds like a zombie I know and then this brain is embedded in what's called the cell aging and when the blocks ready we put this in this equipment called the microtome and Over this microtome will have a camera fix it in a way that we cannot move during the slicing process So every before every microtome stroke we take a picture We call that we call block face after All the brains is like said this is like to go through a staining process with a locyan and they're in bed and they're Mounted on clothing on glass slides and they're again imaged. So we end up with three volumes the MRI the block face and the histology and The histology is also imaged in a way. We don't use a microscope here We use a Canon camera with a macro lens Which is enough just enough to give you some resolution on For instance cortical layers, but it's not as high. We don't have a such a high resolution as a microscope image and The idea that what do we want to hear is be able to align this histology to the samurai problem is During the all the chemical treatment and slicing we create a lot of artifacts first when the brain extracted from the skull the brain has a very soft tissue and it gets squashed and After fixation it shrinks and We have a lot of non-linear and linear deformations all over the brain We also have a lot of tears sherry and we lost the 3d shape when you do the slicing and Also, we don't have the same slicing plane that we have when you do the MRI accession. So we never know which Histologies lies belongs to corresponds to which MRI is lies. We cannot just do this direct comparison Moreover We have a lot of data even though this is not a microscopy data set We have a lot of data our histologies our original size the original resolution of our this histology is point zero two by point zero two Millimeters in plane and it yields about 80 gigabytes one brain has about 300 slides for hundreds lies depends on the size of the brain and it's gonna beauty it's gonna yield about 80 gigabytes of Data if I work with double precision, which is the usual type of data that registration algorithms use and if and The 80 gigabytes if I'm working raise KO if I need the color with sour case gonna be three times more So in the beginning People were trying to do this manually literally segmenting of the images using photoshop Stacking them manually trying to line then use the a software called a meter and Stretching them doing all kinds of stuff, but it wasn't really effective So my role in the lab was come up with a pipeline help automate this process and Basically, what we do is use the block face as a template for realigning the histologies creating intermediate volume and then registering the volume through the to the histology and I'm gonna talk about like Steps that we need to get this registration done First is gonna be a segmentation step. We have some pre-processing and the segmentation is Responsible for getting rid of all these Pluttered black background that it's not important to us and it's gonna call zero if you try to register those images One image to the other directly without using kind of landmarks so what I do is I I Model the problem is a gash Gaussian mixture model First I convert my GB images to the IYIQ space because we notice that this is in this color space. This is our Tissue pixels they tend to cluster to a specific region in the space so it's easier to model the problem and We use it We use Gaussian mixture model and we compute the model parameters using Expectation maximization and this way I come up with a first mask that I can refine to segment the brain region I do the refinement using active contorts And this is an example of the results that we get and those are the original images and these are the segmented images and My next step is that I for each slice and each align the histology this one down here to its Perspective block face and I am using the block face. So when it lies the brain, I lose this 3d shape If I just stack it back I'm gonna end up with a Squashed twisted brain and that's not what you want and the block face Although it's a it has been squashed after the surgery But it's you can still recover the 3d shape from this block face image And as we are careful and you don't allow the camera to move we have this Slice very well aligned it and you can use them for locating the histology so I do for each slice of the registration using a fine registration algorithm and After the registration save all my mates mazes for future use and This is an example of the registration here in pseudo color is the histology It is sort of color for just for contrast Purpose and this is the registered result. I can see I don't know if it's possible to project it But you can see that the boundaries are aligned it This checkerboard view So after I do all the this to the alignment I need to stack my images But before I do that I have to correct it for Intense human genitiy because what happens is that? Even though we are very careful during the staining process We are very careful during the image process careful with lighting conditions Staining time. We always end up with these lights that have slightly different intensities so if I just take them I went up with the stripes here and They're not good for visualization. They're not good for registration So it would try to minimize this problem as much as we can and we do that just buy Using an iterative method where I compute I pick my mirror slice as the reference and I compute the Continuous histogram of this is lies and I compute the continuous histogram for my neighbors above and below this is lies Then I match both histograms. So it's a global Procedure using a fine transform and I compute this parameters using the power optimization then I just remap the colors using the new histograms and Might I get the reference my next neighbor as the reference into this in iterative way for both halves of the volume and I went up with a volume which is Much better than the other one is not perfect. You still see some of the stripes, but it has a lot of improvement and Just our curiosity here. This is a 3d reconstruction of the intermediary intermediate volume surface and For people who've seen a brain they can't tell that this is question and Just out of curiosity. You can you can see like Cut here. This was caused by the knife when they were doing the surgery So after you do this pre-processing we have to with simple histology Which used to be point zero two by point zero two millimeters and why we do that I'd like to have my histology as high as that with the most resolution that I can have because I can see The inner brain structures that I'm interested But I have to do this with same play To reduce memory footprint. Why because most registration tools that we have nowadays. They're not ready for big data They don't work well with big data sizes and scale well, and in our case point fifty cubic millimeters was the highest resolution we could use Without running out of memory in our computer knows this this was run This was ran in nurse computers and that LBL, but it's too You will be running out of memory because this algorithm doesn't scale So I did this resampling using cubic spline and in parallel I was also pre-processing the MRI to get rid of the May artifacts which the first one is the shooting in homogeneity and for those who Have worked at MRI probably have seen this is you have this Variation of the static field in the MRI it always causes this kind of slowly varying intensity Problem which can be a problem for segmentation school segmentation for methods that rely on Intensity in contrast, so I use an algorithm called in three for correct this and here is like the regional This is the corrected and this is the field. It was computed from this volume so After that I segment the brain because again I need to get rid of all these uninteresting structures that just cause error in my registration and I also with sample my MRI why It's our one cubic millimeter volume and I Resample this to beat the histology ball Resolution it's point 15 Why I do that? I'm not getting any new information for that, right? Yeah, but if I don't do this the Registration algorithm is gonna bring down my resolution even more for the registration so I need to have my target volume with the same resolution that my histology volume and That's why I'm doing this resampling right now and Finally for the 3d registrations I Have my histology I have to center my histology to MRI And this is our manual is to a manual step in our pipeline Hopefully we'll be able to automate this in the future But what happens that when I my scanner when I do this canyon right the scanner computes for me coordinate matrix that tells me how my volume is oriented in space and But my histology doesn't have this matrix because I did it manually build my volume, so I do a Manual registration which from which I come I use free free view of the tool free view from which I can Computer and find a matrix and center both volumes Finally I use the defiomorphic registration method. It's called symmetric normalization. It comes in a toolbox called ants and It's a really good method for brain image registration The fact that it's defiomorphic is important for us because it guarantees that the mapping is a smooth so I don't get foldings in my tissue and Which I also don't get disappearing pixels because I also Guarantees me guarantees to me that I'm have I have a one-to-one Mapping I don't wanna why why is that important because I don't want I don't want to lose this small detail there, but Very subtle and I also don't want to have folding tissue because this is unreal for histology And Also any colors. It's not very usual for registration to work with colors, but in our case histology Different staining is will have different colors and they will carry different meanings to carry different information So one of the requirements are having this volume in color What I what we did is like the registration only runs on race scale So we split the original images in red green and blue channels Those volumes in the country volumes and they go through the pipeline Except that I don't need to re-run the registration. I save it all my fine matrices and my deformation fields I just reapplied into these volumes. I Combined in at the end and they have my color registration By this by the time by this time I end up with us around 70 gigabytes of data that's the size of my data set and most Medical image viewers don't handle us kind of data. They don't scale well again They most of them will try to blow everything into memory and they will crash They'll be very luggage and they don't handle our color So We needed something that we could what you could see the color and you could be able to work of This big data set is not so big for people in big data, but it's being all very big very large for medical imaging So use the parallel Which is not a medical image viewer, you know, but it's great for scaling scalable visualization So we are able to split our chain our volume in several computing nodes and This way you're able to browse through the volume Make videos screenshots zoom in zoom out zoom to the structures that we want to see and this is an example of we get parallel and These are our 3d reconstructions in comparison to our MRI and here on Black, right? This is my histology, which was down sampled to point 15 millimeters And this is my MRI and I can see that my histology still holds still carries information from the small inner brain no clay. This is the call the red no clay for instance and It's not visible on the right at all There's another 3d reconstruction done with parallel and I'm gonna show some of the videos that we made using parallel It's a kind of thing that we can do with these volumes This one is interesting because we can actually here see the in the brain structures the brain stem Sorry project projectors not really helping the biggest and here show you so this is the histology and The red and green Structures worth actually the new clay we're interested in So the second project that I'm working which is In terms of algorithm is unrelated to registration, but we are still working the same problem is Cell segmentation classification So basically basically what do we need to do we have this 3d structures and Now we need to know what's happening at the cell level and Basically, you need to quantify what's going on with the cells which cells are dying Which cells have tall tangles which cells are dying and have tall tangles because we want to answer the question Are those tangles causing cell death? This is not well known. It's as high as yet and Found this project Working I'm going to show you with images is tainted by for two markers called hospice 6 and CP 30 one stands for cell death cells that are going to ascertain cell death pathway and the other that is Staying cells that have the tangles and We need to quantify this cell segment then locate And quantifying a reliable way so again our workflow We have a brain that goes through the fixation process this time We don't use the whole brain use the brainstem that is embedded in celloid when the blocks are ready It is a slice it So we have some some of the slides going through abstaining called gellosina and other Slides that are going to be re-embedded and re-slice it and staining it for recall even the fluorescence immunohistochemistry so here they're being staining it for those markers that I mentioned and We are now right now working with the fluorescence images and Basically, what do you need to do? We have several images and the researchers. They have to go to each image Scan the whole image eyeball the whole image and count the cells Yeah, and they have to come the red ones They've green ones and the yellow ones the yellow ones are the one that have they're over they have overlapping They're dying and they have the tangles and we need to do this in order in order to find the If there is any correlation between the tangles the cell death the Alzheimer's stage we need to do this country and this is very Tidious, this is very time-consuming and also we use human tissue which gives us a lot of background We have a lot of debris like this Red structures here. We have a lot of holes like this the white region here and This is pretty different from what we see in literature if you look for methods in literature for segmentation of fluorescence usually going to see images from cell culture and animal model which have a smooth dark background and Very bright cells, so they have a lot of contrast and makes the segmentation much easier We needed to find a better way to do the segmentation. So a more flexible smarter way in this case, I'm working with images Which whose resolution is about 1.6 micrometers? Some more than magnitude more than the previous images so what solutions did we find if we work with something called dictionary learning and There is dictionary learnings well from the signal processing field you can basically represent a signal as a linear combination either as a linear combination of a basis and a vector of coefficients or the inner product of inverse of these bases and the signal and you have the coefficients Very popular basis, they're also called dictionaries of Fourier, Gabor and wavelets and Those are called analytic formulations and they are very popular because they are their Mathematical properties very well understood. They have well understood algorithms fast and simple however, sometimes they don't represent well our data and and then comes the fact that you can actually learn a dictionary or transform from the data and In this case you we use over complete dictionaries meaning that we have more vectors also called atoms Then the needed meaning that I have more vectors than the dimension of my problem dimension of my image in this case and my patch and then no longer or to normal so the idea is that I Find instead of using all the vectors in my base. I find the ones that the ones that best fit my data So the model is the the following I have my Basis D, which is my dictionary and I'm going to have a sparse Representation and I'm going to only pick the vectors. They really matters They really represent well my data and I do this by solving Optimization problem because I no longer can use linear algebra here pure basic linear algebra here because I have over the Determinate system. It's going to give me infinite results infinite Answers so I concentrate I use the optimization and I have a cost function on the vector a That usually enforces sparsity and enforces that I get the better factors here So how do you use this for diction from machine learning? This is used it in denoising this is using in painting, but how do we use this in dictionary learning? Becomes a supervised learning problem learn dictionaries from my Cells and for my break background What it is mentioned is those images. They were commonly counted by other researchers in the lab so we had this ground truth and I learned the dictionary by another optimization problem where I learned I Optimized I optimized the dictionary D and the factor a the iterative away until I meet some criteria. I don't use the a in this case. I'm just interested in the Matrix D and they train these two vectors one for background one for foreground in That and the idea is that the reconstruction that use the right dictionary the dictionary that belongs to the correct class the one that use the smallest error and So I can use this for classification and I can train as much class as much dictionaries as much class as I have and For my actual classification for class because find those images what I did was using sliding window over each pixel and For each pixel I would encode the pixel using my both dictionaries and I would have two new patches and Then I computed the error between I knew I have the I have the original image Which in this case is a patch. I let's say transform back the Encoded patch to image space and they come in a comparable for them. That's how I compute the error and My pixel belong to either background the foreground depending on the smallest error This is the mask I can build a mask using this technique and this is a mask that actually built from this image and I can do I Can post-process this mask and I can use this mask to find my cells and I Can later refine my mask and locate the cells and then classify the cells And the idea is that I'm using this technique to try to build a tool that will improve the productivity of the researchers. So it's not a 100% automated tool We want a tool that we can guide the researchers to wear the cells and not debris basically So our goal is Basically understand what's going on in this new clay having these tools helping the researchers doing studying on time and quantifying the towel and we want to be able to locate this new clay in the MRI and we need for this in if we use Clinical standard MRI you're going to need a histology and so we want to be able to also Align well the histology to the MRI and we want to be able to also use the my full resolution of histology So our goals is One of our ambitions is to be able to scale the restoration the registration in the future What else we can also Let's say side effect and can you use this? Registrated histology volumes to build histology templates high resolution histology templates which are not quite used to find around for brains and Basically any questions That's like both of the projects are ongoing projects. So There's a lot of things to be done yet. So thank you