 So, as you already said, my name is Francesca. I work as a data scientist at the scientific IT services at ETA and Today, I'm going to talk to you about how to boost research with machine learning So why is it a good idea to apply machine learning and research? So we know that machine learning is quite powerful in the detection of the detector. For example, the record. Oh, okay. That's getting a bit louder now Okay, so like for example the recognition of objects in images or the detection of events in time series and Apparently a lot of research projects and a lot of research data sets deal with the yeah quite similar problems and What is also quite important is that? Standard statistical methods failed for some of these problems That's why researchers came to the idea to apply machine learning as a tool to actually tackle these problems And at first I would like to show you some recent applications of machine learning and research So the first one is coming from CERN, which is the high energy physics laboratory Located in Switzerland and they're producing a lot of data during collision experiments And what they basically want to do based on this data is to discover and characterize new particles and Since they have this huge amount of data They need other approaches than standard methods to actually do this detection of these particles So what they for example here did is they released a data set a part of their data For a challenge for a machine learning challenge and just made it available for machine learning researchers So that they can try to find a good solution for their problem Another example is coming more from the medical field in that case It's more about the prediction of epileptic seizures. So imagine a patient suffering from these epileptic seizures And the goal here is that their device is implanted in their head So that these devices can predict these upcoming seizures then also can counteract this seizure So that patients are not suffering anymore that much and Then the last example and I'm showing is also coming from the medical field But in that case we're dealing with image data and what the goal here is to do a recognition of this tissue a recognition between healthy tissue and cancer regions within that image and the idea is that usually medical doctors have to do that task and That algorithms can take over this. Yeah, this work and can assist the medical doctors in this prediction or a classification of this different tissues So and as we already can see from these few examples of shown you so far And we see that we can or we have two different fields of application of machine learning and research The first one is to uncover hidden patterns in the data like from for example You have a huge data set for these collision experiments And you want to get more insight into the data and what also helps a lot here is for example If you're using classic machine learning that we have interpretable models and then Yeah, I get awesome more information about the data set itself and then the second application is to do an optimization of time consuming events for example this classification of the tissue either we have cancer regions or not so that an algorithm can take over this task and Not a medical doctor has to do this in the end so after I've shown you like a Few examples of Yeah current applications in machine learning the next thing I want to do is at first I would like to show you the basic building blocks of machine learning pipelines not only in research was in general for machine learning project and Then I will show you two specific use cases two specific applications of machine learning and research The first one is about the detection of arm movements based on e.g. Signals and the second one is for the segmentation So the yeah, look is it localization of specific cells within an image And all the use cases I will show you here is based on public available data sets So it's nothing which has to do with my work. I'm doing at ETH is something I Had it as a yeah as a project in order to explore these public available data sets Okay, so now coming to you have these different building blocks of machine learning pipelines And usually it starts with a data set which was recorded during an experiment and Then based on the data we recorded we want to make some kind of prediction So for example coming back to the example from the epilepsy patients We have these time series and then we want to say whether there's an upcoming epileptic event or not And then there should be something in between which should bring us from yeah the data itself to the prediction so what I just called here black box, but of course this black box can be filled with more content and What is happening in between is first of all the pre-processing of the data which is quite important and always depending on the data you have and then The second step is always the modeling so that you train a model and algorithm which is then actually taking over the task and Luckily Python provides a lot of different toolboxes which can be applied and I just name here a few But those ones which are mostly used are for example sci-fi for the processing of data Especially for example for time series But also pandas which is quite helpful for the handling of tabular data and then for the modeling itself Psychic learn is really important for classic machine learning models and Then Keras is for the implementation of deep neural networks So now we know these different building blocks we need to yeah get from our data to our prediction but how does it actually look like when we want to implement that in Python and The implementation looks as follows At first of course we have to import the specific libraries we need And also for example helper functions like specific pre-processing we want to apply to our data and Then of course we have to load our data and here we do a split between the data itself and the observed outcome So there's something what the model should predict in the end Then at the second step the pre-processing is done to the data Which means so anything you want to do let's imagine you have different subjects and a medical experiment And you want to have a standardization between different subjects and you apply some kind of normalization Or could be anything else depending on your data And then what is always quite important that you do the split of your data in the training and validation set because you don't not Only want to train your data your algorithm. You also want to validate. That's actually a good model You got in the end so that's bringing you are giving you a good result Then the modeling part itself is then done as follows So you choose a specific model like for example in this case the logistic regression You could also add specific parameters to this logistic regression and then basically just do a fit of the chosen model was your training data set and Then the last step you generate a prediction based on your validation set and then also choose some kind of score to Yeah evaluate how good your model is. So for example in this case the accuracy So these are the different building blocks we need To come from the data to the prediction and now I would like to show you a two specific use cases The first one is to predict arm movements based on e.g. Signals. So why is it important imagine their Persons who lost or for example one arm and they want to use these artificial arms and control the movement of these artificial arms so what is important then is to get the brain activity which is connect to these arm movements and Based on the brain activity predict on and control the arm. So the movement of the arm So how do you how do these experiments look like? So these experiments are done with healthy patients of course because you need both the brain activity and the movement of the arm and At first these patients get a what it's called the e.g. Cap So the cap is 32 different electrodes which are then connected to the brain So just attach to the brain and can measure the activity of the brain activity and then But they're doing at the same time while their brain activity is measured Then they're doing these arm movements. So for example, they're grabbing something. They're lifting something and releasing something again and Then in this case so I will show you a little bit of the data But since we have a lot of time series We needed to do a lot of pre-processing to get more information out of our data before we can actually do the modeling itself So that's why I will yeah walk you through this quite yeah heavy pre-processing steps step by step So at first again, there's an figure of or a scheme of the distribution of leg throws across the sky so this is actually a view on top of the head and This little triangle is the nose and we see how these different electrodes are distributed all over the head and I will show you time recordings for a few of these channels, but not all of them because just There's just too much data to show at the same time but I will show you the recordings of these four different channels which are highlighted here in this plot and These time recordings look as follows so we have these recordings of these four different channels as a function of time and What is also in the data set is the different arm movements So we see here a recording over eight seconds We see these recordings of the eight channels and we see the different arm movements have which have been done in this case It's six different arm movements. It's not important which arm movement it is exactly it's just important that it was an arm movement happening like lifting releasing grabbing and so on and As we already can see here is that it's quite hard to tell just from the time series itself to tell whether There was like an arm movement at all going on or which arm movement for example to make a distinction between different arm movements So for that reason several processing steps have to be done and at first What is done is to to split the data into different time windows time frames The reason for that is so usually when we are talking about this activity or what this arm movements That's not only happening at one specific time. There's also something before and also something afterwards So for that reason we are looking at these windows of for example one second in this case 500 data points are always one second and All the further modifications are done to this specific frame Since we're using a sliding window and we will yeah split the data to All the different frames and so in the end we have the length of the of the time series times 500 Which is giving us the amount of windows. We are looking at them doing the modifications to So and one single Window is looking as follows. So it's just a one second out of this time series and we apply as its sliding window to the whole time series So then the first modification, which is done is to apply a low pass filter. Why is that important? So the brain is operating at specific brain rhythms and we know that above a specific threshold It can rather be Yeah seen as noise what the brain is producing and not really as something which is giving us information about the brain activity So for that reason we get rid of all the high frequency parts of the of the signal and just stick to the lower frequency parts And then as the next Modification the power of the signal is generated Which is just giving you some inside or information about the energy of the signal and is computed by squaring Just yeah every data point and then in the last step We just take the temporal average of all of these one second windows So that in the end we just for all of these windows we generated in the beginning We get just one value patch channel and then we use this data to do the model training and fitting in the end So and I've shown you this quite complex preprocessing of the data and I will follow or continue with the modeling itself So how can a classic machine learning model can look like to actually predict the arm movement based on the data We could use in the preprocessing Here is so what I use in this case is called a voting classifier, which is Provided by a scikit-learn And the nice thing here is that we combine several week classifiers in order to get a stronger one So namely in this case, I use three different classifiers a linear discriminant analysis Random force classifier and logistic regression all these classifiers are combined And then in the end This combined classifier this voting classifier is fitted trained to the training data And then there's a prediction made based on the training data set or test set You know validation set. I'm sorry Not mixing up training and validation set Okay, and so once we've done this Training of the model and then also the prediction. We of course want to know how the prediction looks like if this Classic and quite simple model is giving us a good result and the results Yeah produced by this model looks as follows. So at first we look at the observed event. So that's actually what Experiments have been observed. It's just the time points of the observed arm movements as a function of time And but ever there's a blue line and our movement actually happened and wherever there's white space There was no arm movement and now I will add their predicted events of what the model predicted to be an arm movement and That looks as follows. So wherever there's a dashed line on top of a blue line We see that the model predicted correctly that there was an arm movement Whenever there is only a dashed line the model predicted not correctly and said that there's an arm movement although there was no arm movement and Wherever there's just a straight line the model missed an event. So, of course, this is just Just a short time period within a longer Validation set but to get more impression on how good the model actually is in the end and we can also look for example at the confusion matrix and So the information you can get out of this confusion matrix is first that we have around 70 percent of events which were predicted correctly because When we are looking here at the confusion matrix We see that there are around 9,000 events which were predicted in the right way and we have around 12,000 events in total So almost 30,000 13,000 so we have around 17 percent of events which were predicted correctly and What we can also get out of this confusion matrix is that we have hardly any false alarm Because we have only 100 and certain events where the model predicted that there was an arm movement But there was actually none so as I've shown you for this first use case you could see that this classic machine learning model provides I would say a reasonable good prediction for this quite complex task of predicting arm movements just on these time series and What I've shown you here in detail But in general these classic machine learning models also can give you deeper insight into the data So for example here you could just based on the train model Given prediction on which channel is quite important for the prediction of the arm movements in which channels are not And then also what is quite important is that this model is running or just having a computation low cost So the whole training for this model was running on a single CPU and just took around 30 minutes Which is quite fast and just gives us a good result compared to other methods so That was the first use case where applied classic machine learning and Also was more looking at how we can get a like deeper insight into the data by applying these models And now in the second use case I want to focus on the automatic generation of segmentation images So for those of you who don't know what segmentation images are I will explain in a second But first of all I will show you the the raw data which looks as follows So we see these images and which are just yeah visualizations of brain slices So we see all these different cell types and structures we can observe in the brain and What researchers know want to know is? for example One specific structure so for example what we can see here and why it is just a specific part of the cell and They want to have that highlighted throughout the whole for example stack of images and Since we are not talking about like for example all these different structures within the image just about one specific It's quite hard to do that by for example computer vision algorithms because we just want to focus on this specific part of the cell and For that reason in many cases this segmentation Images for example done by hand, so it's quite time-consuming to generate these images and for that reason the the question is if there's also a way to do an automatic detection for example of these shapes within that image and Before I will show you the the kind of network which can actually take over this kind of task I will show you some some slides on the general Implementation of neural networks in Keras so that you can get an impression of how simple neural networks are implemented in Keras so what you see here on that picture is Just a quite simple feed-forward neural network with an input layer two hidden layers and an output layer and Also all the connections between those different layers since we have all-to-all connections between these layers and Basically what is done doing training is that we feed in an input into the model It's processed during the whole network and then in the end a specific output is generated like for example a classification or a regression based on the problem and data you are using and The night thing is that Keras allows to implement that quite easily or these different layers in Python so basically what you have to do to get this kind of or apply this kind of network is first you Import these different layers you want to use Saying for example the input layer or dense layer which is a layer which is giving you all these all-to-all connections Then you specify the out input Important thing here is that you also name the amount of neurons you want to use Then you have two hidden layers where you also specify the amount of neurons and then the output in the end Then you put all these different layers Which you are ordered or have ordered in a sequential way in one model and then Yeah, specify also the input and output which is used and as I said before or maybe I haven't mentioned but so the the network which can be used to Generate these augmentation images is quite complex So I will show you the general structure But I don't want to go into detail because it's just too much to show now within that shorter time But I want to give you the general intuition about what the network is doing so the network looks as follows and What we want to focus on here is that we have these two different branches So we have this downstream branch and we have an upstream branch and we have these skip connections between these different branches And so if you want to have more information about this type of model I also added the citation so there are these guys which are actually developed the model so you can also read up on this if you want So what is now important here for these different branches is that this downstream branch basically Extracts the what information so what is the shape of the cell we actually want to detect with an image and then the upstream branch More extracts the the bear information So where's that specific type of the cell or the specific region of a cell you want to extract and Then there are also these skip connections that actually gives this information from the downstream to the upstream branch and now let's imagine you define this kind of model in a Different file for example you want to load it Into your Python code And also then train it and that would look as follows so you would Yeah, import your model your unit load this unit And then basically just do a fit of the model with your training data and then also prediction based on your test data So basically that's quite similar also to all these yeah to the training of models But we've seen before for scikit-learn, so it's basically just the same. Yeah Same steps you have to do Okay, so now we train our unit now we actually want to see what kind of prediction it gives us So what is the output in the end? So at first again show you the raw image so that is what is yeah Just a normal image from our test data test data set we put into Our model and then generate the prediction and at next I show you the ground truth So actually how should it look like what would it be? If a person would color all these different regions of the cell and then at next I show you the prediction which is I would say quite a good result for an Algorithm which is detecting these different shapes and objects within an image So of course, this is just one single example, but I can say you that the whole Training or the whole network in the end reached the accuracy of 99 percent Or 98 percent close to 99 percent So it's giving you a quite a good result for for this kind of task and compare to that It's quite time-consuming for a person doing it again and again Okay, so that already brings me to my second summary for the second part So what I've shown you for the second use case is that these deep learning models provide or assist in the Automatization of time-consuming events or time-consuming processes like for example the generation of these segmentation images and What is also quite helpful with deep learning is that it can recognize patterns in complex data sets work for example this shape of the different cells or a part of the cells and Yeah, how that then can be put into the prediction What deep learning does not offer is that it's not giving you interoperability of the model So that means of course we can train the model and we can look at the model ways in the end but it's quite hard to tell which way it belongs to which prediction or for example to which feature and What is the information you can get out of your model? It's in the end. It's a black box, but it's working And and what is also important to tell is that it's Computationally quite heavy. So the training for this specific case Took around Two hours on a single GPU about for example compared to a single CPU it took around two days or two and a half days Which is quite long Okay, so that already brings me to like the wrap-up of the whole talk So what I've shown you is the different applications of machine learning and research we saw that Machine learning is quite helpful and powerful in the detection of hidden patterns in data and research data like for example the prediction of events of our movements in EEG signals and On top on that it also allows or gives us interpretable models and which allows us a further insight into into the data and then It also saw more for the deep learning part of it It gives us an automatization of time-consuming processes like for example the generation of segmentation images and With that first of all, I would like to thank my colleagues from scientific IT services at ETH So this is my group Which is focusing mostly on research informatics, but there are also other people doing consultant work kind of consultant work and high-performance computing and software development for ETH researchers and and So last but not least I would like to thank you for your attention and I will be happy to answer questions Thank You for Nutsiska any questions Thank you very much for the brilliant speech. I was interested in your opinion because I've been reading that Certain criticism have been aimed towards machine using machine learning and scientific research due to In issues with reproducibility of reproducibility with the models I would like to know your opinion on that I think it highly depends on on the use case or the the Specific application. So for example, if we are coming back to this generation of segmentation images, what you could easily do is to Yeah, save the model itself, right? So so save the model structure and also the smaller ways and then it would be quite easy to produce the same result again But of course, I can imagine that their use cases where it's quite hard to like This reproducibility for the research or the application of machine learning and research But do you have a specific example? Okay Badios Hi, thank you for a presentation a short question. Have you used for this deep learning also Keras? It just I used Keras. Yes the whole and does it support GPU learning to Say it again GPU. Yeah, it supports. So basically what it's quite nice about Keras is that So you can run the same code and a machine with CPU or GPU and it directly that chooses the right computational Back end I would say so so if you run it on a GPU machine it uses a GPU and it's yeah The training is a lot faster than and one more question about this image image segmentation Have you used color images to or just black and white in this case? I just used black and white. Yeah, okay. Thank you Anybody else How do you decide which method to use like for example in the image segmentation? Why did you choose a new neural network and why did you choose the you? You net and so for this specific case I use the unit because I was reading up a lot about it and it's yeah I would say why the state of the art method to use at the moment for the generation of these specific segmentation images but in general I would say if there's a Simpler algorithm or simpler model I could use I always start with the simpler one And then if it's not giving a good result I go to the next more complicated one Thank you Thank you for the presentation for the pre-processing the tasks are very domain specific Yes, so how do you know how do you find out what you need to do to prepare the data? Given that you are maybe data scientists, you know, yeah knowledge So for this specific use case basically I would just was reading up in the textbooks And what is the way to do or yeah the way to go for the processing of this e g signals and I Think if you're applying machine learning to your own research mostly you know what to do to your data If you're someone coming from a different field and you have to look into what other the state of the art methods to do a Processing of the data would It's not only the machine learning knowledge. It's also you need to acquire the knowledge. Yeah, that's very important Okay, so that's the time. Okay, so thank you again