 Welcome to the afternoon session of our third day of our machine learning and medicine summer school. It's my great pleasure to introduce the first keynote speaker of the afternoon, Joachim Schulze from the German Center for Neurodegenerative Diseases, DZNE, and the University of Bonn. Joachim Schulze is by education a doctor of medicine, studied at Tübingen and then had a career station in Freiburg and at the Dana-Farber Cancer Institute in Boston, in Massachusetts and then moving to the University of Cologne as a professor of tumor immunology and then joining Bonn University and the DZNE afterwards where he has now leading roles. He is the director for systems medicine at DZNE. He's also the coordinator of the German COVID Omics Initiative. He has won a number of important awards like the Sofia Kowaljewskaya Award of the Alexander von Humboldt Foundation. He's a highly cited researcher in the Web of Science group and he's the founding director of the platform for single cell genomics and epigenomics at the University of Bonn and at the German Center for Neurodegenerative Diseases. He has published a paper this year about swarm learning in clinical AI which not only made it into nature but even onto the cover page of Nature and created a big echo in the community. We are very happy to have him here today and to learn more about your work, Joachim and about the exciting opportunities for decentralized learning in clinical AI. Welcome and the floor is yours. Thanks for having me and from the introduction you realize that I'm not a machine learning guy by training but I'm an MD with an interest in mathematics for a long time and in computation and bioinformatics and the things that we're doing since a couple of years is very clear that if we if we discover and and want to use new data spaces in medicine that we cannot even see with our you know see smell touch with the senses like genomics data that's very clear that we need machine learning and that's why we're investing into this. So you'll also see that what I show you today is the joint venture between Hewlett-Packett and the DZNEs. We were working on that together so some of the slides my colleagues were so nice to share them with me and I have some from the DZNE we use them all together. So I think that's also something that I don't have to tell this audience that in contrast to the the way where you basically have a top-down situation where you have a model and with rules and then you make predictions usually not with a lot of data you then basically use data later on to see whether whether the model is predicting the data. We see more and more in medicine and that this is actually the opposite now we just have a lot of data that we cannot even grasp the patterns now think about this how can I as a as a human being look at genomic data without basically looking at the patterns by algorithms there's no way. Now if you take an x-ray you can see still use your eyes some of us at least if you have genomic data that's out of question and so I think in in medicine particularly you have more and more the the opposite you generate a lot of data and then you have to find the patterns and then later on you basically use these patterns to predict new samples. So this is this is our basic concept of course what that means is you need a lot of data and that's already one of the bigger bottlenecks in in medicine because medicine is is fractured in many small entities as you start with private practitioners having data up to large centers but there's not so much connection at least in the areas where I'm basically working. We all know that the way it starts with machine learning is of course that you are having an institute and you have your infrastructure you have your ideas and you basically think about locally how could you basically use machine learning for example to predict a diagnosis of a patient and whenever you have been there for a long time you also know that not a single place on this earth has enough data really that we're satisfied with what we would like to see with with machine learning if you compare the medical data space that we have with some of the public data space it's just too small so very often what happens is that there's a huge bias we locally measure and analyze it but if you then try to generalize it you take one model from one center and apply to the others usually this is not very well behaving and it's very clear because you're also training in your data the bias of the location and I can tell you that this is even true for a lot of imaging data in because although we should have the same machines and should have the same settings and so on there is differences in the genomics fields where I'm more familiar with we know exactly what the bias of the different genome centers are so single single center studies have a huge bias so of course over the last I would say 10 to 15 years the solution to that was very simple if you need more data we should just take everybody's data push them somewhere centrally and then do everything there because then you have all the issues that I just talked about somewhat gone now and this is true if you then look at also at the performance usually what you have is that the central models really have a bunch higher accuracy and mainly this is due because you have just much more data and the bias from the different places are reduced and of course you can also take care of that if you have access to the whole data now for some areas that might be fine for medicine this is a this is an issue because of many many points I just give you three major ones that I see is one is cost because data transfer and data storage a particular dated application in in data spaces that are large it's not a very good and sustainable idea then there is data security issues because there are rules in the medical field that makes these kind of settings really much more difficult and since the GDPR has been updated a couple of years ago this is even more complicated so although it has good accuracy and demonstrates generic models we have these disadvantages now summarizing that for us we have looked at that particular in genomic data but also in imaging data for example here at the DZNE we have 10 sites and for example at the moment if we want to transfer NMR data from one side to the next we're basically loading them to a hard drive and put them into a FedEx and then send them over and this is not really cost effective long term as I told you more and more if you carefully read what the GDPR allows and what it doesn't allow it's getting very complicated to put things centrally when we go across enterprises or across organizations it's get gets really complicated because then our legal and financial issues that have to be taken care of in a completely new way in science we often have not dealt with that but our institutions are getting more and more aware of that so that's also an issue and then of course in outside of science if you think about companies that would like to work together it's also hard to just give away your data because all the data is value these days so the idea of course to address that has been put out a couple of years ago and that the term out there and it's a huge field is of course federated learning and we have looked into that and we also note that there is federated learning peer to peer basically but if you look at those models and those systems that could be applied to the medical domain there's one thing that still bothers us quite a bit and that's the central custodian so in other words while the data now stay at the edge and the models are trained at the edge so you don't don't share the large data anymore which also deals to some extent with data privacy the problem in most of the systems is still that there is somebody or some institutions that centrally coordinates that all and in the domain of medicine that's our experience this is still not such a good idea particularly if you think about across the world if you have central players that by definition means that there is there's the the chance of monopoly at least for parameter space in this case inside space if it's not data itself so for us the idea would have been you know yes that's a very good step forward and it deals with quite some issues but it doesn't deal with all of them so it solves some of the privacy issues although not completely because if the parameter space is not well controlled you could still basically have attempts to to reconstruct the data by the parameters and for us the most problematic point was the central custodian in those systems that are available at that time and that's why when we came together last year it's an interesting point you have a crisis like a pandemic I think that's a time for innovation we came from two sites at the DZNE we had basically pushed out a catalog before the pandemic what do we actually want to do to have across sites basically have the people to to learn together and how could we basically organize that and we had basically made bullet points what we really would like to do at the same time there was a team led by Englum Go at Hewlett Packard Enterprise they looked it from a technical point of view with a very different motivation the major motivation was that transferring data and duplicating data in central models is not sustainable just cost-wise and therefore they said like we have to keep the data somewhere but then we have the problem if we want to use data across enterprises or express organizations then all the legal stuff starts so we have to make sure that this is not happening beyond what has been done in federated and so we came up with this term swarm learning and then in principle there is no central custodian so there's not one ruling this everybody is equal and like minded there's a partnership the network reflects that also the technical solution and that's very important to us so you basically before you start doing anything you make the rules how the network should work and how every new member should basically get onto this the ownership of the data remains local of course if the network would discuss and think different than they could but that's the default so you really have this and you can also put that into a into a contract into a intelligent contract that is part of the blockchain that's part of the swarm learning principle we also solve the data protection and security locally and also make very much sure that the attempts that you can basically extract from the parameters again data reduce that even further and then then what we have already shown and seen is that since we basically the way we basically integrate the learning results is reducing the bias from that comes from each of the sites dramatically and I can allude to that later on again so how does this actually this process flows how does this work so let's pretend or let's presume that there is a group of institutions that decided to build such a swarm network they have their data and they have their processing power by the way they can be completely different it's very much scalable so you can have swarms with very different sites concerning the power of HPC or not even HPC and as well as data so that's another very big advantage to swarm can actually take care of that the first thing is that everybody registers so once you have the the the electronic contract that decides all the rules concerning the data the data quality the algorithms the responsibilities the rules the rights and so on then a node can register and of course before that and I come back to that there's some technical installations but let's presume this has been done the node registers to the swarm network and receives the model that was decided to be used and then all the sites train the model locally for a certain time window we call this the epoch and then of course once this is done one of the players within the swarm has been randomly selected to collect all them another another option is that the one node that is the first in this round would be the one that collects all the results from the other one so in this case would be the lower left here and then this node shares the merge of the trained models and the merge again has rules and these rules can be decided by the swarm beforehand for example you could take the parameters and have the mean or the median or the average this is this is something that the swarm decides beforehand once you have this optimized parameter set then you basically repeat the whole thing and then you do that as long as the swarm decides to test a certain result and of course machine learning principles that you have to divide training and testing and so on is all implemented automatically so this is this is not an issue anymore so what you basically have as a difference then is that the data and the parameters are at the edge you basically collect them only to make the best model out of all of them and this is not basically put by one single entity but the swarm basically has a principle how this is dealt with this is some pre pilots from the from the from should it pack it in house with some in-house data and what they could show in their in-house setting that the swarm basically behaved like a central model more or less now and of course that's what made them interesting like okay what about if you go into real real world scenarios and this is why we then joined and worked together on this and I come back to that before I do this yeah just to mention a couple of things again you know it has high accuracy we have actually situations where we have been better than the central model and we also know why we can discuss that later and the system itself the technical solution is really very strong in enhancing privacy and also security so this is our idea yeah so we're coming from local learning central learning has a lot of advantages not so much for medicine federated is the way forward from central this is that there we are for sure but we think you know once you have the blockchain technology combined with the swarm as part of the swarm learning and you have also the complete edge parameter and data setting then this is the next step basically and what's beyond that is very important is this privacy preserving or private or permission blockchain network it has nothing to do with blockchain that in the public domain is very often connected with bitcoin or others of these crypto currencies the blockchain principle yes but it's based on a smart contract and a ledger so that makes this very versatile and also very fast now and the other thing is that for the moment being we're seeing that as swarms that have a as a size that is certainly in the in the dozens to hundreds what we are not pursuing because we're more interested in the medical domain from institutions is that we don't see that for the end user market at the moment but that's something you might have to ask them to look back at the enterprise whether they're going in this direction as well the whole thing is dynamic you can onboard new nodes if they accept the smart contract and the ledger it cannot you cannot access the whole thing if you don't if you're not if you're not authorized and that's very nicely protected and this is already nicely productified we don't have the central custodian and we also saw that and this has been tested also by our partners and this is continuing to be done of course that it prevents insider attack from semi honest or dishonest participants because this blockchain is taking care of that in a very sophisticated way so to show you an example that we also used real world data is we were also asked you know how's the performance changing when you onboard new additional nodes and so here we basically took a situation where we had three nodes where we buy by purpose basically had not so high accuracy data at these nodes and you can see that it really took a while when we trained the swarm that there was a decent result this is the accuracy of machine learning test diagnosing leukemia and you could see basically that we reached something around 82% or so again you know a simulation basically saying like we knew that these nodes did not have the best data at hand at the other time we basically use data from hematology departments where the data is extremely good and you could see that it takes a couple of epochs but then they're stabilizing after 40 epochs and then you have very high accuracy and what happens actually if you do this in a different you have six nodes you start basically with the three low prevalence nodes and then you basically take the three additional nodes that we had tested beforehand later on you just let them start with epoch 35 and then you can basically see that you very quickly reach again the same result as if you would not have had the other ones it's actually in the end even better yeah and this scale doesn't tell you but you know it's very important for such tests diagnostic tests that you're very having a very high accuracy sensitivity and specificity and although these three nodes do not provide the best data just because we now are able to provide more data yeah the results the overall results after these 80 epochs is becoming much better than without these nodes so you can actually have a continuous training that's something we're envisioning for the future right now this is not possible in medicine because if you have a diagnostic test whether it's on a test tube and a metric measurement or whether it might be an AI algorithm that leads to a probability you have to basically freeze the the system and then apply with the regulatory or authorities and then you can use that test as it is but honestly you know what we show here already and I'm pretty sure we can show that in many other examples is that continuous learning would be much better because then you can improve and take every new new case that you discovered beforehand and even further improve the results. Short word about data security and privacy so of course this is an area which has I'm also convinced has to be looked at continuously also for swarming but also for federated or central or local most of the things are actually similarly important it doesn't matter really what your structure is and of course these the security aspects for data sets is ownership and governance the attack vectors could be easily theft or re-identification reconstruction or tracing and then there's a couple of prevention mechanisms like anonymization pseudo anonymization encryption difference privacy and this is nothing that is specific for swarm this is true for federated this is true for for central as well as local learning same for algorithms and models again we see that complementary to the setup you have the security aspect are the owners or the algorithms themselves and then the same thing again theft is an issue model inversion adversarial manipulation and then here again things like homomorphic encryption or secure multi-party computation are potential mechanisms to grant that and here we're working with several partners in the meantime one of them is sysbi in Saarbrücken where we also try to basically adapt or even improve for certain questions the security and privacy aspects this really depends on the domain so in in in medicine this might be more important in climate research this might be less important but that's something to be seen once this evolves now when you look more from the from the technology what is basically necessary you have to have a certain HP compute infrastructure we by purpose and HP is doing the same we call that just the infrastructure layer it can be scalable so you could have something like an HPC but you could also have a large workstation to start with this and for some model building even a very very good laptop can can do this just to see whether there is something in of course this we I think that can be all on premise but you could even take swarms of clouds so you know clouds of clouds if you wish so because honestly we don't think that there will be a mega cloud in the future even if there is cloud data you could think about on top of that a swarm that connects different cloud data and then you have on the one side you have your machine learning platform you know standard could be used here the ethereum blockchain and this is basically all settled by the swarm learning library which has been developed by hpe and then on top you have basically you have the data and the machine learning models this whole is containerized to make it easy to be installed so you need basically a virtualization infrastructure and then on top of that it's basically a containerized system that can be installed also the rpi is supposed to be simple so that machine learning models can be easily integrated and the hyperparameters are tunable although we did not really use that yet but there is more in than we showed so far and then you have basically the management commands to control the whole network now and again as I said there's not one central unit that does that it's basically from learning process to learning process is a different member of the swarm you can also look at that from a deployment view so you have five components and or containers one is the swarm learning basically for running the user defined machine learning algorithms one is the blockchain then there is a spire server security identity integration there's a license server and then there is swci command interface so that people can actually interact with the swarm itself so this would basically look like this there would be an organization and again it's all containerized so it's pretty simple if the license is set for the node swci node the swarm learning node with the model the spire server then you connect basically the next organization the next organization the third one they've all having the same set up and then you start basically to connecting that and then once they're all connected they have seen each other the the blockchain says yes everybody who was allowed to get in there is in then you can basically start doing the learning process that's how it works yeah and from the if you look at the the code the idea is there also that to make it very simple so to call basically the swarm there's not so much difference to be done and this is also in the nature paper which we published so this can be easily looked at also the tools can be can be tested freely and of course from our experience we're also happy to share what what we learned with the system so and since it's containerized as I said it's really something if you have set up your system and I think most of the modern cutting edge or state of the art system should be virtualized with container systems to be loaded then this is not something rocket science anymore now I would shift gears and give you some insights and auto on to the use cases because I think that makes makes the the the big difference we saw of course a lot of people that had maybe similar ideas but just ideas and of course we have also a lot of new ideas actually but we always have to show first do the ideas actually work so of course that's our domain and we could convince also our partners from Julia Packard and from the computation science departments which use cases we should choose it's very clear medicine is an interesting one because it has all these problems with data sharing that's actually inherent to to medicine just think about this if you are a patient what you don't like to see is that your data is completely freely available everywhere that's why you have a physician that you can trust that physician cannot even share the information about your health with the physician next door because that's not within the physician patient contract so if your physician wants to share information or wants to ask another physician for opinion he has to first ask you and this is since hundreds of years like that even if you go to a hospital and there's more doctors you actually might not have seen that but you signed basically that doctors can talk to each other about you and if you don't sign then they actually have to deal with you completely different but this is a privilege in medicine and what happened basically so far with central models and so on is basically neglecting what is there for hundreds of years and how it worked for hundreds of years but I think what swarm learning does is basically it does not neglect it it just uses it because it does two things it protects the privacy at the same time we learn together and that's what doctors also do they're learning about insights but not about data now so if I say like I have a case and I have this this findings based on these parameters you never talk about who is that but you basically saying I have now an idea what this this disease is and this is only based on the parameters basically that define these diseases and not on the data itself that are gathered by the patient so that's the basically the major issues and then of course comes other things that are interesting for example demographic biases so if you take a hospital in Europe and one in Asia and one in Africa and one in South America and the North America then we have biases because that's normal people live in different circumstances they have different genetic backgrounds so if we even looking at the same disease then we will have differences in the patterns that we're looking at and so these kind of things are also much more easily seen if you use a swarm so that you can really see which of the parameters are completely independent of the location where you are so these challenges exist and the print as I said the mentoring principles have existed since ancient times so you know doctors learn from each other but they're not learning about their patients they're learning about their insights and the diseases and that's basically the parameter space and the data if you want so so what could you use of course x x-rays there is CT scans NMR images so everything that has images and of course we all know about the large public datasets from Google and others where you can look for patterns you know a medical device pushing out images is nothing else we could use clinical data you know from the history of the patient also from physical exams clinical laboratory parameters and so on what's interesting for the future is certainly the continuous data like EKGs wearables and so on what we know already where machine learning works pretty nicely is in hygiene optimization so really seeing what is the best tool to be done in an operation room or an intensive care unit then there's a huge and completely new fields which is omics data and it comes from the genome the transcriptome proteome and so on and that's why it's so important because you know all the other things up there x-rays CT scans and so on particularly clinical data doctors can use their senses and basically make sense out of the patterns that they see by a patient you know you ask a couple of questions you do a couple of exams you to test and so on and then you take that pattern it's like well that's a leukemia if you now use data spaces where we can have no sense to look at and that's basically and we're trying hard to make sense to translate basically the information of such data into something that we can see um we're losing usually a lot of information then these are spaces where we really could apply machine learning and really accelerate medicine in the future oops that is the wrong so um i'm not sure whether a lot of you have done biology before just a short recap you have everybody of us has a genome these are all the genes that are in your genome is the is the genome the technology and the science that looks at this is called genomics then we have of course every cell has to somehow read the genome the first thing that is happening is a transcription and that's why we then call all the all the molecules that are transcribed from the genome are the transcripts and the and the technology is transcriptomics this is not yet done and there's a next step which then makes proteins in the cell and every cell has different kinds of proteins and that's called translation and we have a lot of experience in transcriptomics and that's a space that gives you a lot of information a lot of parameters there's an undiscovered area where we can learn a lot from for pattern recognition and then in diagnostics actually looking at different diseases while transcriptome data in contrast to many other medical data is highly standardized from the beginning it is high resolution so we have a lot of parameters in the data space it reflects the activity of cells tissues and organs so this is when a disease is changing something we will see changes in the transcriptome so that gives us a possibility to understand differences and therefore then disease specific patterns and then we can extract these very easily so there are you know we know how this transcriptome works so we can also see how the parameter space should work it is more and more used so the data become more and more available even people think are now about medical diagnostics and so with that we are basically having a data space that for the future might hold up a lot of information about diseases where we have not yet used it really now in cancer for example it's very widely used already but in other diseases there is high steep increase of generating of these data what we have done is basically we generated data sets compiled them with a lot of metadata from around the world these are from many different studies these the first we had already done before the look before the crisis before we started to work together with the packet enterprise this was about leukemia and it took several Ph.D. students amongst them Stephanie Van der Teresthal but already before her to collect them and then to ask also the people that basically generated these data back and back to get really good metadata so I think we have a rather nice data set now highly curated metadata extremely good and the parameter space is also very good there are three of them the biggest one is the second one which is called data set A2 with about 8,500 transcriptomes and the reason why there are three is because they have been prepared with three different technologies in the laboratory and there you can already see that you know over time these technologies change and then you cannot just include them what we then basically did is asked a simple question because we knew that this can in theory be answered if you take a central model that's what we had published before and had cases and controls so either you wear leukemia or you wear healthy or had a different disease and then we basically split the data in different notes along these for example studies that had been done but also other other ways of splitting them so we also wanted to see what influences the analysis so we also did different scenarios every node then had their own data that was seed siloed they were never exchanged afterwards so we pretended that each of them would have been from one single hospital but the swarm basically could take the three local learnings and integrate them and then we basically had completely independent data on a test note that could have been a fourth hospital or another institution in another country where whatever we had learned from the local ones or from the swarm would then be tested independently and so we had a couple of scenarios so here's one where you can clearly see that there is one node which had 8% of all samples in this scenario that had similar numbers of controls and cases but we also did somewhere it's really bad because you almost have no cases and that's not a very good training situation and then vice versa won't come no cases and controls and you can clearly see that you know let's take this one first if you take local training with almost no cases you can see that the algorithm that comes out is very poor now so if you then test on an independent dataset which is here the accuracy shown you have basically you know more less gambling which we would have expected but what's interesting is even if we have these ones in here the swarm basically actually uses this information still and behaves much better yeah so the swarm of course takes all that into account measures the parameters and then over the epochs comes up with better accuracy than these single nodes together what was also interesting to us is an even distribution here was much better than if you didn't have an even distribution another setting was was shown here yeah so here what we did is basically we had different clinical studies from different different countries even and we pushed basically like you know there's country one country two country three with all their biases in how they did the studies and so on and see whether the swarm would improve each of the individual studies here and this is also the case you can see that if you then test again on an independent node that the integration of the parameters by the swarm basically resulted always in better test accuracy but also sensitivity specificity and so on yeah and this is one of these signs that there of course there is bias in each of these sites but if you then integrate this the bias is basically getting reduced and that's why you get better results over over time this is shown here also for the technologies so without going into much detail what we also could show is that the data set that we compiled the a1 was done on an old way of measuring the transcriptome that was done with a device which we called a microarray of course there were different types so the second data set was microarray 2 and then the third data set was basically already with a modern way of a transcriptome generation this is by RNA sequencing so this is another it's a completely different technology in the end you get also some values that are comparable but they're not identical now so if you measure something with every one and take the same sample into RNA sec you will see differences yeah and what was really amazing is that if you take then these examples you can clearly see that when you train only on RNA sec data and then later I have a data set that has samples from from all three technologies you see that this node doesn't really behave very well while the swarm can actually deal with that and then even push out better results so this also makes us very optimistic because that's the situation that we have and will have in the future you know when you measure things in the clinics there might be new machines or new devices that brings better data but then the question is can do we have to toss everything that is out there already and the answer here is no as long as we can integrate them in a way that we can take out the bias of the technology or the version of the technology then that and swarm does that actually then we can actually even use historical data to bring better results in the end. The second use case and this is the second last and then we can start discussion I'll show you also a little bit about COVID was tuberculosis because you know it's still one of the top causes of death in the world particularly outside of a pandemic 1.4 million died in 2019 so before the pandemic from TB now this is something in western countries we don't know about so much but this is really happening plus TB a lot of people are ill with TB. TB if it's not multi-resistant is curable and preventable and the point of course is that there are certain countries in the world that have the highest burden. What is worrisome for us here in the western world is that there's more and more multi-drug resistant to be coming back so in the old days in Europe for example a TB was seen only in rural areas then it was more or less gone and it now comes back in the big cities because it's first of all due to mobility so people that come from other countries but also because of the just the sheer size of people living close together and then diseases like TB also have a chance to reappear so that's why we think it's important to not only make better treatments but also better diagnosis because that's still not good enough so we're not detecting every time and you don't want to basically do invasive lung biopsies or lung bronchoscopies all the time so again there have been people that have done data and they had some education and what we did basically is we took the data and took them from these different studies and then had them learn locally and then was also learned by swarm and you can I think you can easily see here how much better the swarm basically performs when you look at accuracy sensitivity and specificity by the way we did random permutations in the sense of that we basically distributed the samples once they were siloed to test local tests and training several different times to use or to understand what is the impact of individual samples but even then it doesn't matter the swarm is basically out performing and of course one of the major issues is that you have more samples information from more samples available we can also show that how it behaves to the central model and you can see that this is on par for accuracy it was even better and the reason was because we had some huge bias in one of the of the data sets and if you take that basically into the central model that bias is not really taken care of while you know this one data set was only at one site and since we treated the sites equally this huge bias was actually much more reduced than it could be reduced in a central model of course you could also in the central model if you know about this bias or you think you know about this bias you could actually tune the parameters for these samples but this is something that would be much more sophisticated than it was done here in this swarm learning model now the the next thing is of course you know we started this all during the pandemic and we wanted to know one simple thing you know could you by just learning together because you know from the from nature to the WHO to to many organizations the European Union and so on they all said you know we have to work together but the reality is for many things it was really difficult because sharing data was kind of still tricky during the pandemic yeah because the European Union for example said yeah you can share data but GDPR rules still completely are in place so do not screw with that and that whole held it up a lot of people particularly across nations yeah and so that was another motivation but then we said like you know can we do something with that and of course you could argue you know a COVID patient is diagnosed by an antigen test or by a PCR for the viral RNA that's true yeah but first of all both of them can be false positive or negative that is not too underestimate and the second it doesn't tell you anything about the disease itself yeah so is this a mild cause is this severe course you know is this something that we need to check and of course if you measure the response of the patient to COVID then that gives you a completely different level and so that's why we went into that and I said you know what if there would be an outbreak scenario where there is a region where there's a couple of hospitals that now have a couple of cases and could we basically use blood transcriptomics to identify those cases that was the question and of course there's a couple of other things in transcriptomes and we have published that independently I'm just mentioning it here we cannot learn a lot about the disease biology we did this by single cell sequencing and by bulk and published that in cell we we have predictive biomarkers we did that together with colleagues in key and we could even predict drug targets that might reverse what we saw in the disease on the transcriptome level these are some computation models how you basically can do drug repurposing but here we used basically the pattern that COVID induced in the patient to see whether we can use that pattern to identify patients now and so we asked a lot of colleagues across Europe from Athens from Nijmegen in the Netherlands from German hospitals that they would basically spare with us their transcriptome data from from their patients and then we had basically here in this one setting we had six different hospitals now these are really hospitals so E1 is one hospital E2 is one and so on we tried to make sure that they're giving us almost the same numbers for the beginning and you could see that there were different numbers of COVID-19 patients so E6 and E8 actually we learned in the end had none yeah so we just got controls more or less you can also imagine that they alone cannot train COVID because they don't have cases we also know a lot about their age ranges you can also see there there's an interesting note E5 there were a lot of from these COVID patients there were a lot of young yeah so this is a children's hospital now interesting situation then we just see what was expected there's more males in the COVID cases and then there's a couple of other other aspects that we also checked and then we basically looked at the training aspects and you can easily see that when you now look at the testing node there's a different node and we did also some completely different so this would be another hospital yeah it's very simple to see whatever we measured more or less the swarm learning outperforms and of course if you take all the parameters across then for sure but what's worrisome is that there are some single nodes if they train their local models like E3 then E3 node and then you test it on E7 it's basically worse than gambling yeah so again telling you that if you cannot integrate enough data and take out enough of the biases that come from individual sites this is very hard to medicine with few data whereas swarm learning really brings you already to very high numbers some of them like the AUC is in the range of what we can see with PCR tests actually and I'll skip that because that only tells us that if we use that basically on the other data set so now this time we're basically testing all the data sets you know and see who is basically able to predict the patients from another center then again the only one that can do it very well is the swarm itself but there's not a single site that could predict all other sites equally well and that's I think is a very strong sign to say like working together here using these technologies is the way forward now what we do already is we basically take that forward we build now networks where we can actually work together you know and have the central and privacy preserving design we know that it's outperforming the single nodes across all the tested diseases we're now trying to do more of that of course we have done a lot of different clinical scenarios we biased a lot for example taking nodes really only from different cities or taking only different ages now so these biases were all introduced and swarm really holds up all the time the different technologies I showed you then what I didn't show you is but we were asked to do that also with x-rays and there we used one of the largest x-rays on on on lung which is from Kegel and here again swarm outperformed clearly yeah and with that I would like to stop my presentation and we can discuss further I have a lot of people that collaborated here is important that the whole packet enterprise team and then a lot of clinicians that helped us to give samples and colleagues that helped us to understand COVID and then my group particularly Steffi and Matthias who basically did a lot of the ground work that is necessary and with that I'll think I'll stop sharing and then I'm happy to take questions thank you very much for for your talk that was excellent and very very exciting to to listen to this advance in swarm swarm learning for medical AI yeah there's virtual applause coming in for your talk thanks a lot of network members so now please raise your hand if you would like to ask a question to give network so Jan is raising her hand please Jan go first thank you very much I have to say that this field is quite new to me so I'm sorry if I missed the answer to my question when you jointly compute a function over multiple participant inputs while keeping those input private and at some point you need to merge models of each participant and there are many possibilities to do to do that in a secure way such as security party computation or secure enclave and I was wondering how you actually chop the methods so the the important part for our our efforts that we have done so far is that we never shared the or we never learned together across nodes so every node learns alone yeah a local model and only shares then the parameters out of the data space that was best performing and then you have different ways of comparing yeah so the the node that is then basically chosen randomly to say like okay I take I take now all the information in and what to do now with that yeah and for the next epoch basically and what we chose is basically we did we we did basically a very simple model all the parameters that came in we took them the median or the mean and then this information was given back for again completely independent local uh calculation um on the on the nodes themselves so there is never across two nodes or more than two nodes um joined model learning it's always local you take the same algorithm but it's always only local and like this you basically only optimize the parameter space it has nothing to do anymore with with the individual patients at each site and it has nothing to do with with I know there's other ways you can basically learn across sites on one model but that's not the case here because this this is a the model parameter that you merge exactly but only the model parameter but and and then again you use that basically as an input for your next round but again you go then local you completely siloed also for the next epoch for each of the nodes the only sign that you give as a note you said like we decided for example to go for a certain time of local optimization or local training and when you're done you just signal basically to the to the node that has been chosen to collect the information I'm done I will now send my parameters that performed best in my local setting to that note so that that can be integrated then on that particular note for this particular round of learning okay thank you you could use it different but I like it because that really means that you know during the learning process the training process you never are you are never connected with anybody else you're really doing it only on yourself so like this the chance that something goes left or right to somebody else is really minimized further yeah that's a very important point for medicine other further questions on zoom so there is a question in the chat that I will read out namely from Jada Lali question for Professor Schulze what do you think is the coolest thing for further development in this area would you go for some AI elements in the structure that's the question yeah so you know for me the coolest thing is that we actually have now a situation that when I talk to medical people that what this thing can do now and that they have basically the opportunity to work together with with others amongst each other peer to peer yeah like you know not like there's somebody ruling now there's an AI group that wants to take all the data and so on there but you say like no no no everybody in the swarm is equal now everybody has you know we agree on similar rights everybody is similarly important and there's nobody ruling this whole thing except we all together like a real swarm then they all open suddenly they all want to join yeah and before when we said like you know we want to use your data for some AI you know the often the answer was you know why do you not need our data and what is what am I getting out of this yeah and why should I give you the data so the mindset has changed completely and that's for me the coolest thing because I know I'm a physician you know I can say that I know how difficult yeah but now I you know you approach them and and they said of course yeah I want to I want to be part of this yeah and I think that's something we should foster quickly and and other people not only us should do that and say like see whether they have the same impression um yeah and for me that's also a European idea you know it's really like where we want to have everybody who brings something in has some rights and responsibilities and that there is not a central person organization country or whatever that rules us all that's something I think in Europe we shouldn't have and this is the European idea so that's for me another cool thing and that can be beyond medicine I have one comment or comment slash question so I'm coordinating the harmonization of the intensive care unit data all over Switzerland between all Swiss university hospitals and we face two big challenges in that the first one is it resolved by your work so there were years of negotiations regarding data transfer agreements legal legal permits to to exchange the data between the hospitals or to bring into a central site so this is solved by your work and I admire your work for the second point is even when we have brought the data to a central site and even if we have all the legal stuff sorted out the data is even if it were coded encoded according to the same code books still not fully consistent so we still have a major effort of data harmonization to do still the data is still not fully interoperable when it reaches the central site so my question to you is how do you prevent this from affecting like the quality of your decentralized model that there are inconsistencies at the single nodes that you cannot like detect or that maybe you have a way to detect it but how do you detect inconsistencies at local nodes in the swarm so Castle this is this is a very very important question and because I take the question a little bit the opposite around it's like where would I start in medicine and and people ask me why did you use transcriptomes that's not yet in the clinic can't you use something that is that there's thousands of data out there and I said the reason why we use transcriptomes it's it's highly standardized compared to many things that are happening so for the first showing pilot and proof of principle we needed that now comes the groundwork you know the one that you're the experiencing what is the what is the next data space where we can actually use what they're doing in the clinic because honestly is a zoo yeah medicine is a zoo there is and so many flavors of of what you think is the same you know there's two giraffes and then it's like you know just put them together and you say no is it like two different animals yeah and and yeah I think we have to train our people we are discussing right now with the the neuro imaging people because they have some very specific questions and they have systems where the standardization is very high so we're actually going the other way around to like where is the next data space that is useful and meaningful across sites usable so I'm admiring now what you're doing because although it's super important the ICU every ICU is a little bit different yeah of course and and and it's very much people driven you know if you have very good people on the ICU it's different than if not and so on and I'm pretty sure the data shows that yeah so um to your last question if I have that situation and what to do is you know makes start with with very few data with very and see what the bias is you know to learn about the bias so we this is the example we did with the transcriptome where I said like we used microwave and RNA sec where we know from you know because we're handling that for 10 years for 20 years now we know what the differences are from a biological perspective even yeah and so we quickly could see whether this form can deal with that or not because we we looked at that without machine learning we just looked at technical technical level and so like this is the difference that we know yeah gene X in this technology has five value and in this has seven and this one has two and this one has ten yeah and so we knew exactly for each of the parameters what do we expect based on the technical bias and then saw afterwards does this have an impact still or not yeah so you have to dissect the data and said like okay I have to start to learn and understand what the values mean and and and then really you know do simple stats on comparison of two sides and say okay now I see what what's how they differ and then see what that impact is on the machine learning thank you very much so it was a very inspiring talk and a very fruitful discussion here and now also thank you for for taking the time to meet the doctoral students of our network this will happen in a breakout room now in the next minute so you will so Katarina will open this breakout room we send another round of applause to you for this talk it was really enjoyable on behalf of the other p is in the network and the general audience I say goodbye now and thank you thank you thank you for having me you're welcome and we'll continue with the general program here at three p.m. century european time with a talk by petechleis