 I would like to welcome you all to this first edition of Talking Trends in Bioinformatics, which is going to be a virtual discussion series. And trending today obviously is the topic of AI and machine learning, which is everywhere from climate change to geopolitics or facial recognition of grizzly bears, even. But the topic is perhaps a bit more discrete on the Swiss bioinformatics side. And while we were preparing for this panel, we had a glimpse of the reason for this. Basically in the same way doctors don't really show off about using stethoscopes, bioinformaticians don't like to brag about using techniques that they've been developing and using for decades, and machine learning is one of them. So this being said, we have 30 groups at least at Sib who are using and developing machine learning techniques, be it on the statistical side or on the algorithmic development side. And so we thought it's a right time to cover this topic and to provide some concrete insights from different domains. I'm Maya Berman from Sib Communications and Scientific Events, communications manager. And I'm very happy to be co-chairing this discussion with Aitana Lebron, team lead data science at the clinical bioinformatics group. So hi everyone. So indeed over the last months we have interviewed several people at Sib who work in machine learning in various fields, from text mining, healthcare, drug design, evolution. And today we're really glad to welcome four speakers to address some of the topics that we uncovered during those discussions. So please welcome with me Alan Bridge, group leader of SwissProt, Andrew Yanovich, who among other affiliation is a senior research scientist in the precision oncology center of Schuve, also senior bioinformatics at Sib. Then Carlos Pena, who is the group leader of the computational intelligence for computational biology based at the University of Applied Science of Ivernon. And Julia Vogt is the group leader of the medical data science group at the ATH. So after briefly setting the scene around AI, machine learning, deep learning, we will focus with them on mainly two topics that contribute to fostering trust among the end users of those techniques, be it clinicians, biologists or chemists. And what we would like to do is to first discuss the importance of data annotation, structuring the data, having representative data sets and how this contributes to building high quality data sets that then generalize well. Then we would like to focus on the models themselves and discuss a bit the need or not for developing explainable models. And we will finish with some reality checks as take home messages. So just to make sure everyone is on the same page, just like to say that AI, so artificial intelligence, can be seen as the global science of attempting to mimic human behavior. So you can think of this like Alexa or self-driving cars. Machine learning is a subset of AI that describes the methods by which a machine can be trained to learn to identify, for example, patterns in data, so to be trained for specific tasks without being explicitly programmed. And so that's why for SIB, we prefer to use the term machine learning because this describes better what we do here. And machine learning can range from very simple models like linear regression or logistic regressions, but also to more complicated models such as neural networks that are found in specific fields like deep learning where you can have basically neural networks that have several layers and are hence deep. And this can be used, for example, for speech or image recognition. So basically bioinformaticians, bioinformaticians, we know this very well. Nothing is kind of magic about machine learning. What is essential is to know the data and getting the right data. And these are both key for achieving good model performance and for identifying also potential limitations of the models. So for this, in particular, in supervised learning, the data sets need to be annotated for the computer to learn a specific task. Here Alan, could you tell us a bit more about the role of biocuration for properly annotating data sets? Sure. Thanks, Actana. So machine learning approaches like deep learning, they require a significant amount of knowledge engineering, which is the creation of knowledge representations from which their neural networks can actually learn. Biocurators have, I think, really an essential role to play in that process. So they combine domain-specific expertise in biology with the expertise necessary to create these kind of machine-readable knowledge representations. And one of the foremost of these kind of representations are things like ontologies. So these are structured formal representations of domain-specific knowledge for us in biology that machines can actually interpret and reason over. And I think most people recognize the role of ontologies as being essential to making data and knowledge fair and integrating data knowledge. But they're also really key to this kind of knowledge representation for the machine intelligence community and to developing machine learning models that humans can also understand, learn from, and use to build new hypotheses. So we make extensive use of ontologies and other knowledge representations in the resources we develop, like, for instance, SwissProt. And I think this is one of the reasons that underlines their popularity and their utility for machine learning. So in summary, I'd say that machine learning, machine intelligence depends on knowledge representation, and biocurators are really essential for the generation of these knowledge representations. Thank you very much, Alan. And we will get back to the use of ontologies in machine learning models later on in the discussion. So on top of duly annotated data, we will now see that creating representative data sets is an essential piece to ensure that models generalize well. So I'll switch to our second speaker, Andrew. You work on digital pathology. In a nutshell, you take advantage of scanned tissue slides to perform image processing and predictive modeling. Can you tell us a bit more about that? Yeah, thank you. So I can use this slide as an example. We essentially go and receive tissue that's been digitally scanned. And afterwards, we'll use a deep learning classifier to identify the cells in the image. And then from that, we perform data mining. And we were able to identify this pattern where if all of the cells are facing in the same direction, it tends to correspond with a patient that has less aggressive cancer. While if the cells are highly entropic in their orientation, it tends to be associated with a patient that has a more aggressive cancer. And this is especially interesting because this is the type of feature or the type of pattern that would be very difficult for a human being to visually see and consistently quantify, which presents a really interesting opportunity for computers to step in and help out. And do you want to say something about the difference between hospitals in terms of collecting this data? Yeah, so one of the challenges is that each of the hospitals tend to have their own protocols. They'll each have their own scanner. They'll each have their own supplier of hemotoxylene and ASIN. They'll each have their own embedding procedures. And as a result, all of the data tends to be fairly heterogeneous. So even though we can train a classifier on a specific site and have it perform quite well, when we take that classifier, that model and transition it to a different site, we tend to see quite a bit of decrease in performance in that model and some fine tuning needs to take place there. So even though we in general have a really good idea of what these biomarkers look like, trying to scale them up remains to be quite a bit of a challenge. Thank you, Andrew. Yeah. So the harmonization of the generation of data itself across data providers such as hospitals clearly seems to be an important factor. Julia, can you tell us about approaches you take in your group to address the generalization issue? Yeah, sure. So we are working, for example, on a model that can improve learning and generalizing across multiple different data types. So in this case, on the slide here, you see an example of images from lung cancer patients. So this is a collaborative project with the University Hospital Basel where we have different data types like images of lung cancer patients like the CT images, the PET CT images and the corresponding tax reports. And the availability of multiple data types provides a very rich source of information and this holds promise for learning representations that generalize well across different multiple data modalities. And questions that arise in this area, for example, how multiple data types can be leveraged, what are problems in dealing with multiple different data types like images and text. And the idea is to learn over all these data types and to be able to generate, for example, missing data, for example, generate a tax report given the corresponding images. Thank you, Julia. And we'll just ask Carlos now, how do you use, how do you address the issue of generalizability of models? And in particular, if you could say something about transfer learning? Yes, hello, thank you. Yes, well transfer learning is a way of addressing some difficulties we have when we are in a context where the data is not that abundant or doesn't correspond to the context in which the models or the most of the data have been collected. Very simply said, not very simply done, transfer learning is a way to take a model that you have trained or pre-trained on a given context with the amount of data. For example, I see the slide there on data sets from the Occidental countries or Western countries or if you have specific data in a context and then you can use those models and then finish training or retrain those models on a specific context of interest. As in this case, which is a project I have in collaboration with the University of Mauritius, they are already training their models on global data but they are also collecting data from Mauritius which is very specific. The people there have a very different ethnic distribution if you want and there are several circumstances that make them quite specific. So they plan to transfer the global knowledge to adapt it to their local context and that's a way to address the over-generalization problem to a specific data set or a specific context of interest. Thank you, Carlos. Andrew, can you tell us a bit how difficult is it to obtain representative data and what other issues might be around the model generalization? So one of the, there's really two challenges here. One is we discussed before that when you go between different hospitals, you have some, we essentially call it data distribution shift. So if you have the same signal when the signal moves over, we can adjust for that using things like transfer learning. The other challenge becomes when we start to look at this idea of long tail data and it's best to think of an example of that. If I go and find a very common mutation and I'm interested in a common mutation, I can go and find patients that have that common mutation and that tends to work quite well. Now, if I start to say, well, I need this mutation and this mutation and this mutation and this mutation, are all of these going to be present or are they going to somehow interact with each other? The number of patients that have that combination starts to basically decrease exponentially. So it becomes very difficult to have a good representation of those patients at that tail end of that distribution. Julia now talking still about this question of representative data sets and acquiring them. Can you tell us a word about the importance in your eyes of national data sharing initiatives, such as the Swiss personalized health network to make models more generalizable? Yeah, sure. So I think the development, the implementation and also the validation of data infrastructures in order to make health-relevant data interoperable and shareable for research in Switzerland is of extreme importance. And the efforts, for example, from SPHM and PHRT are immensely important in order to enable research on larger cohorts of data that are gathered across multiple different hospitals, for example. Because without such infrastructure that allows for interoperability, many technical and legal hurdles may arise even within the same country. And this becomes even more challenging if they are crossing country borders. We would like to develop models that are not tailored only, for example, the Swiss population. So often we work with data from different countries. In one of my projects, we work, for example, with data from Switzerland, from Germany, from China and from Greece. And to get access to this data and to sort out all these legal aspects that can be a very long and time-consuming process. Obviously, thank you. So for those of you who are interested to further explore the topic of biases and the idea of building models that are fair in particular, don't miss, there is a dev forum organized for seed members next week. I'll put the link in the chat. And also if that's your area, the Hasler Foundation has just launched a Hasler Responsible AI program. So I'll put all the details. Thanks. So basically we've just seen how knowing your data and getting the right data from the start really use models that can generally generalize better. And so now we'd like to ask between which type of models now are appropriate. So from those white box models, like linear regressions, which are explainable, we know which features can lead to the prediction, to more black box models like deep learning, which tend to perform better, but at the cost of losing some of that explainability. And as Cati O'Neil puts it, machine learning models can become weapons of mass destruction when they are completely black boxes. And so we wonder, and in particular, in our field in bioinformatics, are there times where black box models are acceptable or even preferred and basically which models are scientists within SAB working with? So we'll start with examples where explainability seems to be indeed essential. We'll start with you. Yulyak, could you tell us a bit about some of the clinical applications that you're developing in your group where explainability is important? Yeah, sure. So if we want our models and tools that we develop to be used in practice in the clinic, then interpretability is crucial because it will be very hard to convince a physician to use a black box model where it's not transparent how decisions are taken. And one first step to explain how a model works can, for example, be achieved with the selection of a few important features with the most important features that are driving the prediction. And one example is here, what you see on this slide is a project with the Children's Hospital in Basel where we have been working on a model that can predict early the risk of a newborn to develop severe jaundice. It's an early prediction, so it can tell us how high likelihood is that a baby develops this disease in the next two days. So this is an important tool as a safeguard against too early discharge of the baby from the hospital. And in this project, we were able to select the top four most important features that achieve a very high prediction accuracy out of 45 features. And this is not only important for explaining what is happening in the model, but it's also important for developing a tool that can be used in the practice. We developed here a tool where the clinicians need to type in only these four variables instead of all 45. And it's a lot more practical because it saves time and time is scarce. So this is highly important in the clinic. So the tools that we developed need to be interpretable and also usable in practice. Thanks a lot, Hula, for this clear example where indeed in the clinic, explainability is something very important. Could you also maybe, or have you faced, basically situations where explainability is perhaps less of an issue? So yeah, we are also working, for example, with heart ultrasound images of newborn babies. And the aim is to build a model that can help detecting heart effects in newborn babies. And we would like to detect which regions of the heart might be of interest, like for example, abnormality detection. So this can help the physician to identify regions of interest faster. And this is especially important for inexperienced physicians, but it also helps experienced physicians because ultrasound on newborns has many challenges. The baby moves a lot, it often starts crying and this makes the process more difficult and also parents and physicians nervous. And to have an attention model that helps detection these regions of interest faster will be highly beneficial. And if the aim is to build such an attention model that guides the physician to a specific region of interest, and then from there, the physician takes over and looks in this region and sees if there is something special by looking at the image himself, then explainability might be of less importance. However, as soon as decisions are made by a model, then explainability or interpretability is really crucial. Thanks a lot for very well explaining this, basically this distinction between maybe segmenting or pointing to regions of interest or making predictions. Andrew, so basically you also work in the field of imaging and we've just seen such an example here. Do you also share this perspective that explainability? Sometimes in this context can be less of an issue or something not so critical. Yeah, I completely agree. Ultimately what we find is that, for example, to identify where the cells are or an image or to identify a cell as a lymphocyte tends to be quite a difficult process that deep learning is able to do, but ultimately we don't care how it does that. Once we have the location of that cell, then we can extract what we would call handcrafted features above that to have that explainability component. So we try and separate those two into some things that we really don't care how they're done. But then once we have the individual components, we really wanna know how those components are combined together so that we can provide that interpretation, because that interpretation will ultimately lead to more trust from the clinicians that are intended to use those biomarkers. Thanks, so indeed very similar also to the approach from Julia. So basically we're a bit left to wonder, I mean, is it only imaging where basically maybe those black box models are more acceptable prior to then extracting features and making an explainable model out of it? Maybe Carlos you can tell us a bit if you have also projects that involve black box models, but in fears that, I mean, different than imaging? Yes, well, I mean, at least at the very beginning in several contexts, you don't really need explanation as a first outcome. In this case, for example, we have been developing models to predict only based on genomic data, if given a phage, a bacteriophage, it's able to infect a bacterium of interest, and we developed models with the University of Lausanne and the Insel Spital for being able to predict and to pre-select phages that could be potentially active against some bacteria. So these models don't relate some kind of explanation, and what is interesting is to have a short list of phages that could be of interest, because of that we haven't really addressed the explainability part at least at this stage of the project. And in another derived project, a project which we more or less conceived after that, we are now using other kind of models based on deep learning for the prediction part, but we are also interested on being able to generate potential modifications to the genome of the phage so as optimize or improve the activity of that phage, for example, being able to infect more bacteria than originally it is, or being more stable or other kind of optimization. So at that moment, we don't really need a kind of explanation. Perhaps when we are finding or proposing new sequences, new genome modifications to these phages, that would be interested at the moment to be able to produce some kind of information of why this modification specific is of interest. Then we are also, even if we didn't need it for the projects, we are also conceiving or imagining now methods that will allow us to explain at a given level why a given phage could be active against a bacterium. So the explainability is always there. Thanks a lot Carlos for these examples outside of imaging where indeed we see how as at least as a first step, also black box model can be very useful. So we have seen earlier that data quality such as that produced by expert curation is essential for annotating data set, for example, and that they support numerous machine learning models. Alan, can you tell us also more about the role of bio curation in supporting the development also of explainable models? Yeah, sure. So the types of knowledge representations that expert curators are engaged in producing representations like ontologies or networks. Obviously they provide training data for machine learning approaches, but they can also help inform the structure of machine learning approaches and make them interpretable or at least more easy to interpret. So one way to do that is actually to use prior knowledge from these kind of networks or ontologies to inform the structures of the neural networks and approaches like deep learning. And this slide illustrates one example of that approach using the gene ontology which I talked about before. So the gene ontology is an ontology of protein functions, pathways, and subsystems, and it forms a hierarchical network. So what we can see here is the utilization of that network structure to constrain a neural network in deep learning. And this is a model called DSEL, which was actually developed by the laboratory of the group of Trey Idica. So within DSEL, we're seeing the gene ontology being used to actually constrain the structure of the neural network. And what that means is that the predictions from this neural network can actually be interpreted according to the pathways and the subsystems defined by the gene ontology. So people can understand the predictions in terms of these pathways. So knowledge representations using ontologies, they provide not only training data for machine learning, but they also provide one way to make black box machine learning approaches more interpretable or explainable. And this is sometimes called white box machine learning. Thank you very much, Alan. So we see combining, I mean, what you just said, also a bit what Carlo was saying so that we have those black box models, but at some point we need to add some interpretability, maybe directly to them, try to understand in the end why the model made that prediction. And so maybe we don't manage to make them fully white, but perhaps we can make them gray and somewhat explainable. Carlos, can you maybe then to finish this part, tell us a bit more about how, what are the efforts that maybe also your group is doing for having more of those gray box models? What we have at least in my experience, the explainability becomes more important when you are in a critical context. I mean, if at a given moment, you need really to make decisions or to assist decisions with these models and you need to improve the acceptability of those decisions, of those predictions, then you would need some kind of interpretability. So interpretability is not new. We have been working on that for several years now. And the first approach is what we call the anti-hoc approach where you build models which are interpretable by design. And that's in this slide, you can see that in the right side. And sometimes you have some limitations on that and in the case, you have really effective models like deep learning and you need to understand a little bit how they made their prediction. Then you can go to what we call the post hoc approach where you extract some kind of explanation out of these predictions. And for example, we have been working this on images and in the next slide, we will see that we have been working with detecting areas of interesting images. In this case, I put as an example the same problem that I mentioned with Mauricio, so diabetic retinopathy. And we have also been able to extract rules that explain a little bit how these features are used in a given context. And we are thinking on additional approaches that could be used to complete the explanation pipeline, going from basic explanation to some kind of augmented explanation based on actual glossary or terms from the domain. And then ontologies would be very important at that moment to use the right vocabulary. And at the end, we will need also some kind of explanatory interface and that's something to be developed in the future. I can ask for them. Thank you very much. And thank you very much also for explaining all the difference between adding the external ability from design in a sense and the hoc versus afterwards, basically a post hoc and trying to extract some explanations, it's indeed a very nice point. So with this, I'd like to finish this part on expandability. I'll pass the floor is yours, Maya. Thank you, Reitana. So we will close with the last section of our discussion, which is going to look at the actual implementation of machine learning tools with a reality check a bit. And we'll see how they are actually used by end users and these end users can be doctors, life scientists, but also by your curators and what people have to be careful about then. So Andrew, you've been collaborating with clinicians for a long time. How do you envision the way your tools are or will be used? So I think it's interesting because some of these tools are already being used in a clinical setting. For example, there's a hospital in Israel that is using a deep learning model on prostate biopsies as a second reader. So the pathologist will go through their normal work and then the algorithm will in the background go and verify that there was in fact no cancer there. And they've already been able to identify two patients that had cancer that the pathologist unfortunately missed but it was caught in time. So these diagnostic type tasks will really be the, I think the first beach front that we'll see these tools implemented in because there is the opportunity for, especially pathology for a visual confirmation by an expert so they can quickly look at that slide and say, yes, this is in fact, what the algorithm says it is. Another interesting idea is folks have already started using dermatology-based apps where there's an app on the iPhone where you can take a picture of a mole on your skin. And if it's potentially cancerous, it gets flagged and alerts you that maybe you should go and have a medical doctor take a look at it. So these diagnostic components are really starting to encroach in our daily life now. On the other hand, the prognostic work that most of our research focuses on in the context of trying to predict biomarkers and things like that, that has a much longer time span simply because it becomes very difficult to validate them without large cohorts and no one can really look at those samples and confirm whether or not, for example, that patient will respond. We need to go and let's say wait five years to see if our guess was correct that that person had a good overall recurrence. Thank you very much, Andrew. Julia, for your John Dees project, you mentioned you have developed an app that clinicians can use. Do you wanna tell us a bit about the feedback that they receive or how, if it's actually used and how they use it? Yeah, sure. So in this John Dees project, I explained earlier, so we developed an app that is available online. So this is what you see on this slide here. So we find out those four most important features. So the physicians just need to enter these four features into this tool and then they press a button, this blue button predict, which yields a prediction how likely it is for the specific baby to develop John Dees in the next two days. And this can then help to decide if the baby can be sent home early or if it's maybe better to keep this baby in the hospital a little longer and observed it a little longer and then check again if there is a high likelihood that it develops this disease. And this tool is still a prototype for research purposes only. So this is why we have this red banner everywhere. It's not a certified decision support tool yet. So this is a very long process and we are in the process of certifying it. And in the long-term, we envision even midwives being able to use this app outside at home so that they can monitor newborn babies at home and then we can hopefully reduce the risk for babies to develop this severe disease by monitoring them more closely even at home. Thank you very much, Julia, for this perspective. And finally, to finish, we mentioned the importance of end users and they can be even closer than we think. So now within CIB, Alain, can you tell us how machine learning is used in your group to support the work of biocuratorial transport? Yeah, so there is a lot of potential actually for using machine learning to support curation, to support the generation of knowledge bases and also knowledge representations. And I think we're just really beginning to explore this. So we see this as kind of a virtuous circle where curation provides high quality training data. Like for instance, corpus of annotated articles which a machine can then learn from. Some machines might learn to automatically identify relevant articles and that's called literature triage which is one bottleneck in curation. And eventually, and this is much harder, they might also learn to extract information about entities and even relations between them from the articles themselves. But that's a much more difficult problem. And these kind of problems are active areas of research by a lot of text mining groups at SIB and also outside of SIB in companies like Google, Amazon and others. So at SwissPro we actually work with experts in text mining from PubMed to implement these kinds of methods in the workflow. So we're using a deep learning framework for literature triage at the moment called Lit Suggest. And this works pretty well for us because we have these large annotated sets of literature that we can use to train the methods. Going beyond literature triage to entity recognition is a little bit tricky but this is ultimately where we'd like to go. So linking text in papers to unique identifiers and ontologies for concepts like proteins and functions and small molecules and then linking the text in the papers to knowledge resources again, linking back to knowledge and data. And that is really the first step in extracting useful concepts from text. So I think in summary, there's a lot of scope for using machine learning to accelerate knowledge capture. But this approach, it depends very much on biological knowledge. It depends on access to expert curated training data to train these methods and also linking the text back to knowledge representations again, created by human curators. So again, it's very much an interaction between human intelligence and machine intelligence. Thanks, Alain, for this nice conclusion and how indeed it's in the end always working together with machine. So to basically, we're already at the end of this short panel. So to conclude and sum up a bit, maybe what we've heard today. So we've seen very nice use cases and application of machine learning from neonatal diseases to digital pathology, biocuration, we'd have into some of the specificities and quotient to be exerted with such technologies not to be on data representativity and how to also achieve more explainability. And importantly, I think it appears clearly that good data science, good data science is really at the core of bioinformatics and the VSIB because ultimately what you see here in this cartoon is that at the center to have good data science, what you need is hacking and statistical skills which is something we have natively, I would say in bioinformaticians combined with domain experts who have the substantive expertise to basically curate and make sense of the data. And I think that's also what makes SAP very special combining really all three articles here. So to conclude, I'd like to thank also all the speakers and importantly also all the many scientists that we had the chance to discuss with while we were preparing this panel. And well, we also hope that you could enjoy this panel. We'd like to thank you for listening.