 Next up, our presenters are Machia Major and André Monteneu. So, Machia is a principal AI engineer at Canonical, and André is an AI ML product manager at Canonical. And today, they will bring us through a case study of a live science company that creates customized treatments based on DNA using cloud-native technologies like Kubernetes and Kubeflow. So please welcome Machia and André. Glad to be here. Today, I'll talk about superheroes, reminiscent of our childhood dreams of being supermen or bad men. I'll take the chance and be a hero, saving people's life. Forget about André for now. I'm Michelle, a life science researcher working on some of the most innovative projects. Genomics-driven drug discovery is a very new and promising solution as drug targets with human support are more likely to be successful. It is a new approach in the industry that uses genome data to develop targeted treatments for different diseases. With the rise of generative AI, if you think of charge-EPT, for example, I thought I could make use of artificial intelligence. However, I deeply care about my patients' data, so there is one question that I had in mind. Machia, is it safe to use highly sensitive data for machine learning initiatives? Yes, so we are doing a lot to make it secure, and today I will be myself a solutions architect because I don't really have acting capabilities. So any project using machine learning is a software project. So obviously, all the DevSecOps principle supply, we are still doing CVE patching and that kind of security. However, what is important for us as ML engineers to remember about additional things that we need to take care of. First of all, the inputs to the whole process. So any kind of data that we use to train the model, and also any data that we use to actually ping the inference endpoints needs to be checked. So we are doing a lot on monitoring for data drift or model drift and make sure that that is safe. Another area that me and my colleagues are focusing on is explainable AI. So obviously, we don't want the model to be a complete black box where we don't actually understand why a certain decision is made. We want to be able to zoom in to any decision point and make sure that it was taken in the right way. And what we will be talking about today, the third area is the MLOps. So with MLOps, machine learning operations, we are looking into setting up processes from drawing blood samples in the sampling station itself to getting the data into the company's data center with Kubernetes cluster, extending that capacity to a public cloud in a safe space and presenting the results of Michel's work in the doctor's room. So far, I know that it's safe and actually the right word is secure to use AI for on highly sensitive data. And if you want to build production grade projects, we should use MLOps, which I shortly defined it as DevOps for machine learning. But if we go back to life science industry and it's available for many others, there are a lot of compliance standards that we need to follow. And whereas MLOps does not play a direct role in following the compliance standards and requirements, I think it's an enabler to protect data privacy. Yeah, so protecting patients' privacy needs to start in the beginning whenever the data is collected, so on the sampling station. And how we are doing that, first of all, we are using a process called tokenization. It might be mistakenly taken for encryption, but it's a little bit different. In encryption, we'll encrypt the whole data set and it will be useless for a researcher like Michel. So with tokenization, we are looking inside the data set and take personal identifiable information, change them to a token based on the hardware security key, like a UB key that you might use to access your cloud accounts and make sure that this data is later on completely safe in the process. However, the tokenization process itself needs also to be tamper-proof. And for that, we are using a strict confinement features of micro-case Kubernetes distribution where the case with untrusted workloads which are running next to our tokenization process are actually wrapped in an Aparmore profile and this is a kernel-level security mechanism to make sure that even if any kind of container sandbox escape attacks occur, it's still safe. So let's look how it actually works. We have a data set of Amanda Ryan, our fictitious patient here, and whenever we run a tokenization process, we can see that data of biological information is still usable, but the name was tokenized. And here we can see the difference between strict and the normal Kubernetes cluster. One of them has all the possible permissions and the strict one has very limited access to the operating system and underlying files. I'll move a layer down the stack because we often use public clouds to drive innovation. We think that we could beat our competition faster because they have a lot of machine learning available if we just think of the marketplace. Furthermore, we have big teams of data scientists and they look for ways and means to easier collaborate. And this is where I have a new question. Is it secure this time to use the public clouds when it comes to AI? Yeah, so public clouds are a great addition to a research use case like we have here because you can spike up your capacity whenever you are training a bigger model, but for most of the work of a data scientist or ML engineer, you are focused on data cleaning, doing exploratory data analysis, and you don't really need that high capacity. So how we are expanding the local Kubernetes clusters in a safe way is using a confidential computing. Confidential computing is basically something that you create a VM on a public cloud like Microsoft Azure and you are using an image which is utilizing open enclave and open source projects to actually configure the confidential compute and underlying hardware features like Intel says GX or AMD trusted zones to make sure that not only data at rest that are on the VM are encrypted, but also any kind of data that is going through memory or the CPU in terms of processing. And it's fairly simple to set up. So you are going to your Azure portal, you are selecting a confidential VM image, any kind of sizing of that that you need for your particular process. And then you set up a Kubernetes cluster there. And this cluster can be attached as a node to your, as a worker node to your local setup. And thanks to that, whenever you go to Kubeflow and run your processes like genome sequencing, you can choose on which nodes it's actually being run. So if you run the process several times, you can see here that some of them were run on the local data center and some of them on the confidential computing instance in the cloud. And this is the way how you can utilize the benefits of hiking the capacity for the moment that you need on the public cloud without exposing this data even to a malicious actor who has brute access there on the cloud itself. Since you mentioned genome sequencing, I would like to add something. It's not just genomics-driven drag discovery that would benefit from AI. There are actually a bunch of other use cases. If you think of genome research, biological insight knowledge graph, or cryo-electro-microscopy. And this is just in life sciences industry. In many others, there are many other use cases. But one trend that I've seen is that we started adopting open source. And you already mentioned Kubeflow there, which is an open source project. And it makes me think that also the AI landscape made a shift and moves towards open source project. So is it worth investing in open source? So the entire architecture that you can see on this picture is built from completely open source components. And basically the glue to them is Kubeflow itself because Kubeflow has the pipelining feature, which allows us to perform different steps of the MLOps process. And each of them is encapsulated in the Docker container. And thanks to that, and the capabilities of the charm Kubeflow distribution, we can relate it to other projects that we need in this particular use case. So if we want to have a model registry, we are using MLflow, we can have a feature store with Feast, we can add any kind of storage capabilities or other things to extend it. And by that built the entire solution. What's really interesting is that with charms you are not only able to deploy the setup, but also perform day one and day two operations. So any kind of security patching, but also scaling it up to a public cloud or scaling it down to an edge location. Let me see if I get it correctly. Kubeflow is a cloud native application that automates machine learning workflows and it enables end users to actually adapt their use cases and make the best out of them. But now I have a challenge for you. Let's move away from the tech stack because at the end of the day, we want researchers, doctors, patients to benefit out of this work. Is it possible? Yes, so in order to give this work in front of the beneficiaries, we need to create inference endpoints. I will not move away from tech because I don't feel comfortable in any other space. And basically how we are moving it to the end users is by creating a local endpoint in the doctor's room, running the inference. And in order to do that, we are using part of the Kubeflow tooling called Seldon. And we can see here that there will be a lot of differences in way how it's deployed. On the one side, we see the big data center, tons of RAM, GPUs and like a very powerful machines. And the doctor's office which is typically just a workstation. So whenever we are trying to deploy a solution there, let's look at Seldon, for example. This is the same version, the same model and the same deployment in both of these locations. And whenever we are actually trying to trigger this inference endpoint, which Michelle would be doing during her research to improve the model, make additional checks and so on, and the doctor will be using to actually see how the particular sample is going for the patient. It seems like there is no difference. One is a little bit faster because it has more compute. But in reality, what we are using here is a thing called composable bundles. And we can see that on the one side we have deployed a very minimal set of just Seldon, Istio and Minio to have a capability to do inference on the small compute footprint. And on the other side, we are deploying a vast amount of applications and entire Q-flow and additional components that help researchers to work better. And the bonuses that both of these are the same Pythonic operators that you can use to deploy on Kubernetes clusters, Bermetal, VMs and public clouds. And this is how AI drives change in life science industry. Actually, artificial intelligence empowers innovators or modern superheroes, how I like calling them across various industries, helping researchers with project delivery, ensuring firefighters are notified about victims and ambulances can reach the victims also faster. Our mission as canonical is to use Ubuntu principles for secure ML ops in any environment, public or private cloud, hybrid or multi-cloud scenarios. I'll invite you now to forget about Michelle. I'm Andrea. And as the AML product manager, I'm very happy to say that the beauty of Open Story is that anyone can use and contribute to our products. If you'd like to make Kubeflow or microcades better, you can just go on our git and try them out, first of all, share your feedback with us. And last but not least, contribute if you have ideas on how to improve them. Together with Mochec today, we talked about secure ML ops at three layers, using data tokenization for protecting privacy, confidential computing at the cloud layer, and last but not least, using street confinement at the microcades. I would like to invite you at our booth, P15, to see the full demo and talk more about ML ops. We are not life science experts. Thank you. So if you want to build this setup and replicate it yourself, you can go to GitHub. The entire code is available there. And thank you and enjoy KubeCon.