 Hi, everyone. Leveraging Kubeflow for next generation AI health care solutions. My name is Aline Spono. I work at Rosh Informatics. So there's a bit of a problem. I wanted to see it here. Yeah, I don't think this is going to work out, but let's see. I'm going to turn my head. So I work at this team called Advanced Analytics AI Platforms and Frameworks. I'm an MLOps engineering lead and technical product owner. My background comes from DevOps platform engineering. I've been recently working with MLOps practices since 2020, but I've been riding the Kubernetes wave since 2015. So what do we do here at Rosh Informatics in this team? So we're focusing on creating MLOps governance for efficient development, defining MLOps architecture reference diagrams, helping with decision making processes, how you adopt really the MLOps frameworks. So some examples of papers that were written here on the MLOps lifecycle. Some of the other objectives that we have is to discover the right set of frameworks and platforms between enable development. And also, sorry, this is really hard to read like this. Can I get some time to prepare, screen, mirroring? Can you help me? How do I mirror this? Because it directly went into this. Yeah, can you help me mirror them? Because it's impossible. It's here to display. Oh, it's just display, mirror, mirror, mirror. Yeah, I think this is a problem. Oh, there it is. Go back. Yes. Extended mirror. Mirror. There you go. Perfect. Yeah, sorry for that. OK, great. So diving right back in. Aside from discovering the tools and frameworks for efficient development, we also develop the tools, which are some proprietary components that solve specific business challenges. One example would be a set of forecasting components or libraries, some MLOps SDKs around how you can work with Kubeflow, and a quick glimpse of our MLOps lifecycle and our reference architecture for how you can actually build up a MLOps platform. So as we can see here, there are a lot of components or services into a MLOps architecture. And it seems that from the cloud-native ecosystem, you can just pick these components and try to build up an MLOps platform that works for you or serves your needs. But do we really need all these services? Do you need to have them all working at the same time in the same environment to actually do these practices? Well, let's see. So what are some of the requirements for building up a platform? Let's start with reliability. I think this is the most important one. The platform needs to be consistent and provide performance operations with really high-quality service. If the platform is also extensible, allowing loose coupling of components, basically you have a modular architecture that you can also extend or integrate with different tools. So it's important to have the platform built with well-known standards. In the Kubernetes arena, there's a lot of those. Also, scalability is important. You need to have automatic scaling of services, which have to support the higher and higher demands. And the platform also needs to be efficient, so providing low cost, the flow maintenance, being easy to use, and being well-documented. So now that we know the requirements, let's put Kubeflow to the test. So Kubeflow has the right potential. It has the right technologies and the right ecosystem. And it's really become one of the main work courses of our MLOP stack. But it doesn't come with its pain points. So we know that there's a lot of hurdles trying to maintain the manifest. The setup is not really for the faint-hearted. Updates are not without risk, and we just need to have better and better integration for profile management, new deployment patterns. I think we all know that some of these went deprecated and the manifests are just, let's say, de facto choice, but there are other options, like deploy Kubeflow. Internally, we also have an inner-source Kubeflow distribution built with HelmChart, some Argo CD applications. We also try to build it with Timoni bundles. So we just really have to have a better distribution. So the gist about Kubeflow, is it really a one-stop shop MLOPs platform? Well, let's see. The platform is dedicated to making deployments of machine learning workflows on Kubernetes simple, portable, and scalable. And it's already been used productively throughout the Roche Group for accelerating drug discovery, recommending next best actions for the patient journey, data transformation and linking, and also other finance forecasting NLP customer app use cases. It's also being used to process and forecast time series analysis on a lot of models for the supply chain optimization. And an estimated 120 machine learning predictors are used by 500 users deployed thanks to Kubeflow. So I wasn't really trained as a chem informatician, so I'm gonna try to keep this simple-minded. During the process of discovering new drug candidates, scientists start with an idea and then they try to predict the molecules on in silico models, which are just some computer simulation models that try to predict properties for molecules. There are hundreds of properties and there are potentially billions of molecules. So it's only maybe 2% of the chemical space which was analyzed, and you can imagine it's really hard to predict on the entirety of these. So what happened behind the scenes of trying to build a prediction engine? We actually had come up with an ML Ops service with a dedicated platform team and a dedicated data science team which are building pipelines to manage the lifecycle of the model. There's automatic retraining on either data or code changes. We need to serve more than 100 predictor endpoints and the whole platform needs to have low latency and to be able to scale up on peak demands. So some of the achievements are in the inference performance metrics. We were able to build this prediction engine with 100% success rate for more than one million predictions per minute. The latency is about like 25 milliseconds per molecule. And the significance of this is actually we were able to meet the demands of this modern drug discovery system using mostly open source tools. K-Native, I think everybody knows K-Native and how simple it is to deploy an inference service. There's lots of configuration options. The architecture is really good based on K-Native pod autoscaler. You can build up different scaling patterns for these predictors. As we can see here in this diagram, K-Native already has a robust architecture where you have independent scaling of the activator service and independent scaling of the revisions. So I'm running out of time, but I think we need to take into account the technical details here. Okay, going to an MLOps skateboard. So what is an MLOps skateboard? It's a highly functional implementation of the MLOps reference architecture where we just try to build up a common platform. Let's say we have this skateboard model built with some bodywork, some wheels. All it needs now is a steering wheel to drive it around. So once we adopt a MLOps skateboard model, the model production line really runs more smoothly. So some of the services or components that we built inside of our CubeFlow skateboard model are foundational components, which touch upon data analytics, data science, a lot of time series, forecasting, recommendation, computer vision, all those data science related aspects. Some data plane components like the data connectors and also control plane components, which talk to other services. An MLOps SDK talking to the CubeFlow pipelines, model serving, some notification services and so on. Here, a glimpse of one of the components that we built on the left side, we have a CLI tool deploying pipelines and on the right side there is this common components library which apparently also Pepsi is building. And yeah, we just have this snowflake loader component imported from our components library and it's much easier to reuse this across all the use cases. So who's doing what inside of this journey? I think every team has their own concerns but the game changer results really come from the continuous collaboration of these areas in the teams. Innersource CubeFlow is our new approach to develop the CubeFlow community internally at Roche with a mission to reduce the duplication of efforts and simplify the landscape. I'm not gonna go through all these points but here you can see an image of our bottom up and top down alignment with the Innersource CubeFlow steering committee feeding feature requests into the backlog and then from the community we find contributors which contribute. They become trusted committers and slowly maintainers. And here we also have like where the lines cross what we do internally and where is the open source CubeFlow community. We try to somehow simulate the working group arrangement so we have our own deployment working group and pipeline's working group. And sometimes our contributions tend to escape this line and crossover to the open source. So we've hit this brick wall. Here are some pointers for future directions which can help you navigate the maze. Continue to develop the right tools and integrations in the platform, explore potential areas for improvement and innovation in the MLOps architecture and practices and join the relevant communities to stay up to date with emerging trends. So now I really hope that together we can continue to do now what patients need next. Thank you, this was my talk.