 Thank you, everyone, for coming here and listening. My name is Maciek Mazur. I'm principal AML engineering canonical, and I wanted to talk a little bit about ML Ops pipelines and how to actually build them to produce models that are usable at the edge, and also show you a demo of the Kubeflow platform. So basically, let me start from a recap about what is ML Ops. How many of you know ML Ops pipelines well? OK, so let me start from the beginning. Basically, whenever you are trying to use some machine learning models in production, there are a couple of steps that you need to do before actually having a ready-made model. So it all starts from data collection. There are multiple data collection engines. You can make an ETL process from databases or collect some events or any other things that you want to put into your training process. Then you do data labeling and analysis and curation, cleaning, and all the process that is necessary to make this data actually ready to be used in training a model. Because typically, data coming from external systems would have a lot of flows, a lot of missing values. Stuff will not be aligned. The keys and IDs to connect them will not be so easy to match. And after this all preprocessing is done, you can actually go to training a model. So either building something simple like regression models or training a neural network, depending on what you need and what is your use case. But in the end, after your model is trained and evaluated on a test data set, you will try to deploy it in production. But it's not the end, because whenever the model is deployed, you need to fetch some sort of information how well it's doing and train it again and again and improve it. So that's the whole process. It's called MLOps. And that's what we'll be talking about. So typically, the set of tools that you will use in your MLOps process depends on what kind of models are you trying to train and what data. So you will get your infrastructure for Jupyter notebooks, like a standard place where all the data scientists start their work, and some automated tools for exploratory data analysis. But in the end, depending on if you have pictures or video stream or audio or any other kind of data, you will set up a pipeline a little bit differently. Also the different setup is because of the speed that you want to do it in. Like there are a lot of ML models that are running, for example, in the night, assessing the production quality. And that's run typically as a batch processing to reduce costs and make things simpler. But there are a lot of models running on a real time stream of data. If you are doing smart city monitoring, which is the use case that we will look into on this demo or any other use case that requires a fast feedback loop, that's something that will be important. And together with AI Infrastructure Alliance, which we are also part of, we created the workflow templates, which are basically describing an end-to-end process that is used in order to have a proper ML ops pipeline and what are the different steps on the way. So obviously that's not enough. Like if you want to actually build it, you will need tools. And the tools and technologies are also part of the blueprint. So typically you will need some sort of way to get data from external systems, maybe enhance it with some synthetic data generators or any other tokenization methods or something to hide private and important information. Then you will have a pipeline that will go through the whole data engineering, experimentation, training, and deployment. But what's the biggest issue and what's the real problem that we are trying to solve in Canonical is the fact that you need tons of different open source projects connected together in order to make this pipeline happen. Because if you want to start fetching data from external systems, you will go with an ETL tool, Apache NIFI, for example, and Spark to process it. Obviously you have data sets that differ every day. There are some changes. So we would like to version control it, just like you version control your code in Git repo. There are tools like DVC to do version control. And then you use different frameworks, TensorFlow, PyTorch, surf models with Kserf. So all this setup is pretty complex. And what's even more tricky is different, depending on the use case. So what we are using to make this simpler is a thing called charms. And this is a construct of like Python and Kubernetes operators, which we use to deploy different sets of tools like that. So the main one that is a baseline of every ML Ops deployment that we are doing is Kubeflow. It's an open source project, which was, by the way, yesterday contributed to a CNCF. It's now an incubation project. So we hope that the user base will grow and we are inviting each and every one of you to participate. And in Kubeflow, the main part of it is Kubeflow Pipelines. This is something coming out of Google, where they develop the pipelines for their own ML platform and open source it. So that's pretty nice because Kubeflow Pipelines are actually making a step-by-step process that I showed on the graph previously in a way that each and every step is packaged as a container. And you can define your input and outputs and run the pipelines, but also restart it from any step in the process, go and look what happened on each one, particular step, and also make complex pipelines where you are doing different triggers, depending on what you are trying to build and show. And what's also really nice is that Kubeflow Pipelines are integrated with all the other parts of the ecosystem, so you can run them from a Jupyter notebook like we will do today. You can hook it up to your favorite IDE, like VS Code, Dataspal, or whatever you are using. And also with Trans with External Tools, like Cateep for Hyperparameter Tuning, you can use MLflow if you want to have a proper model registry and also compare different models and trans. And all the other favorite ML tools that you might be using. It's also pretty good in terms of connecting to a serving pipeline. You can use Seldon or K-Serve or any other serving framework. And with Jujube and Champs, you can actually deploy different Jujube bundles depending on where you are deploying. So in your main MLObs training environment, you can deploy a full set of tools, like with Kubeflow Pipelines, MLflow, and all your data collection. But if you have a serving environment, which is a different case cluster, you can deploy just like Seldon, Istio, and Feast, for example, for online feature store, and no other components. So you minimize the footprint and can have like a configurable deployment depending on where you are doing it. So the use case that we will look today would be license plate recognition. It's just like a sample to show you how the pipeline looks like. We'll not be focused on making like the best model to do it. We'll actually just download the Object Detection Model from TensorFlow Model Zoo. But basically the whole LPR works in a way that you are trying to detect a place on the picture where the license plate is, and just OCR it and have the place as plate numbers. So what's nice, if you want to actually serve this models at the edge, it's something that was discussed in one of the first talks this morning, is TensorFlow and TensorFlow Lite. So you actually have a project called TensorFlow Model Maker, which allows you to export TensorFlow models in different formats. And the Lite one has an option to export a model in a format that is then digestible by iOS or Android devices, Raspberry Pis, and so on. And if you want something a little bit bigger, you are using like your standard TensorFlow model and put it on the edge. So yeah, let's now take a look at the demo. So this is Kubeflow, like a standard instance that is deployed there. And obviously what Kubeflow has built in is like an ability to start and use notebooks. And this is one of the notebooks started on that server. So if you look at the standard way of working of a data scientist, I would try to build basically first a notebook and try on like a single instance to fetch a data set, try to make a model, make it better, do some OCR and so on. But in order to actually make it repeatable and make the training something that I can do all the time, I would build pipeline. So here is the sample code to create a pipeline, which has several different tasks, which like downloads a data set, does a data cleaning, feature extraction, some training and exporting the models. And whenever we run it from the notebook, we can then see our run and all the details. So the process started, it starts with like data download and cleaning. Let me go back to like a previous run just to show you how it looks like completed. So this is the like different steps. You can do many kind of dependencies and start things from one another. And also you see all the like different parameters. So whenever you do, for example, a training, it's a separate Docker container, you can get into it, you can see the logs, you can see the output of your like TensorFlow training and each and every artifact, if you go to like export of a model, you can see that it went to Minio. This is like an open source S3 storage. And each and every step on the way gives you this kind of outputs. And you can use this zip file in the next steps of your pipeline. You can either wrap it in a Docker container with seldom and host it on any Kubernetes cluster. You can wrap it in a snap and put it on your edge devices depending on what is your use case and how do you want to do that. So yeah, basically that would be all, thank you. And see you at the booth P28. There are a lot of demos around AIML there as well. So there is Talking Jellyfish from the Jami Jellyfish latest release of Ubuntu. There are a couple of bots trained to play StarCraft and you can play against them if that's what you enjoy. So yeah, you can basically message me on the social media mentioned here. Happy to talk to you later on about any kind of AIML issues or troubles that you have with Ubuntu or Kubernetes and try to make it better. Thank you. Raise your hand if anybody's got any questions. I'm not an expert on this at all, Machik, but you mentioned that Kubeflow had been accepted as a project by CNCF. Can you talk a little bit and maybe slowly about your experience? Were you using that before in your work and how did it get there? So is that one of the tools that you're using and had been using before this time? Yeah, thank you for the question. So basically, Kubeflow was an open source project for quite a long time. Google open source it several years ago. However, yesterday it was contributed to CNCF as an incubation project and Canonical, the company behind Ubuntu where I work was also the contributor to Kubeflow and there is a distribution called Charmed Kubeflow which we automated and also contributed to the community so we can find it on the upstream project docs. And this is something that we are deploying in white areas of different production deployments. It's in automotive industry, in life sciences and big HPC clusters where it's training models on like hundreds of A100 and video GPUs which are connected together with all the GPU pass-through and also more complicated GPU pass-through type features like MIG and VGPUs depending on what do you want to do and in HPC setups it even works with like InfiniBand and accelerating the training with multiple GPUs across different data centers.