 Hey, what's up everyone? Welcome to AI Days. My name is Salman Iqbal, Kubernetes trainer with LearnK8. I also work as an MLops engineer for Appvia. At Appvia, we work with a number of organizations in helping them with their cloud native and MLops journey. With my friend Lewis Denham Perry, I also run cloud native Wales Meetup. Here in Wales, we talk everything cloud native. You can find me on Twitter at Salman Iqbal. Also, you can find me on YouTube. If you search for Salman Iqbal, you'll come across my channel. I explained some Kubernetes concepts in a short few minutes. It'll mean a lot to me if you can check it out, like and subscribe. Wouldn't it be great if you could write your machine learning or data science code in JupyterLab notebook and magically turn it into a pipeline? I work with a number of machine learning engineers and data scientists who think this will be a great idea. So in this video, we're going to be talking about Kale, the Kubeflow automated pipeline engine that magically turns your code in JupyterLab notebooks into pipelines. But before we dive into it, let's take a look at some of the problems that data scientists and machine learning engineers face day to day and how this helps them. As a data scientist or an ML engineer, we would think that we only have to deal with a few things. For example, perhaps we only worry about gathering training data and splitting it up into training and test data. And perhaps we only care about writing our model and training our model and then evaluating what we got. But the reality is far from it. There's tons of other things that as data scientists or an ML engineer, we have to deal with. For example, we have to actually figure out how to collect this data. Is it coming from an API? How do I pull this? Also worry about feature extraction. Verify if the data is actually correct. And do a lot of configuration. Also, I need to be able to run this somewhere. So perhaps I need to figure out where my machines are where I can run this code. Once the model is trained, I need to be able to serve this model so people can send requests against it. I need to monitor what's actually happening in my infrastructure. And also set up some analysis tool. Basically, what we're saying is a lot is expected from data scientists and ML engineers. You not only need to know about ML, you need to be an expert in software engineering, infrastructure, creating pipelines. That's a lot of expectations. That's not the only thing. There are quite a few other things that we have to deal with. Imagine you work in an organization and you are in charge of three different models. Recommendation, anti-spam and churn rate. Daily you get data and you have to update your model. And you know you keep updating your model on a day-to-day basis on a single day. Do the same for anti-spam based on what the data is coming in. You keep updating it and also you revise your model for the churn rate. Use that on Monday, Tuesday, Wednesday, Thursday, Friday. And if you're unlucky, which most of us sometimes are, even on Saturday and Sunday. Now this becomes very tedious and boring and leads to some errors if you have to do it manually. So there ought to be a solution for this where we can do this in perhaps in a pipeline where it can automatically run. This is where containers come in handy. If you are already aware of what containers are, this is just a quick recap on what containers actually are. What a container is, I can take a runtime. For example, in this case we can take Python. And I can add my code files or any other configuration files there might be. And I can also add external dependencies like TensorFlow and package it all up in a Linux container. Now all my dependencies, all my code, all my runtime is contained within this container. I can take this container and run it anywhere. I can run this on my laptop. I can run it on the cloud. I can run it on on-prem clusters. I can run it anywhere. If you only have one container or two containers, you can perhaps manage to run it yourself. But if you have more than one, if you have more than two, if you have like hundreds or tens of hundreds of containers, how do you manage it? This is where Kubernetes comes in handy. Kubernetes is a container orchestrator, a place where you can run your containers. Here I'm showing you a cluster with two worker nodes on the left and one control plane that is in charge of running your application. This is the control plane. Now think of Kubernetes as the best Tetris player ever. Imagine you have a container that needs to be run. What Kubernetes does is scans the nodes and figures out where to place this container. In this case, it says, I'll put it in the left one. And then the same application comes in again, tries to spread the load. And it puts it in the second machine. And another one comes. This is a smaller application. Perhaps it doesn't require as much memory or as much CPU. So it places it on the left one. And then the next one comes. And it starts placing it. Basically every time your application in terms of a container that needs to be run, Kubernetes manages the placement of these containers onto the infrastructure. And if even one of them fails, it can try and respawn it again. And what it will try and do is package everything fully into the cluster. Now it's time to introduce you to Kubeflow. Kubeflow is an open source community driven project, which makes it easy to deploy and manage an ML stack on Kubernetes. Kubeflow project helps machine learning engineers and data scientists with most of the issues that we discussed earlier on. Even though we're focusing on Kubeflow pipelines, if you're not aware of Kubeflow, I think it might be a good idea for me to give you a quick intro to it. Kubeflow is the ML toolkit for Kubernetes. You can run it wherever you can run Kubernetes as it sits on top of it. You can run it in a cloud of your own choice. If you'd like to run it on your machine, you can use Minicube or you can deploy it on an on premises Kubernetes cluster. You can run it on OpenShift. Wherever you can run Kubernetes, you can run this. Kubeflow installs a number of components on your cluster. We'll mention some of the components. Kubeflow deployment includes support for spawning and managing Jupyter notebooks. The notebooks data scientists and ML engineers can use to experiment with model and data. Kubeflow pipelines, they're reusable end-to-end machine learning workflows built using the Kubeflow pipelines SDK. And the SDK is in Python. Yes, you already write. It's in Python. You can write your pipelines in Python. You don't have to write them in YAML or any other configuration language. You can do it in Python. It basically is an end-to-end orchestration. You can write an end-to-end ML pipeline. It's easy experimentation. You can trial experiments quickly and iterate over them. We'll see that today. It's easy to reuse. You can make components that can be reused across multiple pipelines. These are the two components together with Kail that we'll be focusing on. Let's have a look at some of the other components. Operators are software extensions to Kubernetes that make use of custom resources to manage applications and other components. These operators let us use the functionality of deep learning and ML frameworks that we like. For example, PyTorch MXNet. With Kubeflow, a number of operators are installed. Hyperparameter tuning is a big topic in Kubernetes. With Kubeflow, you can use CAPTIT for automated tuning of your machine learning models, hyperparameters, and architecture. The goal of the metadata project is to help Kubeflow users understand and manage their machine learning workflows by tracking and managing the metadata that the workflows produce. Metadata means information about executions, for example, runs, models, datasets, and other artifacts, for example, files and objects that form the inputs and outputs of the components in your ML workflow. It's like source control for your machine learning resources. Kfserving uses Knative to provide a Kubernetes custom resource definition for serving machine learning models on a number of frameworks. It also comes with your favorite machine learning framework pre-installed. I'm going to list them here. TensorFlow, PyTorch, scikit-learn, MXNet. Now, most reinforcement learning processes for machine learning start with ETL. Extract, transform, and load data. And when that happens, then we need to do some training based on the data and use one of the algorithms from a number of frameworks of our choice. And then we analyze the results. And we predict it. If the outcome is not what you expect, then you have to start again. This is where Kubeflow pipelines come in handy. Kubeflow pipeline, as discussed earlier on, is a platform for building and deploying portable machine learning workflows. Based on Docker, the platform actually consists of a user interface for managing and tracking experiments, as you will see in a second, and an engine for scheduling multi-step ML workflows. It comes with an SDK for defining and manipulating pipelines and components. And we can write these pipelines in notebooks using KL and run them directly from the notebooks. Once the pipeline is written in Python and uploaded in Kubeflow pipelines, this is how the visualizer of Kubeflow pipeline presents. In order to write pipelines in Kubeflow, there's a number of steps we have to go through. First, you have to write the code for the pipeline and then create a Docker image for each of the steps that need to be run. Then you'll be writing domain-specific language Kubeflow pipeline using the Kubeflow pipeline SDK. Once that's done, you have to compile the pipeline and then we get an output in a YAML file, which we can YAML file or a zip file, which we have to upload the pipeline to Kubeflow pipeline. And once that's done, we have to run it in Kubeflow. And if you need to make any changes, you can imagine you have to go back to change your code, rewrite it, can create the image again, compile the pipeline again, re-upload it again. That's a long process. With KL, we can iterate through things a lot quicker. We only have to write the code in the notebook, tag the notebook cells correctly, which we'll see in a second, and then run the pipeline with a single click. You can imagine if you'd like to iterate through and test out the changes in our pipeline really quickly, we can do that using KL. If we were to use Kubeflow pipelines directly, it will take us a lot longer to be able to build an image, upload it, and then come back and make the changes again. So KL, the Kubeflow automated pipeline engine, allows us to take the code that we've written in Jupyter lab notebook, there's an extension there for install for it. We annotate the cells in the notebook with the correct tags, and then we convert it into a pipeline. Let's quickly do a demo of what it looks like. We'll compare how to run the pipeline using Kubeflow pipelines and then doing it with KL. Okay, for this demo, we will be using Minikef. Minikef lets you run Kubeflow locally on your machine. Now, let's have a look at how we can do this. You can head over to the website for minikef on kubeflow.org that gives you instructions on how to install it. Once you get Vagrant on your machine and VirtualBox on your machine, you'll be able to set it up really quickly. You don't have to use Minikef, you can use any Kubernetes cluster you like. Now, once it's set up, you can do Vagrant in it, Erecto slash Minikef and do Vagrant up. After a little while, it'll spin up Virtual Machine and install Minikef in it. Once it's set up, you can head over to 10.10.10.10 on your machine. It will come and will give you this UI and together with Kubeflow, it will also install Rock. Now, once it's installed, you can connect to Kubeflow by clicking the connect to Kubeflow button. In this view, you can see this is the dashboard. There's an area for pipelines. There's an area for notebook servers. There's an area for a cathode where you can do hyper parameter tuning and volume snapshots that we take with Rock. That's where all they get stored. Let's head over to notebook servers because that's what we will be using. I have already some notebooks set up, but when you are setting up a notebook, let's go and see how you can do that. We don't really need to worry too much about volumes. We will give it a name. Let's call it demo one. That's the name of the user. In here, we will use one of the pre-built images from Eryctor. It's TensorFlow image with notebook CPU. You can use any of the other images or you can even provide a custom image that you'd like yourself. You've got CPU and memory that you can increase. Once that happens, we don't need to do anything else. We can just launch it from here. Now, that will take a little time and set it up, but what we're going to do is use one that's already been set up and for it to load. Now, this is a brand new notebook that's already been set up. Let's clone a couple of repos that we're going to be using. I'm going to click on the terminal, make a little bit bigger once it opens up. There you go. Let me clone the Kubeflow Pipelines repo. I'll just quickly get cloned. That's the Kail examples that are cloned. I'll also be cloning GitHub repo for Kubeflow Pipelines. I'll let that load in. What we want to do is see how we can create a pipeline using normal Kubeflow and how it compares to the pipeline that we were creating using Kail. Now, let's head over to the Pipelines. We already have some examples in here that we can use. Let me just show you one of them. Let's click on Samples, Core, and we're going to look for Execution Order. This is what a Kubeflow Pipeline looks like. If you've not seen it, I'll quickly give you a rundown on how it works. What happens is we use this library kfpq for Pipeline, and we use this domain-specific language DSL part of that library. Now, every step in here is a container. We say this is a container, this one uses a bash container, and we run some commands in it, and we parse some arguments. This is another step in the Pipeline, so these are all different steps. Once we define the different steps, what we can do is define the execution order of Pipeline. We can say we want to run this one after the other. We define the two tasks. Step one is echo one up and echo up two. Once we're happy with that, what we can do is basically just do we want to run it one after the other. The last bit here is kfp compiler. This is the bit that's actually going to compile this into a YAML file. How we do that is if I head over to the directory of where this is, we will be able to compile it and upload it and see how it looks. If I just quickly do this, this will give us a YAML file that we can go back and upload in the kuflo pipelines UI. That's a YAML file. We can see this is a YAML file, and we can download this file and upload it in kuflo. Let's head over to kuflo pipelines. In here, you can see there's some pipelines have already been created. If I go on a pipeline, I can upload a pipeline, give a name. This is our normal pipeline. This is our kfp pipeline. We'll give it some description. Let's go to test. I'm going to upload the file that I just downloaded from kuflo. Now, I can create this pipeline, and it'll take a little while for it to create. I can show you one that I created earlier. This is the pipeline that we created. It'll load up in a second and we will be able to see the graph. Now, once that's set up, that's okay. If you want to make a change, I have to go back into my notebook and open the piping file, update it, make the change, compile it again, upload it again. It'll take a long while for me to make any changes if I want to experiment with it. Now, this is where Kale comes in. If you go into notebooks, if Kale is not already installed, you will be able to install it yourself. If I just searched for kuflo, Kale, let's go in here. This is the document GitHub repo for Kale. If I go down, you will be able to see the instructions and how to install it. Now, the image that we're already using, the one from Eryctor, that already has the Kale deployment panel available to us. All I can do is click on this and enable it. Currently, I can't run any experiments because I have a name. I haven't opened a notebook that has experiments, but I'll show you where these experiments are and how they actually work. This is, there you go. Here's the pipeline that's been created for us. We can execute this pipeline, create a run, or whatever we want to do, and we can get an output from this. This is a pipeline that's created normally, but what we want to do is we want to actually run it directly from our notebook. We want to write the pipeline in the notebook and click the button from our notebook that'll run the pipeline directly. Let's head over back and just check it out. If I just head over to my pipelines now, so this is the examples folder. I'm going to pick one of those Titanic examples. This is ready from Kuflo itself. This is an example for basically what we're trying to do is use machine learning to create a model that predicts with passenger which passenger survived the Titanic shipwreck. The details of the notebook are not too important because I don't really want to go into the machine learning part of it, but what I want to show you is how do we use Kale to be able to annotate your blocks, your sections within the notebook and basically run them as a pipeline and how we get them to rely on one another and make them dependent so basically runs as an actual pipeline. Now what you'll see is the notebook looks slightly different in this case. I'll explain some steps and then we'll go ahead and run this. Now every cell you see in a notebook is annotated and there's different types of annotations you can add. So if I click on this edit button here you can see this one is an import type, so it's basically importing the libraries. We've got four different, five different types. We have functions and what that allows you to do is prepend to every step the pipeline. This function will be prepended to every step. We can also set up pipeline parameters if you want to pass pipeline parameters for every time we run a pipeline, perhaps you want to run over the different parameter so we can define the parameters or this is the bit that we used most which is the pipeline step. We'll give it the pipeline step and we can also tell it which one it depends on. The last one sometimes we probably want to skip some cells that are in the notebook for debugging purposes or visualization purposes but we don't want them in the pipeline. So this is import and this is all good but before I even do anything I need to install the requirements that are required for this notebook within the notebook so I'm just going to quickly jump back to my terminal and run that. So let's just pick three install mysr requirements.txt. Now let's just head back to the notebook and see what it's doing. So the first one we talked about is an import and you can see this is a pipeline step. We've given it a name. There's nothing special about it. We're just a normal pipeline step and we've given it a name and what we can also do is make that pipeline step dependent on another step. You can see there's some other steps in here that have been defined later on. Let me show you some more these steps. So now this step is really just looking at the data frame and giving the information on what it is. We don't really need this so we can skip that. So we're skipping some of these steps in this case and when we get down you will see this one data processing step. This cell is now is a pipeline step that we said that's the name but also this depends on the load data step from before that we defined and you can define multiple steps if it depends on. So there's again you know we're basically just going through the data here doing some training. I'm just going to skip that part. Now this feature engineering is a pipeline step that depends on data processing. So we can define them all one by one and in the end what this does is produces the result but we can't really produce the result unless everything's complete. In this case we want it to finish to depend on all the steps that go before them. Now our notebook is all good to go. All the requirements are installed. Now I can open my cube flow here and pick this so I can say oh the experiment I want to run is Titanic because this is based on the notebook that I've just opened given it a name and what I can do is you can see I don't have to compile anything from a command line or anything like that. I can click on this button compile and run. So when I do that it will take a snapshot because in this case we are taking snapshots the pipeline if you wanted to run it beforehand if you wanted to run it afterwards again and what we'll do is it will take the pipeline and upload it into a cube flow pipeline itself directly. We don't have to download the file and do that and everything is defined within the notebook. So snapshot is done and now it's kind of trying to upload. So the upload has been successful. Let's just jump over back to the pipelines. Let's see. So this is the pipeline that we just created. Here we go. This is what the pipeline looks like. I'm just going to zoom out a little bit maybe a little bit more. You can see we create volume in the beginning and loading the data data processing. These are all the steps that we saw right. Feature engineering these steps depend on one another and you know we're doing some of the other steps in the end and then we say the result step is dependent on all these other steps. So that's how we create this dependency graph and all the pipeline. Now once that happens it will actually kick off the experiment itself. Now that is not something very exciting. As you can see the experiment has been kicked off. It's actually running right now. But let's have a look at one that was completed before. Let me click on this. So this is how it ran through but if you've not seen a pipeline before you can see all these steps are running through running the code that you wrote in your notebook. We didn't write a single container. There's no container inside. You can check the logs if you want to check the logs and then you know in the end there is an output as a results output that's that's happened here. So all we did was wrote our code in the notebook and from the notebook directly we compiled and ran it. We never produced anything of a container image or produced a pipeline file and uploaded it. This is the beauty of Kail. We very very quickly we managed to run this pipeline. If you like more details on this topic definitely check out the Kail. Also hit me up on Twitter at Solmaneq.com and I'm here to answer any questions. Thank you for attending.