 So we start. Good morning, everyone. Welcome to today's, welcome to listen to my presentation today. And my presentation will be based on the QIS and build an event-driven machine learning pipeline on Kubernetes. So first brief introduction on myself. I'm from IBM CI lab. And today we focus on AI cloud and in open source Google flow. And I'm the maintainer of many projects. And we have a American colleague who cannot be here today because of some reasons. And he is a focus on AI cloud. He is also a researcher and expert of this field. And he focuses on the model configuration and the background of our solution, which is from the center for open source data technology team. We use open source tools to offer the whole life cycle. So solution for enterprises. For now, this team already offered many AI cloud life cycle open source tools for enterprises. Some modules related with fairness and safety. The first. Maybe for many people, the most impressive story related with AI deep learning is IBM deep blue champion of human beings chess player. And later we also know that wasn't IBM robot can beat human beings winner in this community. This competition in 2011. And this is more familiar for you, I believe. So in Chinese chess, alpha gold beat human beings. And about the very hot topic deep learning. The development phase can be divided into several milestones. Just like the slide show you. And in 2012. And introduced deep learning to the world. And many technologies, which is which are widely used in this field. So the first most popular one is Facebook's face recognition in 2015. This is a very good application. And Siri also gets deep learning in 2016. And alpha gold was popular in 2017. For now, the deep learning compared with human brain. Deep learning cannot be compared with human brain. We still have a very big potential in deep learning. And in the life cycle of the deep learning, maybe first you were focused on machine learning model. And the coding of the model and how to design this model. So you can see that's the time or the energy distribution group chart. So this is the reality. Actually, the time you spend in coding and the designing of the model is very little. We need to prepare a lot and collect the data. And the maintenance or the measurement after we launch the model takes a lot of time. And this is unexpected. And this is a reality. And this is a neural network design for workflow. So we designed this network and start another experiment and test to see whether we reach the performance standard or not. If not, then we were going to do another round. And in reality, we also have some fresh data we cannot get before. And we can use this new data to test the model. We can see that whether the model can adopt the new data or not. If yes, if no, we have to go back and go through another round. Then we will repeat this workflow again and again. And you can see this is the chart in the AI life cycle, what kind of technology we will use. So first the data preparation and initial model and train model, deploy a model. So that's some indicated steps. And maybe we also have some different steps for some specific models. For example, we have many tools available to build the next models. For example, Jupyter and etc. Here, I want to give you an introduction of the Watson's IBM tools. And this is a very user friendly. In this IBM interface, we can take some existing layers, for example, CENCN and some rules. You just drag in the UI. So it's a very intuitive neural network. So this is a very clear structure. After you drag and form a structure you want to have, then you can see which can support a lot of different formats. For example, the notebook or some very popular framework you use very often. For example, Taskflow, Belltouch, etc. And talk about the framework. So we have many machine learning structures of frameworks. Different study researchers have different preferences. And in this platform, we have to manage a lot of frameworks. Then we need to focus on those details. So after the training is accomplished, how can we judge this model is trained or not? Or how can we make sure the prediction from this model is very accurate? So we have to focus on four parts, four steps. So first, is this fair? Whether this model can involve different objects, just like our whole world, we have different races. And we can get some very fair scoring and treatment. And secondly, is this easy to understand? For example, some data are very difficult to interpret. And suddenly, did anyone temper with it? So whether we can make sure the safety is reliable? And the fourth one is this merit accountable. So that's for dimensions we have to focus on. The fairness, explicability and robustness and assurance. First, we will talk about safety or assurance. Is the model vulnerable to the possible attacks? And this is the robustness, two blocks from IBM, which is called Art. And IBM already realized many mainstream robustness, two blocks. And we have different ways to realize that, which is listed in this slide. And for these different attacks, we can have some counter-attack measures. So whether this model is robust or not, we also have different parameters to test that. So here we listed some parameters and we can support the training and prediction. So they have a unified model API. And about this too, I'll give you a brief introduction. This is an art demo. Through this link, you can click and visit our website and try it out. So this is a cat. So you can see that this is a cat. And the machine also gives a very accurate prediction, 92%. And if we have made some attacks, usually we cannot see any change, but it's already changed to an ambulance. And the credit is 19%. So we will give it some attacks. So we can address the attacks. So the robustness can be improved. And next, whether the model is fair or not, about the fairness. In human being society, we have some discrimination. And this cannot be avoided. For example, people may judge others according to the social status or appearance. So machine learning is based on human history data. So machine learning can use some data, which can produce a model. And this is technology related with human beings nature. So discrimination cannot be avoided. For example, this is a model which is testing the outlook or the appearance of human beings. And if this is a model made by cohesion, then if we use a feature from a Chinese very famous actor feature to test this model, maybe this model thought, this is not a handsome gentleman. So that's a discrimination. And now this is AI fairness 360. And this is an open source library. And we included more than 30 indicators into this library and we can also use some function to produce explorable tools which can be interpreted by human beings. And we can make some adjustments about the discrimination of the model. We also have some mitigation algorithms. We can support 10 algorithms. We continue to involve more algorithms into this library. Once we already validate the model fairness and robustness, we can deploy it. And we also need to think about more details. For example, we have many versions of this model. How can we test? For example, AB test, how we can have a dynamic adjustment of the traffic. And about the real-time capacity of performance indicators, we also need to have more validation. And this is Istio, which is very popular recently, which can guarantee the connection between different servers and the load balance and the different indicators of the different servers. And it can also monitor the log-in matrix tracing and assurance and AB test and traffic. So we have many solutions in Istio. And the next question is about machine learning. Life cycle. So it's very iterative and repetitive. And the management process also involves a lot of tools. So we have to involve that with cloud. So why is that? Because cloud can also support the software and the hardware. And the scalability can also be supported. So that's some of the advantages which is brought by KYS. Through the containers and the micro-services, the maintenance is very simple. And through the KYS, we can deploy the machine learning models. And through the container, we can have a distributed training. But if we deploy the KYS in this system, it's not the end of the story. We also need to consider a lot of different aspects. For example, how to apply the tools in different aspects of the machine learning. How to be connected with the resource and the schedule. And if every model has to handle those basic tasks, it's very difficult for us to handle them. And so we have the code flow. Suppose you want to deploy this model in KYS, then you have to be the expert of different aspects. For us, for a new starter, it's very difficult for you to be an expert in different aspects. So the code flow can help you. And let me give you a brief introduction of this code flow. And the code flow can help you to separate the KYS and use the keep flow. We can do some resources management for you. And ML and developers can only focus on the models and the machine learning. So that's a brief introduction of code flow. Code flow is a solution of the AI cloud with a lot of modules. The key module is the framework of the operator of the different framework of the machine learning. And for example, the cafe tour and distributed training and notebook pipeline. And some sub modules, code flow can support all those different aspects. And code flow was introduced in December 2017. And the 6.0 version will be launched next month. And this is a very simple UI about the code flow UI. Maybe it's very simple. And if you are interested, you can check the website. And that's a brief introduction of the pipeline of code flow. And we use the code flow pipeline to do the machine learning pipeline solution. And through the pipeline. So this is an example. First we acquire the data and train the model. Then we will have a check of the robustness and attack. After the test is passed, we will deploy that. So this is the coding of this model. After that, this is the result of the model. So through this DEG chart, you can see the relations of them. And after the pipeline train, this model, we also need the developer to be triggered manually. So if we found that the model cannot reach the expected performance, we have to run the pipeline again. Suppose the capacity of the model is not very good, or we change the data, we have to restart. Then we have to trigger the pipeline again. And we need the data scientist and ID department to trigger that manually. But obviously we need an automatic trigger mechanism. For example, I'm a model editor. And in the GitHub, I have written some more pullings. For example, I added some AI neuron barriers. And now I need to push the pipeline. And I need to run the pipeline automatically. I don't need the manual trigger. Then this goes to another topic I want to present you. So this is a solution, which is divided by three parts, build, serving, and eventing. And we also have a pipeline solution, but this is already separated from the Knative. This is a separate project, so I want to skip that today. The build is very simple. How to separate the source code from some code containers and produce a mirror and push to the registry. And then the service, and the Knative serving. After we deploy the model, how to manage different versions, and how to switch in different routes and how to execute the network routes. It's done based on ECU, just like the first step we set. For all the things we have to use, a model to address, then we can use ECU to handle them. And the eventing, this is published to the sources and to those registrators. And the subscript can receive the event and handle them accordingly. Now eventing can support many different things. But the source of eventing can be tailor made according to the demand of different users. And maybe GitHub is used very often. They can be stable step by step. And the solution is like this. And this is the realization we have done, Knative source. In the hub, in the hub, we have the report, we can get the event produced in Knative. And this is the popular launchers job. And the popular launcher can catch the coding. And according to the event types, this is a push. And they can use the API on the pipeline launcher. Then they can start the Cubaflow pipeline. So this is the very simple modification of the previous pipeline. Because previously we have done some training. And the mirror of the building are L. So with that, we have some modification. We have the push and pull in the source code. Then it will trigger the pipeline automatically. Then it will rebuild the mirror. And use that in the training process. We will have the source code in the future. If we have more sources, for example, some image library or some warehouse. We can also set up some event. And we can trigger that from the data query. And this is a complete process. With many tools and many technologies, we use KOS, these two convective and co-flow. And some open sources of IBM. For example, some awareness and assurance tools. And some structure of the deep machine learning. So we integrated all those different tools into this lifecycle. And this is the AI pipeline solution we offer. So that's all. Thank you. Any questions? Two questions. The first one, the F360 fairness. This is related with the cloud. As no, it's not related with the cloud platform. About the algorithm of fairness. In the machine learning. This is a topic forced into 2017. And we also have a lot of papers which addressed fairness. And AI360 takes some mainstream solutions and realize them. So for programmers, we can use those tools. And we can use in our lifecycle so we can make our models trust. So this is open source, right? Yes. And this can be done locally, right? Yes. It's not related with the cloud, so it can be used in our lifecycle. And just now you mentioned the pipeline. You give an example. So maybe that's a detailed question. I know that we need to do the test for push, but for pull. I also need to do test when I drag something. The pull. So actually this should be the pull request. So that's an error. That's a mistake I made. So now I'm understood. Thank you. According to my information, pull flow is relying on the pull. Pull, pull, pull, pull. Pull is CICD procedure. For example, we submit some pullings which were triggered that process. So a lot of tools can be directed to GitHub. For example, some CID lifecycle is realized based on the K-PAP. So if we deploy that in K-PAP, is that possible? PYS is used in the PYS community. And it's CICD of the PYS. If we use that, we don't need to use it because the pull is managing the codes. In your lifecycle management, how do we use that? So we can package the upper layer. And we use the CICD to... So in this AI framework, we don't use pro. Pro is used in GitHub. And AGO can be deployed in your labs. And how can you get back the result? Once AGO is executed, how do you get the result into the interface? So for the AI for AGO, we can start the task. And we will also write the result of this task into the output or some literatures. And in AGO, we can read the output files. So this is some explanation with our program. So AGO cannot realize that. That's all. Thank you.