 Test, test, test, one, two, one, two. Great, good afternoon. Many thanks for joining our session today. It is a good decision. It's your decision and it's a very good decision. So unfortunately we have only this microphone, but we will try our best here and even for the camera. Therefore, we would like to talk today with you about an MLOps platform on top of OpenStack. My name is Thorsten Deutsch. I am Senior Product Manager and Product Owner for Big Data and Artificial Intelligence on OpenTelecom Cloud, one, two, three. Yes, this was right. And I have together with me two colleagues. So therefore, first of all, it's Bebe. Test, test. Hi everyone, my name is Bebe. Since 2019, I'm working at T-Systems initially as a data scientist, now as a cloud architect in Big Data and AI squad. Hello everyone, my name is Franz Kukuchka. I've been working in the Big Data and AI squad for two years now and I've been working on several AI projects in the past five years. Good, and we solved the solution with the microphone. Perfect. Therefore, if we are talking about artificial intelligence, we have to talk about the underlying platform. And everyone is talking about artificial intelligence in Europe, we are doing this as well. And we have a cloud where you can develop your own artificial intelligence. And this is the first part of the speech to give you a short introduction about the OTC and about the USPs. Later on, we will come to some more details about some demo and of course how to set up a pipeline. So if we are talking about the OTC, the OpenTelocom cloud, we say we are GDPR compliant because we are operating in the open-tailed cloud. So we are operating in the open-tailed cloud. Because we are operated in Germany by a European company and of course our data center are located in Europe, in Germany, in Amsterdam and in Switzerland. Therefore, we can say we are GDPR compliant and of course we have some audit data protection according to European law. So that means here is the data privacy, data security already included. But this is only one thing. In addition, everyone is talking about sovereignty even today in our keynote. You have seen this and we are offering this as well. So like data sovereignty, so you can take care of your own encryption of the data. As we are today on the open-infra-summit, we are based on the open-stack. Therefore, we have a technology sovereignty and in addition, the operational sovereignty as we are operated by Deutsche Telekom agi. That's a short introduction about the two USPs of open-tailed cloud. So in addition, of course, we are secure and we are reliable. So that means we have a huge, we have an SLA available on our services. Therefore, you can have a look. This is a summary of the open-tailed cloud. If we are talking about the USPs, then of course it's one thing, but I would like to say the USPs are really a great USPs and you can see it on our numbers how huge we are. We have more than 100,000 VMs of our customer on our open-tailed cloud. And in addition, more than 140 million people are using the services on open-tailed cloud. You can imagine how big this is, how huge this number is. As how many citizens do we have in Germany? 87 million, something like this one, yes. So 82, a lot of, yeah. And you can see we are really a cloud which is based in Europe for European citizens. And there are a lot of people who are using on our open-tailed cloud. Let's go to Model Arts. Model Arts is an MLOps platform on open-tailed cloud. Which is an end-to-end development platform for AI development. And everything starts with data. Therefore, you can place your data on our object storage and choose just Model Arts and use the data with Model Arts. Most of this preparation is like data pre-processing and all this stuff. You know, if you are developing an AI algorithm, you need more or less 80% time for all this data preparations. And this can be done with Model Arts. As we are offering some tools like data labeling, version management and all this preparation stuff for the data. This is very simple. You can use it via GUI on open-tailed cloud and we can show you this later on. And as soon as you have done all this data pre-processing, data labeling stuff, you can train your algorithm. This can be done with Model Arts as well. So this whole model training stuff is very simple in Model Arts. And we are offering several functions like built-in algorithm, pre-installed frameworks. Even you can take your Docker containers with you. So that's very simple. And you can use all cloud resources like GPUs with Model Arts. And very simple, you don't need to take care of all this handling of these resources. So as soon as you have done this model training in general, you can do this model management like model evaluation, precision tracking and all this stuff with Model Arts. So Model Arts shows you, for example, the accuracy and also some more figures how your model is working. And in addition, as soon as you have your trained you have done your data labeling, you have done your training, the model evaluation, then you can deploy your model. Very simple with Model Arts and just a few clicks. So just click like on a GUI or of course you can use the API. And it will take two minutes and the API will deploy directly on our cloud. So that means you don't need to take care of all this deployment. You don't need to take care of all this performance management. This is done inside of Model Arts. So deploy it very quickly on OpenDailer.com cloud. So for this whole chain, everyone can do this and it's very simple via GUI and API. So in addition, we are planning some kind of AI marketplace which is currently not available just in planning where each customer can share his model with other customer. So therefore, you don't need to train your own algorithm. Even you can use all these models from other customers as well. So when we are talking about this whole process which is here, let's say more or less simple on our slide deck, we have some different kind of user groups in the AI development. So even in this round, we have some guys who have a huge experience but even some guys who have never done some experience on a console. And therefore, we are offering different type of functions. One case and that's the typical case is ILOV coding. You have a lot of experience. You know how to develop this. You can use all this built-in frame or the pre-installed frameworks. You can use the Jupyter notebook which is very simple to use. So therefore, we are offering all these features for this, let's say experience group but you don't need to take care of all this resource handling in the backend. So that's the first group. The second group is you already have some experience but you just would like to deploy a model very quickly. So in this case, you can use a built-in algorithm like ResNet which is available. So a lot of built-in algorithm for image classification or object detections are available and can be used. You just need to upload your data to OBS and then use it very simple in model arts. And the case three and yeah, you just would like to use a model and train it very quickly so you don't have any experience. You can use model arts. You just need, let's say 30 minutes time and a little bit data and you can train your algorithm by your own. So that's very simple, 30 minutes, everyone can do that via GUI and of course, we have some examples for you. So as you can see, we are offering this model arts a lot of features and services for different kinds of user groups and therefore we will come now to Bebe. Thanks, Kostin. So as Kostin has mentioned earlier, so model arts is an AI platform for AI lifecycle management. It can be used for image annotation. For example, like marking the critical factor in the medical images, it can also be used for like algorithm development and model training, but notebook instance service, as well as further developing AI products. For example, auxiliary diagnosis for COVID-19 and for cervical cancer. We have shown model arts actually covers several sections and today I will mainly focus on the case three which is the auto learning part and in model arts we call it XML. Oh, sorry. So in model arts, like the auto learning part is called XML. It covers the three projects at this moment. We can simply click create a project and then we will create image classification projects. Kostin mentioned earlier that we can use a preloaded data set saved on OBS. So, but for saving the time, we were just looking at the existing image classification project which is called XML COVID-19. It covers over 100,000 long CT images with three different labels, like pneumonia, like normal and COVID-19. By simple click the train button, we will reach the training configuration and here we can setting up all the training parameters. So, we have already existing trained model and we trained before and in the training model page, we will see all the training results, like all the details like the training accuracy at one score and other details. By simply click the deploy, we can deploy a real time service either on a CPU or a GPU instance and it also supports auto stops. We can look at the deploy service and this is the service that we have deployed before and we can redeploy it and after clicking okay, we will see the deployment task is already submitted and the service is deploying. But by this, I would also show the service deployment within model arts. As you can see here, the service, real time service management covers all the user guide and configuration and monitoring and other details. So everything is in the service deployment management and we can also use this like a simple example to test our model how it performs. We uploaded COVID-19 CT data and after a few seconds, we will see the prediction results and as you can see here, it also said like highly could be a COVID-19 image. So now we go back to the XML part and actually in the deploy service page, we will have the same inference services. We can play with the other, let's say, other medical images and to check it. So what I want to mention here is that so once we have this real time service running on open telecom cloud, we can do further development. In the next slides, I will show a very simple medical webpage which built beyond the real time service that we've seen in the last slide. So this is how the connection is built between open telecom cloud and the full AI products. So it's a very simple examples and that's all about XML. I only covers the XML part, so now I will give the time to my colleague, Ferris. He will cover more sections of the motorized and also he will present how to build AI pipelines within open telecom motorized service with OpenStack extensions. Thank you. Here I will show you a quick demo about our OpenStack-based Model Arts Automation Pipeline. The most important thing to note is that what you will see is a result of an automation script that just invokes OpenStack commands and replaces those lots of clickings we would do when creating a fully AI pipeline on the GUI. So this solution is automated and much faster. Here you can see the main steps. I will iterate through in the following minutes. So first I will create a data set, then I will label it. After that comes the creation of the training job from that labeled data set, then I will build a model from that trained job and after that, the last step is deploy that built model as a real-time service. Here you can see my automation script with the OpenStack commands, variable initializations as past definitions. When looking at the past definitions, I'd like to show you that in object storage service or OBS, which is our S3-style storage system, so in OBS, those data and the work folders are completely empty, whose path sign defined in the shell script. So as you can see, the data folder is empty. Here I will store all the pictures from the flower pictures for this data set and the work folder is also empty. This will contain the annotations, the labels, the manifest files, so all the metadata for the labeling. So I go back to the automation script and show you the other parts as well. So the data set creation command is there. The trained job creation, the showing the trained job, so attributes, log messages, and everything that are necessary for the training. So this is my automation script. I will exit from it and run this script. Here you can see this test argument. The argument test means that the word test will be in the name of the data set service, so to make this unique. After that, I'm going to the dashboard of Model Arts again and show you the main services of Model Arts. So data set, notebooks, training jobs, and model management. We will use three of them, data set, training jobs, and model management. Notebooks contains the Jupyter notebooks. Data set can be found under data management and data sets, I will click on that. And here you can see that MA chain DS test, which the OpenStack script named it. So this is our data set. As you can see, it's labeling type, it's image classification, this is what I set it in the OpenStack script and the labeling progress is 0%. I will have a look at the automation script. It's still in progress as you can see because it's creating the labels, so it's not finished yet. That's why we have this 0% of labeling, but in a few seconds, when I refresh the data set dashboard, then you can see that it's already labeled. So 100 slash 100 and the last step is in progress which is the publishing of the data set. So in a few seconds, this should be also finished and we have newly created data set that is containing 100 flower images and I will show you how it looks like. So I click on the data set name and as you can see the labeling progress is 100, all zero one labeled. So we have labeled all the flower images. Here you can see tulips, dandelion, sunflowers, daisies and roses. So these five categories of flowers, 20 images of each, all together 100 images. I show you a couple of them. So we are gonna need these flowers because we want to train a model that recognizes these flowers based on the flower type. So the corresponding labels can be found under the images as you can see, some roses, tulips, basically that's all about the flowers. So we have all of these. We are going back to the data set dashboard and here you can see this input data set path and output data set path. So if you remember, I showed you in the OBS that they were completely empty. So now they are not. They got populated with files. I will show you that in the data set versions, we have these storage paths as you can see. So this is the OBS URL which defines the location of the manifest file. The manifest file contains the annotations of our data set. So I go into that folder, data and import folder. Here you can see the five categories of flowers in these five folders of flowers. And in each folder we have 20 images just to show you how it looks. So I clicked on the sunflowers, the URL, and as you can see, this is how a sunflower GPG is present in the data set. So I go back to the test pipeline. I showed you the data folder which contains the data sets. Now I showed you the other one which is the ver folder. So this has the V1 folder which is the version one. If you remember, we were in the data set versions and I click on the manifest file. This is also accessible via URL. So I click on that and open it in notepad++. So as you can see, the annotations are there, the name of the image, the annotator. So who was it annotated by? The image classification, the pass, the usage of it, which is trained in our case now. So we have this in JSON format and 100 elements of data sets of flowers. This is how the manifest file looks. So we are going to the next step. We have a working data set. And remember that our onto mission script is working in the background. So I go to the training management, train jobs. Now you can see that it's running. So it says it's running, which means it's not ready yet. I click on, it's a training job is one in the top, as you can see from the date. So I click on its name to show you the configurations of it. So it's still running. We have the version which is one. The specifications can be interesting for you, which we used for the training of it. We are using the ResNet as the algorithm and the IH in distance or flow and we are using Python for that. So I go back to the training job dashboard and as you can see, it's already successful, means it's finished. So now we have the data set ready with the train job ready. So the third step is the model management models. So let's see, our model is there. As you can see, MA chain, MO test, which we named in the open stack script. So it has been created. I will show you some of the details of this model. You know, we have this basic information, but the parameter configurations can be more interesting for you because it defines what we have as an input for this task. So it's an image, as you can see, it's a flower image and what the output will look like. It's a predicted label. So which type of flower we have from the five categories and the scores. How many percentage it gives to each category because it can be important for the training if we have 50, 50% for two flower types because they are that similar. So now we know how to find tune or hyper parameter tune or model. So as you can see, we have this basic info as the train job, so runtime environment, AI engine as well and the OBS path, which I showed you a couple of seconds ago. So the last step is the service deployment. In model arts, we have two ways for that, real time service and batch service. Since we want to get our images predicted in real time, we are gonna use the real time services, not the batch one, and as you can see, it's deploying 9%. So this is the most time consuming part next to the training of this whole model arts automation chain pipeline. Its name is MA chain asset test. So if it's working fine, then let's see the open stack script. As you can see, it's finished running. So let me show you a couple of things. The dataset version, the train jobs, the logs, the attributes, everything can be found in there if you are curious about something. It's already 19%. So while it's loading, I'd like to show you the subwind, left subwindow in the model arts. So data management, training management, model arts, service deployment. So all these were defined in the shell script, in the open stack automation script. But you know the main difference that we have this magenta style cool looking GUI where we have to click from these buttons if we want to get from A to B. But the other way is the model arts, open stack CMD way, which can be automated. It's black and white, but you know it's much faster. And while I was talking, this whole deploy has been, this whole service has been deployed successfully and we are gonna use the prediction tab for the inference. So I hit on the prediction, but you know you can have the question based on what should we predict. So I'm looking for some images in Google that our model has never seen before. So you know, because that's how we use this whole inference based on images that our model has never seen. So I downloaded one daisy locally. I will upload it later and one tulip as well. I will upload that one as well to model arts, service deployment to see what the label will be. So it's already uploaded to tulip, I hit on predict. It's scanning a few seconds and as you can see it's predict label is tulip and the score is 100%. This can be understood because tulips are quite unique and different from the other flowers, but the other one, the daisy can be a bit trickier because if you think about it, daisies have a little similarity with some flowers. So let's upload that daisy image as well. It's being uploaded. I also hit the predict button now. Now comes the scanning and in a few seconds, this predict label is daisy, as you can see the percentage is 97% and 2% to some flower, which can be understood because the mentioned similarity. Okay, many thanks. So we have spoken about the OpenTelecom Cloud. We have spoken about the ML Ops platform, model arts on OpenTelecom Cloud and how to set up AI pipeline. We are already here for all your questions. Please use the microphones if you have some questions. Otherwise, after the speech, we are here as well and answering your questions. One, two, three, everyone would like to go to the microphone, right? Yeah, many, many thanks. One, two, one, two, it works. So could you please elaborate a bit, like what's underlying this ML Ops platform? Like how exactly is OpenTelecom, whatever you use is actually used in there? Yeah, some technical details. So we developed this with our vendor together. So therefore, we can say this platform is directly developed for OpenTelecom Cloud. So we can't move this directly to another cloud and say this is behind. What we are offering is we are offering some, let's say we are offering this open source framework like TensorFlow and even we are taking care that all this algorithm which are available, of course, open source. And in addition, we are taking care to choose like standard containers, like Docker containers and all this stuff. So really, we are taking care of all this open source mindset. Totally fine. But you need to use the microphone, yeah? Otherwise, someone kill me here, yeah? Yeah, perfect, perfect. So regarding the ML Ops platform itself, the model operationalization functionality is pretty neat. But what about experimentation management within it? Sorry? Experimentation management for ML Ops platform. That doesn't support it? Like what's the functionality there? Would like to someone answer? Expectation. Expecting me? I didn't get it. Yeah, I did not get it. Yeah, sorry. Yeah, so part of whole machine learning pipeline or safe process is the very last piece, last mile, you might want to test multiple models at the same time and you want to observe which works better and all this stuff. And you want to, it's not just champion challenging stuff. It's not only about observability, but also about keeping track of all the changes you made along the way. Yeah, so it goes. Some like A and B testing. A testing plus and a ledger for all the experiments that you've done. And so I think you are talking about like, so when you build with different algorithms, you have different models and how to compare it. So actually, model arts offers this comparison. You can also do it within model arts. And meanwhile, like each of the model, every time you train one, you will have a new model and we have a model management section to manage the different models. So like a version control of the different models. So that's a part. And besides of this, let's say we offer like, like simple click to deploy to a real time service. But meanwhile, how to, which model you deploy and when the service need to be stopped also is fully controlled by the end users. Yeah. Thank you. You're welcome. And of course, we are extending this platform all the time. So this is just the beginning as we are now in the better phase and this will end. So this will be released as a public soon and I can do some marketing as well. So of course you can use it free of charge with limited resources, of course. You need to take care, but just training, just one training job, one influence job in parallel, but you can use it directly free of charge. And of course we will extend this all the time with some additional features. Therefore, if you would like to use it, go to our homepage and of course you can share your feedback on our community portal with us. Therefore, many thanks. Oh no, there's one question. I'm sorry. Happy. Yeah, you are speaking of Docker container. So for all your platform, are you using Kubernetes on top of OpenStack? So yeah, and do you see a value of you are showing some OpenStack commands so based on the feedback of customers, is it useful to have this unique CLI for managing VMs, managing AI workload? Is it something you see some, you get some good feedback or is it useful? So last but not least, it's always good to use this kind of open source because if you are developing your artificial intelligence algorithm, you need to take care that you are not only based on one application or not only on one cloud or even you need to take this with you. And therefore, for my point of view, we need an underlying platform which is, let's say, Open and which gives you the possibility to take everything with you. Let's say these in simple words. Because you said so it's to the question of underlying components. It's not MLflow, it's not Qflow, it's something more proprietary way. So it's why just to understand, you said it's open about the way of acting but perhaps you don't have other communities contributing to like, to module to your solution. Are you are perhaps taking from other communities on the MLops technologies? It's your own technology, in fact, you said no. Yeah, it's our own technology but we are taking care to use this open infrastructure, let's say, the OpenStack and therefore we created this pipeline based on OpenStack with OpenStack commands. Thank you. You're welcome. Okay, many thanks. We are still here for some more questions but enjoy your day and many thanks for joining our session today. Thank you. Thank you everyone.