 Yeah, yes. All right Okay, hey everybody, we're live with Stefano talking about ML apps. What's what's going on with that? Take it away, sir. All right. Yes. Thanks everybody. Wow exciting So let me go immediately and share my screen so you can see some of the slide in the meanwhile I say hi everybody. This is fascinating ML and DevOps are quite interesting combination There is a lot of talking recently about optimizing learning It's the fun. Are you sharing your slides yet? I Cannot see it. Let me try it again. Okay There we go Thank you so much. Yeah One final thing could you close a little window on the upper right-hand side? Yes Yeah, just close it Yeah, thank you Awesome So yes, so why not the best of the two words the DevOps and machine learning together so I Won't just to share some experience and see how we can combine together Use the best of our machine learning operation or ML Ops or as it is called Right, so ML Ops is a way to recognize a Way of automating all the experiments that we build in machine learning This doesn't have to be necessarily as your machine learning studio It can be any application of machine learning So what actually means in a nutshell is bringing your machine learning code into production today If you have worked with machine learning before that the scientists typically They start Working on their own laptop. They beat an experiment and they like to experiment a lot and because machine learning is great Obviously, then it's really hard to find don't that specific Experiment that you're looking for that the solution that the business needs if everything is Consolidated on the data scientists laptop. So we really want to make a way of continuously delivering any improving The process all the good stuff that they've also brings but apply specifically to machine learning. So You may stay now on your laptop as a data scientist for some time and then before you get a formal dev environment Or a staging environment or you at T Eventually bring in operation now the reality is that from my experience a lot of data scientists that they Work in isolation or in their own team because you know, that's the environment They know the best and they do stuff like are or Python Which is typically a bit different from other mainstream the programming languages that you find for web or mobile application and They work all the experiment and also machines. They're running some GPU for example that are specific for deep learning also for some advanced machine learning and At the end that what happens? They just throw over the wall this code Straight into our production environment. There is very little Management around it, you know, it violates all the best practices when it comes to good software engineering There is where ML Ops comes in so ML Ops or ML operation or DevOps for ML call it the way you like it brings machine learning to great software best practices now ML Ops gives you really a ton of advantages and I can speak now for hours about it, but just to give you some ideas is all about Being able to reproduce and audit your process, you know When a model a machine learning model is built from You want to have some sort of verification or audit ability On it making sure that you're using the correct data set making sure that when there is a change this change is probably tracked and a New version of the model is created and then when you go into a production There is a pipeline that is followed rather than just taking the model throwing all the way around and making Hopefully sure that it works correctly Then because we are using the option basically you'll be able to automate this validation I'm checking that the code and the model itself is a correctly profiled and version and At the end you have also the possibility to measure the performance of it and feed that is back to the original code to the original And the region model to improve the accuracy of the model itself This is the advantage of automating this process Not only you train the model you deploy into different environments but you're actually making improving over time with collecting feedback and Retraining the model over and over again until your accuracy improves over time So what happens? This is no typical scenario in a In a Context of a dev process that is not particularly Automated from a machine learning side. Maybe the developer on the the traditional app developer on the top left corner Is already using a strong DevOps pipeline So they build the application in Visual Studio VS code they deploy it they deploy automatically with DevOps pipeline and all works well, but your data scientists, you know, which is just down there in the corner Maybe they're using something different like a Uptemnet and Notebook or a machine learning studio. They don't really fit into the typical scenario of Context of Development of the mobile of the web Developers so what happens we want to make it better right? So we want to make the data scientist first-class citizen developer in all our DevOps practice so we're looking into No, starting making this process be more Reproducible so now you want to reproduce these over and over again and making And automated as much as possible. So instead of just throwing away the the code from all the Laptops of your data scientists straight into production and you know, if you ever work with machine learning studio, that's what happens Machine learning studio is fantastic because you can build your machine learning experiment with low code or with UI that simplifies your machine learning experiment and then just with that one click Publish into production publishes a rest API. You have a public endpoint to completely serverless hosted in Azure and Protected with a token so high availability high scalability is perfect It just that it goes directly from the dev environment Which is in the browser in Azure machine learning suite or with a click and controlled into a public endpoint So where is the DevOps in there? Where is the auditing? Where is the control that we want to have to make sure that the correct version is deployed that the model is Properly trained that what the any changes version and so on and this is where the the operation on ML comes into action. So instead of just throwing the code Into production straight away, you want to check your model into a model store. That's the difference and also is like a Source code repository, but it's specifically for models for all your train data set and What does this model store does? Well, it compares models against the different models Variation of the models over time so you can add a validation step You can you know, you want to validate not just after you check the modeling But after you roll it out into production you make sure that the things haven't changed and finally after you roll it out into production you want to use Sophisticated the deployment tools like a bit testing or canary and they all feed this information back So collect the limetry collect performance of the model and use it for retraining Improving the accuracy as I mentioned before and then you carry on with the process of publishing these over and over again This is where ML Ops helps in bringing the power of automation in DevOps to machine learning as well In Azure, this is extremely simple and extremely integrated This is the beauty about it because we are talking about Azure machine learning studio We are talking about machine learning virtual machines or containers running in Azure But we are doing this also in the context of Azure DevOps So the two tools really talk together in a very very nice way Now there are a lot of Combination of tools that can be used But you know to make ML Ops easier in some way already today you can start with your DevOps pipeline and You know automate the deployment of machine learning training services There are a few things that I want to mention from this slide, which I believe are quite important First of all, you don't have to do everything necessarily in the cloud Meaning with the machine learning studio as a low code environment you can also use any SDK Python SDK or R SDK and Use ARM templates For building your infrastructure as code So whatever you're using is a CPU or a GPU for some specific framework It's all completely automated and it's part of the deployment The build and release pipelines So speaking of the pipelines You may be familiar already with the Azure DevOps pipelines the support of a play through our languages the platforms any cloud extensible with different frameworks and also support obviously for containers the like AKS for example now in this context As we'll see in a moment yet the possibility to Build the tasks that deploy your infrastructure for running a specific GPU that contains a specific framework in Azure or to build a Release pipeline that publish a specific endpoint that you have a previously Built with the Azure machine learning studio and Moving forward going to the most specific. I have just a couple of slides and I'll show you this in action There is a Sort of creating an awareness of machine learning into DevOps. So think of Azure DevOps being really ML aware and All these elements are now part of the different releases the different stages and like you would normally do with any code like in Node.js in C sharp in In JavaScript whatever programming language you use for your web or mobile application It will do a subtly the same for a machine learning Experiment and model. So that's probably The the advantage and the difference from the traditional DevOps to the ML Ops Not only you're controlling the code itself You're also controlling the models the versioning of the models into this model store All right, I Mentioned before let's make the data scientists like our first-class Developer so they can also benefit of the automation tools. This is what happens with Release pipeline so the pipeline that can capture all your All your infrastructure requirements unit testing code reviews and any Any tasks that you really want to automate and there are tasks that are specific to machine learning like training the model itself and Scoring it evaluating it retraining it if you are familiar with Typical workflow of a machine learning experiment You know that there is an initial step where you load the data set and then you split the data set into what is meant for training and what is meant for Prediction so of for testing then you have the training process itself for the scoring Which is the prediction part of it and the evaluation which compares one? Algorithm one train model to another one and look for what is the best result in terms of accuracy in terms of recalls in terms of Other factors that identify the performance of an algorithm over another one so all these Conditions that are typically done by a data scientist Looking into a tool like Azure machine learning studio that provides all this information on screen But they are a manual task. You can automate as part of a release pipeline and and publish these also as a Release note. So this is the the benefit that you get into these automation part of it and Yeah, so this is some way summarize what I just mentioned that you know you package your model and you can also Use containers. So there is a very strong integration with the cloud infrastructure as well If you are on a path or completely serverless platform Absolutely fine to use the publish endpoint in a machine learning studio but if you're running some Custom code or some specific frameworks like pytorch or tensor flow then you may want to run this into a specific container and Obviously you have all the benefit of Azure in managing your own container cluster Starting from an image that is already pre-configured can publish the model validate and and run all the other steps from for for deployment and Monitoring on AKS on Azure Kubernetes services so at the end from From a lifecycle perspective You have your automated deployment You can package it you can You know you start with with your board with with your product backlog item with your code you build it deploying publish into operations environment into containers into Serverless and have a Monitoring capability all across it. So you really have an end-to-end Solution for your DevOps also on machine learning So all the tools that you are already familiar you can totally use them as In a machine learning experiment with the addition that some specific pipelines for build testing and release are optimized for machine learning and code and models because Besides having the traditional build and release pipelines you also have a training pipeline and a model management as mentioned before so the possibility to manage the a model version and All the changes in the model store and the possibility to train Retrain the model all part of the same pipeline and this pretty much summarize what I mentioned from End-point perspective all this information is also in GitHub. I'll send you the The URL the GitHub repository at the end in my last slide. You have the entire ML Life cycle managed. So let me go now and show you Very quickly some of these Inaction I have MLops demo starting with a video anomaly Detection code this code is all on GitHub on the Microsoft repository So it's all public open source free to use it. I have a bit of a Python code I mean to mean a bit here, you know, you can do a few things like Make splitting the images Recognizing some specific components of the image itself. So after I get all the different split also basically Subsection of the image I can do a bit of optimization Unfortunately, we don't have the time to go into all the details of the code I just want to show you a few things like the training part of it Here is where you start the training and then after doing the training configure a bit of parameters there for Optimizing the performance of your train model and these are all documented on the GitHub repository and Then go into the execution itself of the pipeline So now I want to define the pipeline that I start coding and look at this may be a bit Overwhelming if you've never done Python before but your data scientists, I promise you they will know where to put hands This is an ordinary code for a data scientist that is able to write Python code But you see the steps so initialize the model run the the training process itself and then at the end Monitoring also the performance with some matrix that Understand the weather the model is performing the way is expected to perform And then if I keep on scrolling here, here we go we go into the pipeline itself. So No, sorry, that was the matrix once I go to that I find the pipeline Yes, here is where I registered my model and then I so once the model is registered Is added to the pipeline and I can start deploying whether it's a GPU or a CPU or a container with specific configuration. This is all part of the The config the the configuration of the pipeline itself Which is deployed infrastructure as a code. So for example here I'm also storing some video data in a blob store. So I procure a provision of some additional Infrastructure in in Azure and I can control access Completely from code. So I have the possibility not to automate these Pipeline of a build and release using Python in this case. You can do the same also using our Although Python is getting a particularly popular Lately in the machine learning studio and here we are at the end. I run the pipeline So the pipeline is built I can validate and I can run it and they will run in My source code repository. So I'm here. I have a list of all the experiments that I built And the pipelines that they have so if I dig into one of these I can see the performance of of each execution So I need to dig into that see all the steps that have been completed and By step and see what happened each parameter each stage of this step Gonna go a bit quicker here. So back to my experiment. I Can show you basically how the performance is represented on a graphical level and How the execution of the pipeline can also optimize the deployment of Resources so pipeline Let me be the new pipeline here The first thing that I need when I build a pipeline is to define obviously the resources of the artifact that I need in this case I am building a Machine learning experiment. So I configure my service endpoint and after that I can add another artifact which is the Where the code is the repository itself all is in Azure DevOps, so it's all integrated. So it looks easy to configure If you are familiar with the process, you know, you select the source code repository You select the branch and then you just add this artifact to the pipeline And after the artifact there are different tasks that you can execute to the one that I need There is the Azure CLI because with the CLI I can run command directly on my Azure portal so I can use these CLI command extension for creating any resource as the Python code was As implemented before There are some initialization script that contains some Some parameters that I need to provide and again this will be all provided into the The the GitHub repository, so I'm doing a bit of copy and paste here to make it a bit quicker and once I have my deployment Pipeline I'm basically ready to go just save it and I can then Release it and and the release will carry on in the usual way I have the pipeline I define the artifact and this will Start will spin off for some Containers because in this case Remember in the Python code. I have allocated some containers or the blob storage and it will run Very quickly and hopefully will show It is will show how all the containers have been deployed. So not Not extremely sophisticated way of working. It's very Linear if you are familiar already with With DevOps and the only thing is that you have to control these Azure resources with the CLI so using Python or are From a code perspective, but this will help you to automate the process itself now On GitHub repository, you will find this diagram that basically Represent in one picture everything that I just said and this is a great resource to start I know that not all the boxes may be Readable from here. So let me zoom in in and go step by step The first part as I mentioned is the training part of it part of it So it's a data scientist that you want to know update the code and publish into your repo and Then then build your pipeline and this is all about training your model and after that There is the process of evaluation and registration of the model That is then published to a pipeline a release pipeline So before it was a big pipeline and then the second part is the release pipeline Where you have an opportunity to decide which environment? Which infrastructure on whether he's a container so whether he's going on a KS or Completely pass is completely up to you completely automated you define the steps and then it just go one by one So get started. There are two GitHub repository on the Microsoft account ML ops and ML ops Python, which is more optimized obviously if you are familiar with Python where you can find all this information and That's it. It will help you get to start all open source plenty of documentation and if you need obviously any support any guidance, I'm really Available to help reach out to me social coordinates on screen my Twitter and Thank you very much. Hey, Stefano, can you hear us over there? I'm gonna switch to Q&A. Can you hear us? Okay? Oh We can't hear it. Can everybody hear Stefano out there? We're trying. No, you're fine. I we're trying something on our side of here and someone on the chat Say we're confirmed that you can hear myself and Stefano We can't hear him. That's what I was afraid of. All right, so we're gonna get the echo back here Hey, can go ahead and speak Stefano now. Yes. Perfect. Now we can hear him, right? Yeah, I guess where do you guys hear the echo still? la la Yes, sweet. So we're gonna wrap this up. There weren't any questions. So I think everybody was scared It was great content Thank you so much for for sharing this from sharing the slides everybody. We are going to be putting this on On a github repository with all the slides for all the speakers or the for the conference So if you want to go look at some of that later on, uh, we'll well, that's where we'll find it We'll post it on twitter and I think I'm sure we'll probably blog about it. Oh, yeah, definitely. Yep So Stefano, thank you so much for taking the time to speak with us and sharing your knowledge Everybody we're gonna get we're gonna hang up here Stefano and get david up and going. Thank you so much