 So next up is what's new in ML.NET with Cesar. Thank you. Cool. So let's get started. Welcome to this session about Machine Learning.NET. My name is Cesar Delatore. I work as Program Manager in the.NET team and ML.NET team. The goal for today is to cover what's new about ML.NET since we launched it in last May. So if you want to get a comprehensive introduction about ML.NET, many things that we don't have time today to talk about, you can listen to the session that we had in build in May, and there's a link. But today we're going to be focusing on what's new, and very short intro about ML.NET, but then focusing on things like the new database loader, deep learning, what new things we have with image classification, and training on TensorFlow, and finally a sneak peek about Jupyter Notebooks as well with ML.NET. So let's start with a quick intro on ML.NET. First of all, ML.NET is most of all a framework, and it is cross-platform and open source, and it is built for .NET developers. We aim to make it simple, and so you can create your own machine learning models without having a lot of knowledge on machine learning. Of course, then you use just .NET natively. Then another important point is that with ML.NET, you can create your own models. So it's custom AI, a supposed or pre-built AI where you just consume services. Here, you are creating your own models for your own business domains and your own data and images for instance. Then we are also extending ML.NET to other frameworks or other libraries like deep learning, TensorFlow and many tools that I will be showing later. Finally, it is trusted or proven on a scale. Because even when we launch it publicly in May this year, there's been around more than eight years that ML.NET has been used internally by Microsoft in large products like Bing or Windows or Office or Azure as well. A few products in Azure are used internally for many years, ML.NET. So you can go and search more about in there. Another point that I want to highlight about ML.NET is that ML.NET because it is a framework, it runs anywhere. You can run it on-premises, on your servers, on your laptop, or in Azure, or any Cloud anywhere you want, and because it is cross-platform, you can run it on Windows, Linux, or Mac OS. So the frameworks that we support are .NET Core. That's why it's cross-platform. But also for legacy applications, you can also infuse machine learning into those applications with .NET Framework, and we even have Python bindings with Nimbus ML. So you can generate also ML.NET models with Python, and we support 64 bits and 32 bits. So what can you do with ML.NET? There are many scenarios that you can do. You can create models for having text about customers feedback and create a model for sentiment analysis, or for recommendations about products, or predictive price of something like a house, or predictive price about a taxi first, like the demo I'm going to do later, or anomaly detection, etc. You can go to that URL and see many samples that we have in there and do any of those scenarios. Then the other important thing about machine learning is that it's not just like building an application. You need to focus first on the model, on the data, which is what you see on the left. You need to prepare your data and build and train your model, and that's the back-storage process that you do as a model creator, as a developer, and you use tools on the CLI, maybe or console applications to build a model. Once you have the model, which is a zip file when you service that, then you can deploy that into your end-user applications in production. So that would be the model consumption. So there are two very different cycles when working with machine learning. Then what are the ways you can use ML.NET? Basically, we have three ways. One is just use the plain API in C-sharp or F-sharp. You can research the docs, so you can go to the samples and do a similar case than the sample and just write your code and train your model and then consume it. But then we also have a tooling to make it a lot easier especially when getting the start-up. You can use model builder within Visual Studio or the CLI, which is doing a very similar thing that model builder, like training your model and generating that for you. So I want to highlight that the easiest way to create an ML model is precisely using a model builder in Visual Studio. So I'm going to do a demo about that. Basically, when you model builder is right now in preview, so you need to download the V6 extension and install it on Visual Studio, and then you can run it in Visual Studio and create a model. So I'm going to do a demo about that very quickly. Basically, you can see here, I have just Visual Studio and the console app is completely empty. From there, I'm going to right-click and add the machine learning model. So this is like a wizard where you have different scenarios or custom and you can get into machine learning tasks. But for this demo, I'm going to do the price prediction scenario, which is under the covers of regression. I want to predict what's going to be the prices or the taxi first of taxi trips, depending on input variables like distance or time and so on. So basically, you can load the data and train the model with that data from a file or from a database. In this case, I'm going to use a file and the file is in here. It's basically a CSV file with a bunch of data about many taxi trips, how long were those taxi trips or how long in time were there, or and then of course, what's the fair amount in the past. So with that data and we have here like 20,000 records or more, we can train the model and I need to select what's going to be the column that I want to predict on, which is the fair amount. So with that, I can train a model, let's put 20 seconds and start training. So something interesting here is that it's not just training with a single algorithm. Under the covers we are using AutoML, which is looking for different algorithms and hyperparameters combinations, and then selecting the best model for that dataset. So in this case, just in 20 seconds, it's been looking for a few models and you'll see that the best algorithm in here has been for so far, like GBM regression. But you don't really need to know all these algorithms when you use a model builder because it's doing that for you all those tries and finding what's the best model for you. So now that the training is complete and this is just a demo, if you have a huge data and then you might be doing this training for minutes or even hours, but in this case it's quick because it's just small data. In here, when you go to the evaluation, then you can see what were the models, the best five models and you can see the light GBM regression was the best one. Finally, we can generate the code and the model. So if I click here and add projects, we get these projects added into your solution. Here you can see the model.save is precisely the serialized model that was for the best algorithm. So you could just use it in code, like in the code we have here generated for you. Just in a couple of lines and I'm going to put a breakpoint, you can see that you can create simple data, sample data, and then predict with a sample data. So let's debug it. Okay. So we just got sample data from a record, from a file, but you could also create your own data or hardcoded or could be coming from your application. In this case, you can see the sample data as input data, and then we do a prediction with that data calling the method predict from the prediction engine. Here you can see the prediction that in this case is like $16. So if I just move forward, you can see that for that test data, the actual value was $17.5, and it was pre-closed the predicted value. But this is not all. It's not just about generating the model and how to run it. We are also providing you the code that we actually use for training the model. So if I create a right model builder and then create model, this is a custom function that is going to create the model. If I go to the definition, you can see here and I'm going to be running this in the background. You can see how it is loading the data. So let's see the code. Loading the data from the file, then we are building the pipeline, doing data transformations like converting all the text into numbers because machine learning is all about math and we need everything in numbers, finally concatenating all the values as input features, and finally adding the algorithm it's going to be using and then training model. When you call fit, then you train a model and then you can save it as the zip file that we were seeing. So you can see here in the actual training, we have the metrics, how good was the model, and then the prediction that we saw previously. So that's how you train a model with model builder. Now let's move to the new things we have in ML.net. Well, before that, just to mention also that with the CLI, the common line interface, you can do the same thing that I just did in Visual Studio. You can also do it with the CLI. If you're just in a Mac or Linux and just an editor and you're not using Visual Studio, you can also use the CLI with a similar line or command below, ML net, auto train, the task, where the data and what you want to predict, and then you will get the code generated on the model in the same way. So let's see what's new in ML.net. First thing is the database loader, and basically what you can do with the database loader is just three, four lines of code. You can access a database and use it for training, and it's going to be loading streaming data, can be huge amount of data, and it's going to be used for training your model as needed, right? You can use it with a SQL server or any relational database supported by system.data, so it could be also Oracle or Postgres or MySQL, etc. This is in preview, but it's been used for a few weeks by customers, and let's see a demo about that, and how it is to use a database instead of a file like we were using in the other demo, right? So in this case, I have a project that I generated from model builder. It's pretty similar to the previous one, and I just made a change in the class for loading the data because I don't need the attributes, but then the only thing that I also did is I commented the code that was using for loading from the text file from the file, and instead of doing that, we're going to put here the code for loading data from a SQL database, right? So this code is very simple, should be very familiar for any .NET developer. You can see that I have a connection string here, in this case it's a SQL localDB, but it could be any other database. I have the SQL statement to my table where I have the same data that I had in the file in this case, and then create a database loader and finally provide those parameters. In this case, it's SQL Client, but it could be Oracle or it could be any other database, and finally it will load that data into the iDataView, and the iDataView is able to stream data as needed when training, right? And then the training is exactly the same code doing data transformations and then calling fit, so that's nothing new. So if I just run it, you will see that it's training the same way, but now it's reading from this SQL database that I have in here. Just going to show you. So now it's cross-validating and training and getting the metrics, and it's doing the same thing, but it gains the SQL data, right? And let's move on to the next thing. Okay, so another important area where we are focusing a lot lately is about deep learning in ML.NET. So deep learning usually you use it for scenarios like computer vision, scenarios with media. Like you want to predict things about photos, images, or video, or sound, and then deep learning is based on neural networks. And so you have also some entry data, and then, for instance, you could predict if a picture is a dog or not a dog based on images instead of tabular data like we do with classical algorithms in ML.NET. So the leading libraries in deep learning are, for instance, TensorFlow or PyTorch and also about interoperability Onyx and their Onyx runtime. And our strategy is to integrate ML.NET with TensorFlow and Onyx and probably in the future with Torch. And so, but then creating a higher level API and that you can run also locally and create your own models for image classification, for instance. So before getting into the scenario, I want to highlight kind of a little bit of insights. So in the market, or in the deep learning world, many organizations like Google or Microsoft or universities have created complex DNN or deep neural networks, architectures and pre-trained models like Inception V3 or ResNet or many others focusing on specific scenarios like image classification or object detection. And they train those models with a huge amount of material or of images. And then you can use those models in ML.NET. The difference is that later on we are going to be able to train and do transfer relating with those, right? But let's start with a simple scenario, image classifier, right? So you could use one of those pre-trained models and then you get a picture and it says it is a dog or it is an elephant or it is a flower, right? So you would be using any of those pre-trained models like Inception V3 or ResNet and you can just use that model from TensorFlow and just score it or use it in ML.NET. You can see the samples that we had previously. And same thing for object detection, which is not the same. Object detection, you provide a photo, for instance, and then you detect different objects, like in that case, a dog or a person with the same object or the same photo. And same thing about this case, a cat or a dog within the same picture, right? But in this case, it's just about using those pre-trained models. But the important thing, this is the key question, is what if you want to classify images based on your custom domain, based on your own images and your own labels, right? So for that, you need to train your own model. And for instance, what if I don't want to just say, hey, this picture is a flower? I just want to say, I want to be more precise and say this is a rose or this is a tulip, right? So for that, you can train a model in ML.NET. And what we're doing is using our approach called transfer learning and kind of deriving from those pre-trained models and then train on your own images, right? So let's, I want to show you very quickly that you can do that with model builder. And here I have another sample where I just have an empty project. I can add machine learning again. And in this case, I will click image classification. I select the folder where I have the images. So this folder with images basically has a one subfolder per class or per different type of images, like they see it's one subfolder or roses or some flowers on tulips, right? So we basically are gonna say that we train this model based on those classes, right? So this would take a few minutes. I wanna move on and show it how we also are working on the API and doing the native transfer learning that I mentioned, right? So basically what we are doing is we are using the internal or low level TensorFlow Bidings for C-Sharp in TensorFlow.NET. And with that, we are retraining your model using transfer learning. So those models like inception of the three maybe was trained with many images, many photos that are comparable similar to the ones you want to use. So we are taking that knowledge from that pre-trained model and then retraining with your own images in the last layer of the TensorFlow model. But that means that we are really creating a TensorFlow model under the cover. So it's a deep learning training as well, right? And there's a big benefit that we are doing with this API for image classifier, which is we are doing that transfer learning for image classification in a single line or maybe three lines with all the parameters that you can see in here for all that transfer learning, right? So just with this code, we are able to do all the transfer learning and then finally generating the TensorFlow model, which is a PB file and the ML.NET model that you can use, right? So I'm gonna run the sample and here you can see the code using the API and the new API, which is image classification where we are providing, hey, I wanna use the, in this case, this pre-trained model ResNet or it could be inception with free and a few parameters on if you wanna use the callback to get feedback and so on. So I'm gonna debug and start running the training. It'll take a few seconds while I'm showing you the images. So in this case, it's gonna be training with around 400 images, right? Like four images per class and if you compare this code with the code that you'd need to use by just using TensorFlow and low-level API would be like 400 lines and in this case, we are able to do it by just adding this single call to this estimator image classification, loading the images from the folder, specifying the architecture, saying how many tries you wanna do on those images with epoch and that's it. And here should be the training starting. So it's gonna be, now I did the first epoch, it has like a 80% accuracy and in a few seconds we'll get many more epochs or training with the images. And finally, we would get the predictions that we have. So I'm gonna keep going because we have more content that I wanna cover, okay? And another thing is about data exploration and machine learning experimentation and formats that are good for learning like cursors and hands-on lab, right? So if I talk about that machine learning, what would you think? I probably would think about Jupyter notebooks, right? So this is what we are working now as well. We are providing support for .NET in general, C-sharp and F-sharp on Jupyter notebooks. Right now it's, we are working on it. If you wanna try ping me through Twitter and I will provide access on how to use it, but the public preview will be coming pretty soon in a few weeks, right? So basically with this support, we have a Jupyter kernel for .NET which is based on .NET Try and we can run ML.NET code. So I wanna show you a demo where I have here Jupyter installed in my machine. You can install it very easily with Anaconda and then launch it. And after that, I installed the kernel, right? So once you have it installed, you can see that now I can create a notebook for C-sharp or F-sharp. And here I have one of the notebooks that I created using ML.NET. So first of all, you can see here that I'm installing the NuGet packages that I'm gonna need for ML.NET or Xplot for plotting data and then that would take a few seconds getting those NuGet packages. Finally, I'm declaring here the input data classes and here I can have the code that is loading from the file like the previous code that you just saw but I can just run this piece of code and then show the schema and it's running for real and then showing data about how this data set is like I have these fields and or I could also show a few records. For instance, in this case, I see that what kind of data it is. I could do data transformations and show it later and even further, you can also use Xplot in plot charts like in this case, the distribution of the taxi trips per cost. Look, I can see that for instance, most of the taxi trips were in between five and $10 or I can also plot other interesting information about the time versus distance and then different color depending on the first. So I can see that the fur is pretty much related in this case to the distance and the time, right? And or if I just wanna see the relation between the taxi fur depending on the time or depending on the distance and but then depending on the passengers, for instance, I can see that there's no real relationship here, right? So you can get a lot of insights when plotting data in Jupyter and then of course you can also run and build your model here. So here we have the ML.NET pipeline for this model and I could just run it here and see the data transformations or I could append then the trainer and call fit and see how much time with this instruction, percentage, percentage time and then I see that it took three seconds to train this model, right? Or I can do predictions and then display the metrics and you see it in a very neat way in this document. So basically you can document your exploration or your model creation in a very neat way with Jupyter notebooks or finally I could see kind of how are the predictions versus the actual values or even in this case, it was a regression so I could be plotting a regression line like in this case, what's the regression line and where are the predictions versus the actual values, right? So, or even saving the model. So just highlight, stay tuned for this. We're gonna be working on it and pretty soon it'll be available in preview. And with that, just finishing, just wanna highlight that there are many other important features in ML.NET like for instance, how to take into account your model lifecycle in your CI CD pipelines in DevOps, not just for your application but also for your model or model explainability and future importance so what are the most important input columns that you have for your model, cross-validation or how to deploy your model into ASP.NET core applications or Azure functions. And by the way, also wanna highlight that we have a ML.NET YouTube playlist so you can take a look to many short videos about ML.NET in YouTube like one single video just about cross-validation or one single video and I'm pretty short to talk about how to deploy into a web API or things like that, right? And about the roadmap, I also wanna highlight a few topics. We're gonna be evolving and investing on deep learning not just for image classification like I was showing but also for object detection and doing this natively in deep learning in terms of flow. We will be moving the database loader as GA, improvements for Jupyter and moving that into a preview. And then we are also working a lot with the Azure ML team because that's all another important scenario on how to integrate ML.NET with Azure ML. And finally just a few links where you can get ML.NET resources like how to get started or where are the samples or the docs and the videos. And with that, I really thank you and that's all, thank you. Questions, if you wanna stick around so you thought that was really cool, I always like learning more about ML.NET. What's happening? I benefit from all of these machine learning tools but I don't actually get involved with the models, I just rate the rewards. Okay, so let's see. Well, ML.NET, whoa, whoa, whoa. I hit the wrong thing. The question was, will ML.NET support CNTK or the cognitive toolkit I think? Is that what CNTK stands for? Yeah, CNTK, yeah. So initially we don't have CNTK in the roadmap. These efforts are right now on TensorFlow, Onyx and could be also torch in the future. Okay, very cool.