 I'm standing between you guys and lunch. My speaker notes didn't come up, so I'm going to wing this so that we can get done on time and everybody can eat. So good morning. My name is Audrey Resnick. I'm a senior principal software engineer and data scientist with the Red Hat OpenShift Data Sciences team, and I'm going to talk to you about what's the deal with managed services and model delivery. So if you're a data scientist or if you work with a data scientist supporting them, you'll know that when they create the model, there's a lot more than to creating the model. You want to be able to get data to the model. You want to be able to deploy the model, monitor it. So we're going to go into some of those items. We'll take a look at a model's role in an intelligent application, kind of get into what managed services are. Sasha, thankfully, went ahead and covered some of that, so I don't have to go into detail for that. Thank you. We'll take a look at who uses managed services, and surprisingly, it's not just the data scientists when we're talking about intelligent application creation. And then we'll kind of look at how managed services help you along with the model delivery, where you find them. And finally, I'm going to click through a very quick demo of the Red Hat OpenShift data science platform and its managed services that are available. So when we take a look at intelligent applications, we have to take a look at the model's role in them. Now, intelligent applications by themselves are not just one small thing. They are a distributed system. So there are things that work in conjunction all the way from data verification, serving the infrastructure, going ahead and doing some configuration. And when we go ahead and take a look at these intelligent applications, we'll see that the model code is just a very, very small part of that. And the model code, or the model itself, has to be able to make its way through these distributed infrastructure. So it has to have a way to interact with things such as feature extraction. It has to be able to interact with some of the analysis tools. And you look at that and go, wow, that could be, like, really complicated. That could be, like, kind of hard. So is there a way to easily create a model and be able to use this thing called managed services? Well, there is. At Red Hat, we went ahead and we created something called the Red Hat OpenShift Data Sciences Platform. And I'm going to go into detail with that. But that's just an easier way that we can go ahead and help the data scientists be able to go ahead and deliver their model and monitor it. So when we take a look at managed services, we can divide them into four groups or four categories. And within those four categories, we actually will have a number of personas that are going to interact with them. So if we take a look at the first category, we want to go ahead and gather and prepare the data. So that means we're going to look at data storage, data lakes, data warehousing, stream processing. And it's really our data engineers that are going to get totally excited about this category of managed services. Then we go ahead into actually developing the model. So when we go ahead and develop the model, we're going to bring the data scientist in. And they're going to go ahead and create the model, work with the algorithms that they need to solve the particular business problem that they're trying to solve. I just want to mention that you'll notice at the very bottom, we have IT operations keeping an eye on everybody. We'll get back to that. Then we want to be able to deploy a model in an application. We can get in with CI CD pipelines, and that's where you're going to have your application developer or machine learning engineer helping out. And then finally, we want to look at model monitoring and management. So we want to see if that model that we've deployed has any drift. Is it giving us some of the answers that we thought we were going to get, or do we have to correct it and retrain it? And that's where both the data scientists and the application developer or machine learning engineer will come into conjunction. Now having all of these services and having them available for everybody can actually be a nightmare for IT operations, right? You want to give your users the latest bells and whistles, but at the same time you want some sort of platform, or you want some sort of services that you know you can be, how do I say it, very comfortable with. You can depend on those, and you know that they're not going to help create any outages when they're actually trying to help create a model. So let's go and take a look at kind of the model lifecycle and where these managed services fit in. Now remember I told you there were four. We wanted to kind of extract and transform the data. So we can actually go ahead and instead of building something ourselves, with the Red Hat OpenShift Data Science team, we said, well wouldn't it be cool if we could just go ahead and invite a whole bunch of different vendors, open source vendors in, so that that way you have a lot of choice. So when you're extracting and transforming the data, yeah, you could go ahead and use Apache Kafka Streams to go ahead and pull in some of your data. But wouldn't it also be cool to use somebody like Starburst Galaxy, so that you could go ahead and curate your data. You really want to unlock that value of your data by making it very fast and easy for you to be able to access that data across the hybrid cloud. Next we want to take a look at creating models. So we want to be able to either use a Jupyter Notebook or something like that for some exploration. But maybe at the end of the day, we're really interested in what Anaconda has to offer because they might have an extensive set of data science packages or libraries that we could use in our Jupyter Notebook projects when we're going ahead and doing some of the experimentation. The experimentation we're coming up next. So another one of our internet service vendors or IS fees is IBM Watson Studio. So when you're doing this experimentation, you can go ahead and use IBM Watson Studio to see if you can manage your AI models at scale and be able to see if there are any issues when you're trying to deploy them at scale. Now when you're going ahead and you're done with your model and you're testing and everything done with your experimentation, what you want to do is actually go ahead and deploy those models as actual services. So you can use an ISV such as seldom deploy and it's going to really help you simplify and accelerate the process of deploying and managing your machine learning models. And finally, when you get your model out there, you want to be able to make sure that that model is performing optimally. So you want to be able to monitor that model performance and you want to be able to glean any meaningful analytics from the model. Now, this whole path that you're seeing this curvy path is kind of the model operation life cycle. I want you to keep that in mind because we need to see where would these data services or sorry, managed services actually live. So we're going to start with the Red Hat Managed Cloud Platform. We want to have a platform that is very stable that will allow us to work not only from a hybrid cloud, but say on-prem public and even to various edge devices. Whoops. Next, we'll have Red Hat Managed Cloud Services. So these are the cloud services that we provide to our customers and you'll notice that kind of in the center there, I don't know if my mouse will work, we have the Red Hat OpenShift data science platform. Now on top of that, we have what we call our ISV Managed Cloud Services. So these are internet service vendors such as Starburst Galaxy or Anaconda that we have into our Red Hat OpenShift data science platform so that you can use some of the OpenShift services that they have. And then we have customer managed ISV software. So if you wanted to say, for instance, in a model to take a look at quantization or go and take a look at inferencing, you could use Intel OpenVINO, which I apologize, I forgot to put the icon up there. You can use sell and deploy. Now, remember I told you we had those four categories. Those four categories kind of sit on top there. So you have gathering and preparing the data. You have developing the model, integrating the models in application development and model monitoring and management. And of course, you're going to be able to retrain the models. So this whole Red Hat OpenShift data science offering actually sits on AWS. So it's a cloud offering right now. And what I'm going to do is probably go through a demo. I think I have enough time for that. At least I won't have to have it as a live demo. But one thing that I wanted to mention about this entire platform is we have the depth and scale basically without lock-in. So the capabilities that we have are really in conjunction with Red Hat and our service partners that we brought into this ecosystem. So that way you have this managed cloud platform. You can use the Red Hat portfolio and services. And you can take advantage of open source products through our partner ecosystem. OK, this is going to be very quick. I'm glad I'm going to be clicking through it. So one of my colleagues actually worked with the London City Metro. And the London City Metro wanted to go ahead and to be able to monitor cars within the metro area. They wanted to be able to recognize license plates and see is that car able to park here? Does that car actually have a tag so that it can use these certain metro ways? Is this car containing somebody who did something bad that we want to track? You get that idea. So here I have a picture of a car. What we're hoping that the machine learning model will do is be able to take that license plate to write it and to actually grab the plate numbers. And then once we get those plate numbers, we can use Apache Kafka to go ahead and store that information, possibly get an amber alert. In the meantime, we'll be pulling a lot of those license plates into our various warehouses and also into our vehicle registration database. And of course, the City of London, their metro services, can then perform more business analytics on the data that we've gleaned. So what does this look like if we are trying to use this Red Hat OpenShift dedicated platform? Well, because the Red Hat OpenShift dedicated platform sits on top of AWS, you are going to need a cluster to actually use Red Hat OpenShift data science or what we affectionately call ROADS. I know I'm going to hell for giving you the acronym there. So we're gonna click on the ROADS menu option. And then what you're going to see is basically a menu of the managed services. And of course, there will be managed software available for use to actually work with. In the background, you'll see that I've chosen one of those items that is Jupyter Hub. Notice the other ones when I go ahead and hit the explore icon, these are all the different managed services that I can go ahead and choose to use. And there's plenty of documentation, there are tutorials, there are quick starts so that if you want to learn more about the products and you're not very familiar with them, you can go ahead and use those utilities. So I'm going to go ahead and choose Jupyter Hub. And what that's going to allow me to do is to go into a Jupyter Notebook image. I'm going to take this Notebook image and then basically wrap it up in a container so that I can deploy it on OpenShift. But I want to customize it for myself. So the first thing that I'm going to do is say, okay, if I'm doing some machine learning, what am I going to be working with? Am I going to be using just a standard data science package which may contain things like NumPy or Pandas or Scikit-learn, or am I going to want to work with something such as this license plate detection where I may have to use a lot of the PyTorch libraries that are available? So I'm going to click on PyTorch. And then I'm going to take a look, whoop, sorry, on TensorFlow. And then I'm going to choose a container size. I'm going to choose a large container size and this is also going to give me the ability to choose the type of RAM and CPU that I want to use. If I'm working with S3 storage, I have the ability then to go ahead and enter the credentials because we don't want to go ahead and put in the access keys and whatnot into our actual code. I know people that do that, it's fun to hack into their code, but we don't want to do that so we can use these environment variables to save that for you. And then we're going to start the server and we're going to have that server spin up. And then we're going to pop right into JupyterLab. So we've given the data scientists now the ability to have an environment where they can go in and create their work. What they probably want to do is to clone a repository that already exists. In this case, we're going to be cloning the license plate workshop. And then when we go ahead and clone that, we will see all of the files that are associated with that. What we'll do is we'll do what a data scientist does. We'll go ahead and start experimenting within that notebook and see if we can actually create a model that will be able to successfully extract that license plate and to actually tell us that it's been successful in extracting that license plate. Now, if I want to go ahead and deploy this application, you're not going to deploy this application as a Jupyter notebook. I know people who have done that, please don't do that. What we want to do is really package the model as an API. In this case, we're going to use Flask to help us accomplish this. Then we'll go ahead and launch our server to see if we've been able to successfully deploy something internally. Then we'll test that Flask app. We have a status of okay. And now we're ready to go back into our OpenShift dedicated environment where we first launched the Rhodes platform from to see if we can actually go ahead and deploy this now on OpenShift. So we're going to create a project for ourselves in OpenShift dedicated. And then, yes, you guys have been good. You've been checking in your code as you've been working on your model. So we even know that everything within the GET repository is perfect. Sometimes this can be very hard for a data scientist because they like to save all of their code on their local laptop. But you as the machine learning engineer who is working with this data scientist are going to encourage them to check their code into GET. Yes, you are. All right, so we're going to go ahead and now from the GET option, we're going to have some other things that we can do such as the resources and advanced options. What we're really interested in is making sure that we click on the routing option because we want the route of this API. We need to be able to access the API from another location. So that very important route, we're going to go ahead and copy that, probably test that within a browser to make sure that we can actually hit that API. And then if you want to, you can go back into a Jupyter notebook and then using that route, you can use either a curl or you can invoke a web request to actually see if you can hit that API successfully. And then to test your deployed AI ML application, you can take that route and even go back into a Jupyter notebook and you can put in the actual API address. You're going to give it an image. So in this case, I'm giving it an image called car.jpg so that it will see if it can pull the license plate number from that image. And wow, it actually worked. We actually have a car and we were able to successfully predict the actual license plate number. So all of this is on Red Hat managed cloud services. And again, this demo was concentrating on the Red Hat OpenShift Data Science portion. And remember we're trying to, with Red Hat OpenShift Data Science is to have a platform that's fairly open so that if you have a specific open source vendor that you like or if you have specific requirements where you want to use not only Red Hat products but open source products, you should have that choice to be able to do that. So what did we learn today? Well, we learned that managed services or in particular managed services for data science are really important to a data scientist. They're just not going to sit there with their model. They have to have some way of actually going ahead and deploying their model, training it, testing it. And they have to do it in such an easy manner that they can accomplish that task themselves. And of course, IT operations will be there to help them with that journey. I think I got finished in time so that I'm not preventing you from lunch. So thank you very much.