 As organizations apply CSED principles to model development lifecycle, they're increasingly looking at application platforms powered by Kubernetes. My name is Will McGrath and I'm a go-to-market specialist for our OpenShift data science offering. And I'm joined by Abhinav Joshi. Abhinav, do you wanna introduce yourself? Yeah, absolutely. Hi, everyone. Thanks for joining us today. My name is Abhinav Joshi. I'm a senior manager in the OpenShift Product Group at Red Hat focused on the AIML go-to-market strategy and execution. And I'm looking forward to the talk today. Back to you, Will. Great, thanks. Yeah, we're both excited to be presenting at the Open Data Science Conference West. We're gonna talk through some trends that we're seeing in implementing ML Ops on hybrid cloud Kubernetes platforms. And then Abhinav's gonna quickly dive into several customer case studies. So let's get started. This spring and early summer, we conducted a survey through Pulse, an IT executive community portal. We found several interesting trends, the first of which is really displayed on this slide, close to 80% of enterprise IT and data leaders. So they are deploying AIML projects on hybrid cloud or a mix of hybrid cloud and Edge and on-premise environments. Moreover, open source tooling factors strongly while only 11% relies solely on open source AIML tools, some 95% use some combination of open source tooling to launch their projects. And finally, the last slide we have on some of the trends that we're seeing from these Pulse surveys, AIML has really popped up as a workload Kubernetes. You can see here, it's the number two workload behind database or data caching workloads. Not only that, but data ingest, data cleansing, data analytics you can see on the slide is not too far behind. But what are some of the ML Ops challenges and reasons why AI application platforms powered by Kubernetes are becoming so popular. I'll turn it over to Abhinav Joshi to dive deeper, but let's just look at some of the ML Ops challenges. The talent storage as we've seen, not only for data scientists, but for people like cloud architects and ML engineers, the lack of self-service infrastructure to allow your data scientists to access that infrastructure, GPU environments very quickly and the complexity to really operationalize the AI projects. A lot of these AI projects fail as they start to go into production and have challenges scaling. These are all kind of areas where Kubernetes can really help. And Abhinav's going to dive a little bit more deeply. Abhinav? Thanks, Will. Yeah, so what we've seen with the typical software development project, especially with the cloud native technologies like containers, Kubernetes and DevOps. So these are technologies and the operational practices provide the much needed agility, the flexibility, the consistency, portability and the scalability for the data scientists to be able to develop, train, test and share their machine learning and deep learning models in a repeated way without having to worry about the underlying infrastructure resources. And for the developers and the DevOps engineers, these capabilities and operational practices, then they can easily containerize the models and make them part of the continuous integration and the continuous delivery pipelines for the faster delivery and updates of the AI powered intelligent applications. Now, what you see on the screen is a conceptual architecture for ML ops built on containers, Kubernetes and DevOps best practices. In order to execute on the AI lifecycle, what you need is the data engineering and machine learning, deep learning and the DevOps software tool chain. As in, yeah, some of the common examples are the TensorFlow, PyTor, Jupyter and Notebooks, Spark, Python and so on. And also a set of data services to build your pipelines and there's like the SQL server, the NoSQL, the new SQL databases, the data lakes and so on. And all of this has to be supported on a secure hybrid cloud platform that's powered by containers, Kubernetes and DevOps capabilities with the self service like access that like empowers the data scientists, data engineers and the developers to be more agile and collaborative throughout the whole process without depending too much on IT operations for the individual activities. The hybrid cloud platform should have the integrations with the hardware acceleration, say such as NVIDIA GPUs to help speed up the machine learning model development and the inferencing task. And then finally, the hybrid cloud platform that's powered by containers, Kubernetes and DevOps should offer consistent experience like on-premises in the public clouds and at locations and also be able to effectively manage by the IT operations teams. Now, what you see on the screen is are some of the examples of the organizations worldwide that have operationalized the MLRs with containers, Kubernetes and DevOps to achieve the much needed business outcomes. The healthcare organizations such as HCA Healthcare achieve the data-driven diagnostics, financial services organizations such as the Royal Bank of Canada are improving the consumer banking experience. Energy companies such as ExxonMobil are optimizing the oil and gas exploration and the business workflows. Now, let's spend a few minutes to talk about a success that we've seen at HCA Healthcare. Now, previously, the detection of sepsis at HCA's hospitals was done manually by the nursing staff. So speaking of sepsis, it's the body's extreme reaction or the response to an infection. And it is a life-threatening medical emergency. Now, the manual diagnosis of sepsis, it required a lot of time. And also it meant that the patients were only getting evaluated once every 12 hours. But the mortality risk from the missed or the delayed diagnosis is significant. And the team wanted to create an application that would support the nurses in the diagnosis and the treating of the sepsis quickly and not to take them out of the picture. So what HCA Healthcare wanted to do was they wanted to meet a few primary goals with their project. First and foremost, they wanted to improve the detection of sepsis and get to the diagnosis faster. They wanted to put the user-friendly software in the hands of the clinical staff that would help them make better and also faster treatment decisions. And then also they wanted to build a software application that could easily scale across the hundreds of clinical sites that they have. And then finally, to be able to collect the near, the real-time data from the hospitals to support the decision-making throughout a machine learning model. What they thought was they could do this by giving the data scientists and the developers the ability to rapidly build, deploy and update the models and associated the AI-powered software application to help detect sepsis. Now, the HCA team, so they recognized this wasn't just like technology had a problem. So they worked in partnership with the hospitals. They engaged the key stakeholders and the senior management very early in the process and also evangelized the solution to the leadership of the clinicians and nurses. And the team engaged the sepsis coordinators like who are the end users very early in the software development process. And once the app was ready, they implemented a training program as well and like help the clinicians be able to adopt to the new processes. And the application is called SPOT. So that application that's powered by AI has been developed and runs on the open ship. So that's our Kubernetes platform that has the integrated DevOps capabilities to be able to collect the real-time data from the hospitals, which then recommends the actions to the nursing staff based on the outcome of the model. And the result is that a five hour decrease in the diagnosis and the treatment of sepsis and helps them save a lot of lives. The second key example I wanna talk about is the energy company ExxonMobil. They are one of the largest publicly traded international oil and gas companies. So they rely heavily on data for key upstream, midstream and the downstream oil and gas processes as in exploration, the logistics, the refinery monitoring, the detection of leaks, the incident response and much more. And AIML is key to their business. With over 100 data scientists teams, so they wanted to help make them more productive and happy and also make sure they could easily share the models with the stakeholders for feedback. But the existing platform that they had were not sufficient. As we'll talk about, the data scientists were spending a lot of time just configuring the laptops and still could not share the work easily. Yeah, and then the explosive growth in the analytics capabilities and the tools opened up the opportunities for the ExxonMobil team to do more with machine learning. But that also quickly led to like a lot of challenges. The data scientists had to install and configure a set of five to six tools for each of the project on their laptop and some of which required the route or the admin access. As the work got more sophisticated and complicated, the data scientists were spending more time setting up the infrastructure. Then they were like actually to build the models that can help the business. And also there were the inconsistencies with the setup across the tools and also the people. And that created the security and operational challenges for the IT team. And beyond that, it was difficult to share the models with the business users in the field. And the data locality like issues meant that the data scientists often had to take the valuable time to travel into the field to be able to refine their work. And the feedback loop was critical to their work but I slowed down the project or the projects that we were working on at the same time. So what the team did was, so they began by listening to the data scientists and the data consumers to understand the problem and find out how they worked. The key was giving them a much easier way to access the tools that they were already using with a consistent machine learning platform that also met the security needs of the organization. So what they did was they kicked off an iterative process of containerizing the key configurations on Red Hat OpenShift, the Kubernetes platform and also being able to run the POCs with the small data science teams. So they also evaluated the people, process, culture and technology transformation and help the teams onboard successfully to the new platform. The ExxonMobil team, they partnered with the Red Hat consulting team to architect the solution and also to be able to implement the best practices. Now, one of the lessons that they learned was agile development processes could be applied to the data science and the AI projects as well. And with that, they were able to bring the speed of DevOps to the AI projects and to help the business. So now the data scientists are able to create the image files from the secure repository quickly via a capability in the Red Hat OpenShift Kubernetes platform and be able to publish the models to a URL with a single click and letting them to get the feedback from the key stakeholders. So the end results of the deployment have been amazing. The productivity of the data scientists has gone up by 10X and now there are 30 teams on a single platform and the data scientists can focus on writing code and building the models and it's easy for them to share the models and get the feedback quickly from the key stakeholders. And the security and the operations team can also be assured that the software packages and the code that is being built and deployed by the data scientists and the developers are compliant. And then finally, I wanna spend a few minutes on one more success story here that's at Verizon and Verizon Medium. To be able to help the developers use the 5G data insights and be able to build innovative new services, Verizon and it's a division called Verizon Media created a platform called Leo. Think of it as an AI platform that also has the key applications that can help the business and consumers. The Leo provides data insights for the multiple use cases such as the crew safety, touchless travel, video analytics, the drone based delivery that is gonna be like real soon, the smart manufacturing and much more. By certifying the Leo platform on top of Red Hat OpenShift, Verizon has gained the scalability and the responsiveness to benefit from the real-time data from the network edge and also to be able to stay ahead of the competition in a new and fast-growing market. So Red Hat OpenShift, the Kubernetes platform, it provides Verizon a consistent foundation across the different environments at all the edge locations for the innovative app dev at scale and including the automation capabilities and the comprehensive and a continuous security across all the thousands of sites. So now Verizon has been able to successfully migrated more than a thousand containers to the new platform from its previous solution. So at this point, I'm gonna hand it over to Will who's gonna talk about our strategy for AI workloads and wrap things up for you. Back to you, Will. Thanks, Abhinav. It was great to hear how companies like HCA, Healthcare, ExxonMobil and Verizon are really gaining success operationalizing ML through Kubernetes. And I just wanted to spend a few final moments kind of explaining what Red Hat is doing to kind of really make ML Ops real. Red Hat is really not known as an AML software company, but we are a platform company. And so we're investing a significant amount of resources making containers, Kubernetes and DevOps principles real to help accelerate AML projects through production deployments. Secondly, we've been investing in a project known as the Open Data Hub project for some five years now through our office of the CTO. What it does is it really stitches together some 20 different open source ML technologies into a Kubernetes operator. So we have a meta operator that kind of stitches all those together. Technologies like Spark, Uflow, Hive are all kind of pulled together, Jupyter examples. And we actually work with a number of commercial AML technology partners also to provide best of breed Kubernetes, best of capabilities on top of our Kubernetes platform. And finally, we've released a cloud service known as OpenShift Data Science to provide an experimental sandbox for data scientists to quickly develop, train and test ML log models before deploying them into production in a container ready format. So we hope you actually enjoyed some of the success stories that Abenov kind of went through. And if you wanna learn about some 20 other different types of case studies where they actually use in Kubernetes to kind of roll their ML models into production, feel free to check out the stories that are listed here later. It might just inspire you on how you can actually create best practices for ML ops. Thank you again and thank you for joining us. And we look forward to a tremendous conference for you. All right, thanks all. Hope you have a good conference and we'll see you back soon. Take care, bye.