 Welcome to the Cloud Native News Show, airing live from KubeCon Paris. And in today's special AI edition, we are going to be talking about how CNCF projects are playing matchmaker to build a partnership of Cloud Native and AI. My name is Nikita, and I'm your host for today. We've got an exciting lineup of guests and topics straight from the heart of the Cloud Native AI universe. But first, let's dive into today's headlines. The biggest news today is that a new AI working group has been launched in the CNCF, and they've published their first white paper. Tag runtime and tag observability have joined forces, putting an end to the community's most awaited desire. The first gen AI project in the CNCF gates GPT, embarks on a sandbox journey. And the working group plans to add this and more to the new CNCF landscape. Projects have started tagging their releases with machine learning-related features. And to name some, Kubeflow and OpenTelemetry have been leading the way to pass the bait into other projects, and it's like the Olympics have already arrived in Paris. So let the Cloud Native and AI games begin. Now our beloved Fippy gets a new AI friend. The Kids Day workshop comes up with a new book to introduce Fippy's new companion, Kimani. And you can get this book at the CNCF store. Now talking about adolescent years, Kubernetes enters its second decade. I mean, had on, I witnessed Kubernetes literally crawl its way through, learn how to walk, and now run in production. And finally, I'm really excited to see it fly into the AI galaxy. Clayton Coleman, who was just on the panel, said inference is the new web app. And Bob Killen said, if inference is the new web app, Kubernetes is the new web server. Now talking about inference and AI, a new star has been born and has appeared as a cap in the alpha stage. You guessed it right, I'm talking about DRA. So to take us back into how this star came to life, we've got our first news correspondent joining us on the show. So please welcome the DRA whisperer, Patrick Oli. So DRA, what is it? DRA is an attempt to improve how Kubernetes handles accelerators. And that makes it so relevant for AI today. The new API addresses all of the limitations of a traditional device plug-in interface. But DRA in Kubernetes is just a framework. It doesn't manage any hardware itself. It needs drivers for that. And those drivers need to be implemented by third parties. That leads to the big question that everyone keeps asking, when will DRA be better? To answer that question, we first need to look at which part of DRA we are actually talking about. In the original cap, one of the core concepts was that Kubernetes should not need to understand anything about resources. Users ask for resources with parameters that are in a format defined by a vendor. And the vendor driver then looks at those parameters, matches those against the resources that it has on its nodes, and assigns some of those to the user. These opaque parameters are the biggest strength and the biggest weakness of this approach. Once implemented, changing Kubernetes again should not be necessary anymore. But on the other hand, cluster autoscaling needs to understand these parameters. It needs to know which resources are on which nodes. There are some ideas how to plug the necessary vendor logic into the autoscaler. But in the end, we decided to make parameters less opaque. This new approach is called structured parameters. The key difference is that now we have built-in types for parameters and for resource information. And Kubernetes itself manages that. For the user, there's no user-visible difference because a DRA driver can take the custom parameters and convert them into the entry parameters. The vendors choose which approach to implement. With structured parameters, they have to work within the constraints of what is supported by Kubernetes. In return, they get cluster autoscaling support once it is implemented. Only very basic functionality got merged into Kubernetes 130 right before the conference. And more work is clearly needed. So the challenge for beta now is to pick something that is both useful and can be maintained in the long run. Doing everything just may be a bit too much. So let's discuss together what is needed and when it is needed. There's going to be a more in-depth talk later today. You can find me there or at the Intel booth. Thank you. I'm so much Patrick, so much Patrick. And I can't wait to see DRA graduate to beta. I just received some breaking news updates from a ground correspondent straight from the project Pavilion. Can we get connected to the maintainer of case of Dan Sun to learn more about what the mood out there is looking like? Hello, Nikita. So energy out here is electrifying. And all about the crowning of an AI, I attended the color-coded event yesterday. So how we are buzzing with discussions about new cutting-edge features across CNC projects in areas like batch scheduling and image distribution. We are also seeing new faces contributing to CNC projects as AI developers seek to enable their workloads in the cloud. That's great, Dan. So how has it been for case of to ride the AI wave? The case of community expanded significantly as the use of larger language models and a generative AI has grown. This has compiled case over to scale and ensure highly responsive inference for these workloads. With case of transcending to the CNCF, we are committed to collaborating with the AI working group to establish streamlined ML apps lifecycle. I can't wait to see the incredible insights and the connections that emerge as the conference begins. Thank you so much, Dan. So as you all can see, CNCF maintainers are hard at work. So please be sure to drop by the project Pavilion later today. Now to take the cloud native and AI romance further and to show us a sneak peek of what CNCF projects are up to, please welcome our next news correspondent, technical leader of Tag Runtime, and the chair of cloud native AI day, Rajesh Kakodkar. Thanks, Sankita. I come from an environment where scaling clusters has become boring. But now we are in this new world where one has to start with an email to get access to GPUs and then follow up with yet another email until they're caught up in a series of bureaucratic exchanges. This clearly shows that the need of VR is to democratize access to AI infrastructure. You've seen how Kubernetes has sowed the seeds for the ML infra plant. And now let's take a look at how cloud native is adding water to grow this forest of an ecosystem. 44% of the Q-Flow users have been asking for model registries. And the community has heard this feature is getting introduced in the Q-Flow 1.9 release. Volcano has been leading the way to support multi-cluster AI job scheduling as a CNCF LFX project. Taking machine learning to the edge, Wasm Edge has been making strides to run LLM applications across CPUs and GPUs. And Cube Edge's Sedna has been forging the synergy between Edge and cloud with features like unstructured lifelong learning. While on the other hand, you've got OpenTelemetry, which has been working on defining the semantics of observing LLM applications. Looking at all of these projects in the cloud native artificial intelligence space is like assembling the Avengers to be bearers of open source AI. This relationship of infrastructure and cloud native and machine learning is symbiotic in nature, which also warrants a deeper discussion on how AIML can also benefit cloud native. But Raju's trusting Jedi blindly so hard. And that's exactly why integrating artificial intelligence in cloud native requires nuanced discussions around these topics. And yes, while this is all so nuanced, I think there's potential in building deep learning models with Kubernetes data. Absolutely. And I can't wait for more innovative projects to come up in this space. As much as AI is a high-tech Swiss Army knife, but just as a stubborn Charlotte, at times you need bit of a human touch to loosen things up and get the job done. So friends, we can see that this is an uncomfortable space right now, but let's embrace this discomfort as a catalyst for innovation for a boring future of cloud native and AI. And I can't really wait for what that boring future looks like. So thanks for that call out and thank you so much for joining us on the show, Raju. Thanks for having me. So joining us next is the technical lead of working group Artificial Intelligence that we've heard so much about in this conference so far. She's going to spill the tea on what this boring future of cloud native and AI looks like. So please welcome to the show, Cathy Zhang. So much for joining us on this new show, Cathy. Yeah, it's my pleasure to be live on our news show. Thanks for having me. So to start off, you are the technical lead for working group AI. So can you walk us through what the working group has been up to? Sure, and to meet the community interests, the AI working group was formed in October last year in CNCF. The working group is dedicated to establishing initiatives and nurturing community expertise to meet the demands introduced by AI in the cloud. We have been working on the white paper which describes what is cloud native AI and also identifies the challenges and future potential solutions for deploying and running AI workflows in the cloud. The white paper was just published yesterday. Another delivery of the working group is a landscape of cloud native AI projects which provides a view into existing cloud native ecosystem for both AI developers and cloud engineers. This landscape will be merged into the existing CNCF and cloud native AI project landscape. Yeah, and I can't wait to see what this new landscape will look like, especially the AI landscape. So looking ahead, what do you think is the future of this partnership of cloud native and AI? I think the future of cloud native and AI will be shaped by innovations in the areas of cost efficiency and security. And to stay competitive, more organizations will be focused on building frugal cloud infrastructures with high GPU utilization rate and leveraging technologies such as factional GPU allocation and GPU telemetry guided on scheduling and auto scaling. Securing AI models and data sets is critical, not just during the data storage and transfer, but also during the execution of the AI model. Following secure supply chain on software principles will be important. On the convergence of cloud native and AI will drive enterprise AI, HAI and PCAI, enabling processing closer to the data source. This is especially critical for real-time processing in devices such as autonomous vehicles, smart cities, and the IoT devices. Yeah, that's an amazing summarization of what the future holds for us. Thank you so much, Kathy. And everyone, if you are now convinced that you want to get involved in AI, you know now where to go. So thank you so much again, Kathy, and thanks for joining us on this. Thank you. It's now clear that AI isn't just knocking on our doors. It's moved in, it's rearranged the furniture, and it's now demanding to be part of the family. As a vice-chair of the CNCF TOC, I'm really looking forward to seeing more projects spring up in the space. In fact, even K-Serve is proposing to transition from the LF AI and Data Foundation to the CNCF. And we have a very special guest joining us to talk about the future collaboration between the two foundations. So over to Ibrahim Haddad, Executive Director of LF AI and Data. Hi, everyone. My name is Ibrahim Haddad. I'm the general manager of LF AI and Data, which is a number of foundations under the LF focused on supporting open-source AI and data projects and helping these projects grow and accelerate their innovation. And with AI being a horizontal technology, we got through across multiple other technology domain and cloud-native is one of them. So I look forward to collaborating with the CNCF community and working together on different projects and potential integrations between our projects in use. Our community consists of over 100,000 active and very passionate developers that are super involved in our projects and our communities and are eager to demonstrate that the open approach is the right approach for AI. Therefore, we look forward to teaming up with you on any AI-related project and working together to prove that open-source is the right model for AI. Thank you for the opportunity to address you today. I'm actually sitting somewhere in the hall attending this conference. If you see me, please grab me and I'm looking forward to hearing from you on potential collaboration opportunities. Thank you. Thank you so much, Abraham. So let's keep the momentum going, friends. Dive deeper, get involved in the discussions and make magic happen. And with that, it's a wrap of the cloud-native news show. And here's your host signing off. Stay cloud-native and stay fabulous. Thank you. Thank you.