 Welcome to this special Cube presentation here in Las Vegas at Google Next. We're on the 62nd floor in the Penthouse Suite with Persistence. This is an accelerating innovation program with Persistence and Google. We've got two great guests, Lily McNeill, as director, I'll found product management, Google Cloud AI. Thanks for coming in. Thanks for having me. OK, and we've got Rajesh, Yankar, Vice Senior Vice President, Google Business Unit at Persistence. Thanks for coming on too. Thank you. So we were talking before we came on camera around the evolution of where AI is. Some hype people are excited about AI and the confidence it's starting to see. We had Google's keynotes today. We saw the news announcements. The needle's being moved. There's still more enthusiasm. Confidence we're getting there. But we're in this mode now where people are talking about, OK, we're experimenting. Budgets are being allocated into AI now. We saw last year, budgets were being taken from other departments. Now we're seeing real movement on investments. But the real conversation is, what's in production? That's going to be the proof. The proof will be in that stat. So the question here is, where are we with production workloads? And are we in a good spot? Absolutely. It's a great question. I think where we are right now is there's a lot of potential to move these pilots and POCs to production. But the organizations who are going to be successful with that are those who have spent the time to identify the right use cases from the get-go. Organizations who've just deployed agents or models without really thinking through, where do they have the opportunity to generate value for the business? They're never going to see that ROI. So first of all, I think it's organizations who think about the right use cases, then move into the technology. And what they really need to be doing in order to get across that chasm and not fall into what we like to call the trough of disillusionment is think about it from a platform approach. If you don't have the right tooling in place, you're not able to customize those models. You can't deploy them at scale. And you don't have the right services around MLops to actually monitor those models in deployment. You're never going to realize the benefit. So I think there's a lot of potential, but there's a lot of work that needs to happen in order for organizations to realize the real benefit of generative AI. So there's a platform conversation. Quick follow-up on that. How does that change post-GNAI? Because pre-GNAI platforms are well understood. Computer science theory is always out there. We kind of know that world. We've been there, done that. Then the conversation shifts to, okay, the platform has to create, enable generative generating results. We see prompt result reasoning is being discussed. So what about the platform are people focusing on? What's the big, is it the data? Is it just more of a system architecture? Is it less data-centric with the scientist and BI department? What does this new platform conversation look like? Or are people still experimenting? Yeah, I think we're starting to see what that looks like. The good thing is that the workflow remains relatively the same. And so if you think about it from discovery, to customization, to deployment and monitoring, those are the same steps that we were thinking about when it came to predictive AI. What's different is that you need different services to do this with generative AI models because they act differently than predictive models. And so you need to think about services across the tool chain such as prompt management. There are new artifacts that are being created that didn't exist with predictive AI. And so we just today announced a ton of new functionality around a prompt management, tagging, and ultimately assistive technology in that space as well. Then when it comes to ML apps, again, some things are the same. So when you think about a model registry, you wanna be able to see all your models, whether they're predictive or generative through a single pane of glass. But there are also new types of tools that we need such as evaluation. So that's where things like auto side by side come into play. So the way we like to think about it is, you still need one platform and you wanna realize the efficiencies of having predictive and generative in one place. But there are some steps along the way that look different and we're building tools to help customers with those. By the way, general availability of automatic side by side, it was announced, as well as rapid evaluation, which I thought was clever because now we're gonna bring in smaller models. Rajesh, this is the new environment. ML ops, I mean, AI ops has been talked about before, Jerry, we'll get there in a second, but you're starting to deal with operations. Model garden, last eight months ago, we talked about model gardens, so we love that. Kind of gives an impression I'll get all your models organized. Now it's like, okay, I got model building and now you got agent builder. So you have now the progression happening. So we're in a good spot, aren't we? Yeah, I mean, what Lily just said, that's where the partnership comes in. So the Google has model garden and all the right tooling and the vertex AI. So the platform and everything that Google can do as a platform provider is there. But the key question that enterprises are asking us is, how do I make that work in my ecosystem? That is for any non-trivial organization, it's quite messy. Data is in silos. There's strict governance and security requirements and there's so much going on in enterprise that how do I make all of this work in my enterprise? And that's where we come in. Start with ML Ops. As I enable all of my line of business functions to start using this platform, who's keeping track of costs. So there's a phenops angle to it. There's model observability. How is the model changing with all the prompts and the answers? Are they still giving the same consistent answers? There's a different kind of testing required here. So who's handling that? So if you keep adding all of those things, that's where I think we come in. That we have deep expertise in Google Cloud and all of the new tooling that's coming in. Work closely with customer engineering, the product team and so on. So we bring that expertise in. But then we also understand how the challenges that enterprises have. And that's where our layer can come on top of that. When you see these inflection points, not to digress a little bit, you think about how people think about it. And we're really, this is a bridge to a new future. But guess what? You got to build the bridge. And you guys are helping. You guys are unique value providers, just so kudos to the persistent. This brings up the question back to Google because the announcements, I'll just read a few of the highlights. 130 models in Vertex AI, a million token contacts went into the largest in the industry. 700,000 words, one hour video, 30 hours of audio. Cross modality analysis, huge point. Because that brings in now the reasoning aspect of AI, which creates, will create more complexity, hence opportunity for the platform. So, okay, it's a daunting view of you're an enterprise and you haven't skilled up, you may have one person, maybe your IT people haven't gotten up to that digital transformation, accelerated gap you have to be there. It could be challenging. So the question I have is, are we really talking about a new operating system? Because it's a platform of platforms. You got AI systems that look more like, give me more GPUs, give me a custom processor. Workflows might have dedicated silicon in the future. So we look at a world where the entire environment's changing. Now, if you're just an enterprise, you're like, okay, I got to run my workloads. So this production conversation is a forcing function to saying, where are you and how do you think about it? Because it's not just one workload. The data's got to be rethought through. What's the data equation look like? So these are like the big high level questions. How do you guys talk to customers when they'd say, I don't know what to do. I'm over my head, or maybe, I mean, it's an opportunity. When these new opportunities change, companies will either succeed and take territory and grow and make money, or they might fall to the wayside. So this is the reality of the business psychology right now. I don't want to be on the wrong side of history. So how do you guys talk to customers? What's the baseline? Stay internal, use your existing data, get a couple success momentum hits there, get a success here, success there. Don't try to swing for defenses. What's the approach? How can you, how would you describe that? Okay, I can take that. The, I'm just reflecting on the recent conversations like the last couple of weeks. The question really being asked is, there are a bunch of use cases. Everyone did hundreds of use cases over the last year. Each company of our size, we've done a hundred plus. And within a company, there are task forces now that are creating use cases. There are committees that are ranking them and picking a few use cases to take them to production. But the key question really is that, are we comfortable to open some of this technology to our end customers? Or let's keep it internal yet. So a lot of the use cases we're seeing are all internal. Employee centric, customer service agents centric, but they are still internal productivity, enhancing use cases. But the broader question in terms of the value chain, if you were to zoom out, is the models are quite compute hungry in the training stage, fine tuning stage, as well as the inferencing stage. And so no wonder the GPUs are now the bottleneck. And if you go another step further in the value chain, the power is going to be the next bottleneck. That's why you keep hearing about the need for power generation. So we are really looking at a whole different paradigm of... The constraints are different. Absolutely different constraints. Takes you all the way down to how do we source the power, not just the compute. So it comes decked out to, okay, Google's enabling all this. Companies can take advantage of it, whether it's retooling, say VMware. They want to maybe bring the lift and shit into Google Cloud, but then talk about some folks that they were seeing momentum there. Encapsulated existing workload that might have been written in a language run by an employee that's no longer there. We've heard these cases like AI can come in and learn about environments. This is an opportunity. And you're in the product management side thinking about this with customers. Is there a pattern to this? Is there a success point we're seeing now people can do? What are some of the things you might have seen that people could learn from? Absolutely. I mean, I think just to build on the prior points, it's going to be change management, right? There's going to be a work that enterprises need to invest to actually figure out how do they work with this new assistive technology? And that's going to take some effort. And as I mentioned before, they really need to start with the right use case where there's actually enough headroom for them to realize value with the cost in mind back to the FinOps conversation. A few patterns where we've seen folks be successful is obviously those internal chatbots. I think that's a really great place to start. It helps organizations realize some efficiencies. Though, I think we're moving in directions of code generation, code completion, code assistance, that being a huge opportunity and enterprises really benefiting. What I think is going to be really interesting is those organizations who are able to take that next step and actually launch consumer facing applications powered by generative AI. And we're starting to see that today and I think we heard some examples in the keynote. And I think that's where we're really going to realize a lot of the magic of this technology is when we move from just B2B and interior facing to B2B2C. Yeah, that's great. And that's a great point. I would just want to say that it's interesting because the feedback we hear when we talk to folks in the media is they want to get momentum because there's a lot of, I won't say doubters out there, but there's also like people who are like trying to get visibility on the economic side of it, whether it's the revenue side, how do we monetize this? Or do we build our own AI or do we use AI and manage services are out there and you guys have a lot of them out there? So there's a question of how do I balance the two? And one, can I get someone to actually build AI? That's all another discussion. So just to touch upon the change management that she mentioned, I think that's super critical. It has to be addressed not just in terms of the tech team are they ready to handle the new workflow with a new software lifecycle with AI playing a critical role but also the users of it. Because there's some fear there. It's being seen as an automation tool, but there clearly are maybe the bottom half of the pyramid of cognitive work. There's a risk of getting automated. And so there's the change management is super critical as we also address all the technical aspects of it. What's interesting, we did a little bit of research at theCUBE research on our team. We found that most digital transformation projects would fail. This is prior to the gen AI when it started to hit was because the executives were all at the table getting the cash and getting the funding and the IT teams were busy toiling away, running the operations. And they weren't really moving along that journey of how to manage some of the leadership challenges around how do you operate the new transformation? So a gap between the people have to execute and then the leadership at the top. So change management, one piece is the human piece. This is an opportunity, augmentation, reinforced learning, agents who are not just chatbots but actually doing some work. So the question is, we see that gap being filled like platform engineering, for instance, or ML ops, filling in the gap, but still requiring humans obviously to do that. So that's an opportunity. So the question is, with agents coming online and generate AI being so strong, what areas do you guys see will be eliminated that was built for a pre-gen AI world? For instance, you could argue that microservices, there's a lot of things that go on there because it wasn't generative. We mentioned tickets as an example. So is there areas that you can say, well, we don't need that anymore because the generative agent will take care of it. So we'll see some things that have been deployed that we don't need anymore, only because the better mousetrap is here. Can you guys share any insights into what you see that might fall by the wayside because we don't need it anymore because it's been replaced? I think what we'll see is that there are always some mundane tasks that you and I do every day, whether it's writing emails or dealing with calendars, that I think will benefit greatly from AI and automation and gendered AI specifically. And so I don't know that there's a category of jobs necessarily that I would say would be more greatly impacted, but I think there's some horizontal aspects across a lot of different roles where we'll see, hopefully, people have free time back and they can use that time for more creative tasks and tasks where human judgments needed. The speed with which these agents are evolving and their capability to do planning, as that evolves, you can let your imagination run wild, you can automate almost all the tasks because if you're, the first step is for the agent to plan and say that I'm going to approach this with these 10 steps and then have another draft round with some human input and it starts getting better at planning, you can then imagine so many different cognitive use cases that can be automated. I mean, just the opportunity for open up creativity too as well is going to see a lot more, you know, kind of leveling of jobs and kind of democratizing and giving people opportunities with their free time to do what they want. The question, so that brings up my favorite question, which is for the people who are like learning about AI, like what does it really work? Is hype, is there really there there? How would you give an example of an example that'd be so obvious? Like if there's an example of January AI and it's so obvious that would convince someone who may be on the fence or doesn't understand it to believe it? I'll give you a couple of fun ones. So the first one, I have a friend who'll remain nameless who used generative AI to actually create, they were the best man in their brother's wedding, the entire best man speech. And I think that saved a lot of time and also I heard it landed very well. So I think that's a fun one. And another one I like, well, what I like to do is figure out how it can help me in my day-to-day life. And so a recent example is I'm in the process of doing a home renovation. And so anyone who's looking for lighting or furniture knows that you can just scroll for hours and hours on the internet these days and never find exactly what you're looking for. With generative AI and specifically with Gemini 1.5 Pro, you can actually ask the model. You can give it a picture of the room that you're looking for. You can give it a description of your taste and the types of things you like. And it can actually help you find the right item for your exact living room or space, which I think is really interesting because it's the creativity that we talked about. It's a little bit of the advanced reasoning. It's the judgment. And then it's also applying some sort of taste, which I never thought was possible. And so those are a couple of fun ones that I like to share with folks. Real world. I mean, think about the decorating side of how much time that's gonna take. Oh yeah. Page loading. And then you change your mind. You got to go to another site, log in. Oh, I've spent too many hours writing. So this is a huge time saver for me. And be great if you can get the bids coming in too, automate that process. Oh yeah, that would be very helpful. Rajesh, we actually, someone else used that example, so that's why I remember that one. So Rajesh, what's your take on this? Because I think, what's an obvious no-brainer? That's obvious. I get it now, example. Yeah, it happened to me. And this example, I'm telling you, most parents can become mom, super-dads helping with homework. And this is not just, it's like simple homework. Deep high school physics, AP physics, level problems. It's been, I don't know, 30 years now since I've done physics. And I was helping my sophomore high school son with something. And I literally took Gemini, took a picture. I have the Gemini Advanced, the paid version. Took a picture of that textbook problem. And it did a step-by-step understanding, analysis, explain what it is, solve it, and then offer it to do the calculation because Jenny, I can't calculate. It can write a script, it wrote the Python script, and then give the answer. I had that in front of me. And then that helped me recollect my own physics, able to help, imagine that. He has to, otherwise I had to go and write to the teacher, go extra help. But now you can, but my mind was going somewhere else. If it can do this for physics, you know, what can it do in an enterprise? And there are so many situations where there are handwritten, there's going to be audio, there's transcripts, and there's text, and there's a problem area, simulating all of that, putting a plan together, and then helping you execute that. I can really see the tremendous potential. If you can solve AP physics, it can do so many things. Yeah, that brings up a great point. The reality is not only obvious, but also think about, if you bring that to a business, as people start solving these problems, where do you store them? It's a workflow, it's a use case. So there's been a lot of discusses around knowledge graphs, neural networks, and so you can almost imagine that the data modeling will change now too. So I know you guys are a lot involved in data, and Google with BigQuery is all over, all over the news this week, all over, pretty much all the announcements with Gemini in there. So the data structures are changing. What's the story there? Explain how that's changing and evolving, and setting the table for these kinds of new ways to run AI? It's the way, without geeking out on the question, I would love to spend more time. There's a concept of latent space, or in science, think of it as a space in which data coming in from different modes, whether it's spoken words, digital, digitized audio, video, images, text, if they all mean the same thing. If you're talking about decorating a home, it could be an image of the living room, it could be an article text, it could be a snippet of a transcript from a podcast. If all of them are talking about the certain aspects of interior decoration, they all reside in the large model in the same space. Doesn't matter where the data came from. And once they are there together, that's what the multimodality of these models come in. It could be DNA, we were talking about before, we came on camera. It could be DNA, yeah. It doesn't have to be language anymore. That's why these are just in large models, or foundation models. With DNA, it could be any large amount of data. It turns out, yeah, our human body is an IT problem now with the DNA, with the genome being sequenced. Drug discovery, molecules, what Google's deep point has done with Alpha Fold, and how that's unlocked. It's tremendous in a research bottleneck that was there earlier with understanding how molecules fold, the structure of it. And it would take months and years to do it, and it's just accelerating science. So the way I think the data structures are changing in a fundamental way is just understanding that in limit terms, it's a space in which no matter how the data was digitized, if they all mean the same thing, they're going to be grouped together in one area, and then you can query it from any angle, speech to speech. Translation doesn't have to be speech to text, and then text back to speech. If I'm speaking in English, in my tone, my voice, now if you've seen demos of some of the research that Google's doing, I could be speaking Japanese with my tone and the same emotions. It's just changing the way they store it. I mean, it's more changed, probably, that we didn't even know. You guys probably see things in the lab that are coming. Rajesh, really great to have you on. Final word to end the segment is, as production workloads start to hit, we're starting to see more and more this year coming online with AI use cases, how should companies be prepared? What should they be thinking about? How should they be rolling out these production workloads? What should they be mindful of knowing that there's still a lot more going on and coming into the market? Share your thoughts as we start seeing more production workloads. What are we going to see? How should people think about it? And what should they pay attention to? I think we learned a lot with predictive AI, and I think it's going to be really important that we keep that same mindset around MLOps as we move into productionizing these generative AI models. And as part of that, I think it's going to be really important that companies who are moving in this direction deeply understand their posture on responsible AI and set up the right governance processes to ensure that the generated outputs of these models are aligned with their AI principles. There's a lot in this space that's unknown and organizations need to go into this with their eyes wide open. And I think taking the time to set that up upfront will save a lot of potential heartache down the road. And if they can get someone to do it, find someone to volunteer, it's a big job. Yes. Definitely. Well, you guys had a lot of great announcements. Congratulations today. Rajesh, final word on what to prepare for. Think about production workloads. Yeah, I'll approach it from the technology angle. Data is the key. And getting the data house in order, modernizing all the data platforms that a company has, that's something that needs to start now because the AI sits on top of it. There's only so much that you can get out of the box. Everything else has to come from your business data. And now you have this magical technology that can consume data, whether it's lake, lake house, warehouse, whatever the technologies that you used, modernizing all of that and preparing to integrate that with some of the new AI platforms. Thanks so much for your time. Deploying and optimizing AI workloads, ML ops, a lot to pay attention to, but the value is there. And so much more headroom beyond that. This is theCUBE, I'm John Furrier. Thanks for watching.