 Welcome back everyone to SuperCloud 4, generative AI session. This is our quarterly program, SuperCloud 4th edition. Gen AI is the topic. We're here in the exclusive fireside chat with Google Cloud's Junyang VP of Cloud AI and industry solutions at Google Cloud. Jun, thanks for coming back on theCUBE. Good to see you, CUBE alumni coming in remotely into theCUBE for this awesome, exclusive fireside chat for SuperCloud 4. Thank you for having me, it's great to be back. Well, I want to just say that we were super impressed. We gave you guys very high marks at Google next this year, just the AI craziness going on in the hype market. Really delivered a lot of strong demos, real product, real next gen functionality we do at AI. Vertex is the program. A lot of people really leaning into this thing, next gen is here. I mean, this is happening, it's legit. Not just hyped up, the game is matching the hype. That's pretty much the conversation. Now a lot of people are trying to get their arms around it, you hear words like guardrails and you're going to be careful, but this is happening. A lot of change super fast is changing the industry. Obviously you're an industry solutions, your VP of Cloud AI. I mean, you got the perfect job. You got the AI and you got the impact side with the industries. Every industry is going to be disrupted and refactored, major inflection point. Google just celebrated its 25th anniversary in the web, okay? 25 years ago, the web changed everything. Now AI is changing everything. Welcome to this fireside chat. Thank you, thank you. I mean, I think as you talk about the web, right? We think about, hey, the web is the first way, we changed mobile and then now kind of AI, right? So it's kind of like web first, mobile first, now AI first. And so from a Google's perspective, when we've been the AI first company for a long time, and if you think about our products and so forth, whether it's on the consumer side or on the industry and on the enterprise side, pretty much all our products have AI built in. So it's great to be able to kind of take a lot of those capabilities and bring it to our enterprise customers as well and allow them to really be able to harness the power of AI. You know, Rob Stretcher, who heads up our CUBE Collective Research Analyst Team, we and I were talking after Google Next, you know, the big buzz was Vertex, obviously very successful, and then Duet AI, which is now embedded in all the products. But there's a lot of other announcements going on that was in the kind of the cloud. I won't say it wasn't important, but it wasn't taking the main press position and the discussion was around AI. But again, this is trickling throughout Google Cloud's products. From storage, we talked with Satya and we saw Mr. Lo Meyer, another, both CUBE alumni out there, their impact, you got TPUs, everything's changing within Google Cloud right now with AI. So it's not just the fancy Vertex open model garden and Duet AI, the next gen coding, it's everything. It is, it is. I mean, we think about AI as really just the integral part of Google Cloud, right? Pretty much every product has AI built in, right? You talked a little bit about TPUs and GPUs, those are the fundamentals, right? The AI models all runs on our TPUs and GPU stack. So very much, you know, kind of very much an integral part of the stack. Then of course, as you rise higher up, you think about customers wants to be able to build AI or build application with AI. This is where Vertex AI come in. And of course, when we think about using AI in our Google Cloud products, whether it's in a workspace, Duet AI for workspace or Duet AI for GZP, right? So those are kind of the areas where we have taken our AI capabilities and really make them usable so that customers can take advantage of them. Not, don't have to know anything about AI, they're just make their life easier. So whether it's developers who's trying to, you know, code to getting some code help, or it's a, you know, it's just regular business user trying to use, you know, Google Docs and be able to say, hey, help me write something, right? So those are kind of easy ways as applications that's kind of taking the ability of AI and making that a reality. And of course, Duet AI is built on top of Vertex AI. So this is kind of our way of a dogfooding and actually really using our own products for various purposes too. I want to get into some of the demos we saw at Google Next and all some things are happening now since then, which got the most uptake where the traction is. But before we do that, I want to talk about what your role is. What is your job there? Because you're the vice president of the AI and industry companies, industry verticals. What's your, what's your job? Yeah, my job is great, right? So my focus is really taking our AI capability as products and services and making that available to enterprises. And we've taken all of those, the portfolio and the really kind of branded under the Vertex brand name, right? So with in Vertex, we have the development platform which allows customers and developers to be filled their AI powered application use our capabilities and using our models, using our search capability using our extensions and connectors and so forth. And at the same time, we've also built a set of application that customer can consume really out of the box very much like a SaaS service. So if you think about the context center as AI solution that we offer, this is really a targeted at call center agents allowing them to do their job easier allowing customers to be able to interact with voice bots and chat bots to be able to get help they need. And if you think about, we also have discovery for retail that's a very much industry solution that's taking our AI capability and search capability combining them together to help retail customers to be able to search and find things more easily in their own catalogs. And that's been a very popular industry use case as well. So really kind of we offer the whole gamut from the platform to the solutions. You know, AI is like the fountain of youth, I call it because I feel like I want to be 25 again because it's so exciting. And Dave Vellante and I were both talking on our Q pod a couple of weeks ago that this inflection point reminds us of the PC era that changed how things, how people used applications and their expectations. The web obviously changed what was pre-web and everything went online. That changed user experience and then expectations. Mobile you mentioned came out and said, okay, that's an app store, that's apps at SAS. I said that cloud was an inflection point but I walked that back because I don't think we've had a bigger one. I think AI is bigger than cloud because SAS was built on top of mobile and cloud. But I think AI is that this inflection point and I want to get your thoughts on this because in every one of those inflection points every single industry was changed because the expectation and the user experience and the format of the service or product or content was refactored for that era. So we are in the dawn of this new era of AI. I'm sure you agree with it. If you do, that means every industry has to refactor. Yeah, I think every industry has to think about how they're going to do things. If you think about AI, right? I think about this way of a generated AI is really a very large, massive scale of democratization. You think about the earlier stage of AI that's really kind of more in the hands of the leaders, right? The data scientists, the ML engineers, people who have the depth and knowledge of using AI. And us, a generative AI and these kind of foundation models come to life and where these models are good enough to actually be out of the box and doing a variety of tasks out of the box. Now you actually can involve many more users like the developers who's at least 10x the size of the data scientist population. And of course, we talked a little bit of some of the solutions and these solutions can be used directly by the business users who doesn't have to know anything about AI and really just kind of enjoy the benefit of AI. That's another again, 10, 100x bigger than the developer population. So you think about just the impact, right? Really kind of going from a smaller set of persona into a much, much bigger set of persona. And now we think about really making AI useful and helpful for everyone. And I mean, like everyone. You know, people ask me why I'm so excited about AI. And one of my answers is I've never done so much GitHub code review, browsing. I've never done so much paper reading and talk about academic papers. I've read more academic papers in the past eight months than the past eight years. I just read a paper last night that was put out by the FDA, the US FDA using artificial intelligence machine learning and the development of drug and biological products. It's positioned as a discussion paper and request for feedback. And I bring that up because when I was talking to folks it was at this conference this past week in Montreal about this, their first reaction. Again, these are smart people. They're kind of techy, but they're not inside the ropes. Their first reaction was this hallucinations. We got to be careful. Like they went to the doom and gloom. They didn't say, they were totally negative. But the FDA is not saying it's bad. They're just saying, be careful of the data. So we're at this discovery phase in every vertical. Again, people are putting out papers. Is there a right approach? What should people be thinking about that are watching this as every industries are doing these discussion papers? The request for feedback. Is there a best practice? Is there a playbook that you can share that Google sees, Google Cloud AI sees as a way for people to move the needle without creating a lot of gloom and doom fear? Yeah, for sure. We spend a long time kind of talking with customers about use cases. What's the right set of use cases to get started with? I think there's a few sort of characteristics that people want to think about. One point I want to make is that this is one of those areas that you want to start now and then be able to iterate and improve. You might not get it right the first time, but you need to get into the game and start learning about it, see what it can do. And this technology is also happening and involving and innovating at such a rapid pace. I'm sure, like John, you're following this, like every week, every month, you're seeing sort of progress, right? And this is not something you see in other technology space where you make weekly progress on. And so it is pretty amazing to see just how quickly has matured and how quickly the ecosystem has built up and all that. So coming back to sort of customers, especially enterprise customers, we started thinking about what are the right set of use cases to work on, right? So what are some characteristics of the use cases, right? I think use cases is that something that, one is that you think about, these models are nascent and the generative AI is powerful, but it's also nascent technology. So you want to find use cases where it can create value and but at the same time, it can tolerate some level of faults in there. And because none of the models going to be fault proof, right? So if you think about use cases, like for example, I want to use my model to be able to help me to do marketing collaterals, right? And giving some prompts to creating marketing materials and so forth. That's a fairly fault tolerant use case. And you can take and use that as a basis and then be able to kind of adjust and modify, but it helps you to be able to explore a variety of design directions, whether it's in terms of the images or in terms of content or tweets and so forth, very quickly and be able to kind of accelerate. So that's an example of a good set of use cases to do. Another set of use cases you think about is that things that's actually grounded on enterprise data to reduce the amount of hallucination. So this is where we think about, we have a lot of customers and the enterprise customer all have a large repository of enterprise content. And they have lots of documents, they have lots of content that's sitting everywhere on the website. And often they want to have a Q and A with their documents, right? They say, hey, how can I summarize my analyst reports, right? Find out what my earning is for the last quarter and compare it with the three quarters before that, et cetera. Those are very common questions that we get. And those data actually are sitting in your repository. This is where we actually bring the search, power of search capability and a large language model together. We call this kind of a grounding with enterprise data so that the results that's generated is actually grounded on your enterprise data. Again, that's another way to be able to reduce the hallucinations and so forth and really making sure the data you get back is relevant and it's coming from a trusted source. What's interesting about Google is the word context and retrieval are constantly used in AI, retrieval is the rag they call it, retrieval access. That's key part of the new vector database. All this new goodness is coming in at scale. How do customers get started? Because that's the number one question I get is like, look, we're definitely coming all in. Top management saying we need to be in AI. Bottoms up as more the developer and traction the practitioners. As you said, democratization is here. Everyone's going to refactor their companies. We see that clearly. What are the steps? Is it get vertex? Is it get do it? What should companies do? What's the playbook? Yeah, I think it definitely gets started with vertex. We actually have put out a number of free training courses there as well where customers can learn about our products, our tooling, gets hands-on training as well. And once they have it, come to a vertex and come to Google Cloud, it's pretty easy to get accounts set up and start using it. And I will start with all this generative AI. You start with the models. Think about the use case you want to use. Are you trying to do a coding use case? Are you trying to do a language use case? Are you trying to do a chat use case? Pick the model that's appropriate for you and start playing with the prompts. What's the right set of prompts? How can we do some prompt tuning and then potentially even put in some people to kind of tune the model further if you're a little bit more sophisticated. But if not, generally start with a model. That's a good place to start. And then we think about, okay, how do I to make the model, augmenting the model with more capabilities or more data? This is where the things around extending the model or extension capability allow you to extend the model connected to your enterprise application, for example. So give you a simple example. If I'm trying to build an HR bot, I want to ask about, hey, what's my vacation balance? And that information is obviously not sitting in the model. That information is sitting in your backend HR database. So we need to call the extension to the HR database and pull back the information and to be able to answer that question correctly. So those are kind of a set of capability around extending and augmenting the capability of the models. And then the third thing I would say is that we have taken a lot of these very common use cases, things around chat bot, things around building a search bot and so forth and really made that a super easy process as part of the vertex search and conversational capability. We're really in a matter of minutes, you can spin up a chat bot and then people always get surprised because we do a live demo with our customers and some of these customer visits and show them how in a few minutes we can actually build a bot and be able to answer questions coming from their websites or build a search bot to be able to search on their content. And it's usually better than whatever they have created previously. And so those are kind of, I think the bar of kind of getting started is so much lower now. And I think, you know, even a person who's not necessarily super technical can actually get started on this process. And we have really created, you know, I would say low code to low code kind of a process to help people get started. And then of course, things can get much fancier, much more complicated and over time. Yeah, and I think that's the beautiful thing about the cloud. You can do more as you get more experience. I want to ask you a question on the adoption side because one of the things we saw with the early days of cloud, you had shadow IT was the buzzword, you know, democratization. There was just get developers lower cost resource to get things going, iterate as you pointed out earlier. Now we're in an era where, and by then cloud one dot oh was about hackathons. You can see hackathons out in the field. And that was using dummy data and or simulation data. Now companies can do hackathons inside their companies. I'm seeing enterprises kind of shadow AI, if you will, prove it to me, get the demos going. So when I say demos, it's like just enough to be dangerous and real, but yet not scaling. So you have a couple of things going on here. I want to get your reaction. How does Google cloud help facilitate and accelerate the, you know, playing around with data because now you can do things with real data, right? You can put a vector database in there. You can start playing with things, write some apps, they'll do integration, vertex, model garden. I mean, all these things are in play. Okay, now I got it. Now I want to scale it into production. That's the progression. And by the way, these hackathons are going on involved inside the companies and most of the demos are competing. We actually help a number of our customers to run hackathons, right? Because a lot of time they want to explore different ideas and really getting their developer and business kind of folks together and start kind of explore some of these ideas. And with R2, they can actually, you know, in 24 hours, 48 hours, build a very respectable prototype and to show what is the power of, you know, XYZ use cases. And then it sort of brings much more emphasis and in terms of what is the possibility and getting the executive more excited and then taking a few of those use cases and think about going through what you're talking about, you know, productize it and go into putting that into production. This is where a lot of the capability of vertex kind of really shines, right? Because we build our tools to be kind of running production. These are things that we, you know, like took years of experience from Google itself where we have lots of AI-powered applications and looking at the toolings and so forth that's necessary for those and pulling that into vertex and making this available to our enterprise customers. And so, you know, things like, hey, look, how do I monitor my, you know, my model as it's going through and how do I make sure there's no drift and so forth? Those are kind of model monitoring capability that's already built into the system. And then as customers are, you know, getting feedback on their, you know, on their application, often they want to be able to take the learnings from the feedback and actually feed it back into the model to improve the model. So this is where we have reinforcement learning with human feedback as built in of the part of the process as well. So this is a way to kind of help the model to improve over time. And of course, all of these, you know, really runs on top of our, you know, very robust and scalable infrastructure and we call it planet scale infrastructure, right? And it really allows you to think about going very large, right? We have customers really in the matter of a week going from zero to, you know, like to really a hundred and then we're able to kind of support those with no problem because our, you know, infrastructure and our models and so forth has been, you know, truly tested, you know, battle tested, you know, over a course of, you know, many months. So therefore we're quite confident with those capabilities. You guys got the computer, you got the TPU, certainly super cloud enabled. You guys also have a lot of partner ecosystem has been booming as well. And there's a lot of data moving across environments as you get distributed computing at the edge. Okay, model support, moving compute to where the data is, moving workloads to where the data is. We're going to see data at the center of all this and the models and the model garden is going to be key for customers to choose. Can you just quickly talk about how customers should think about and what advice you'd give around which models to choose in the model garden you have under Virtec, which is one of the key features. Yeah, yeah, a model garden is something we introduced in the beginning of this year, right? So it's only been a little over six, seven months and it's already the second most, the most popular, you know, pages to visit on Virtec's just kind of showing sort of the interest in this type of, you know, models and started as a starting place, right? And within model garden, we offer, you know, both the Google first party models and across the different modalities, whether it's language or speech or images and so forth. We also offer open source model like stable diffusion, open llama and code llama, et cetera, as well as the kind of third party partner models as well. Our goal is really kind of giving customer choices, right? They should pick and the model that's best fitting for their particular use cases and have the flexibility to choose. And you talked to them a little bit about, hey, you know, how should a customer choose? I think a customer think about it, think about, okay, again, goes back to the use cases and the business need, right? And then also understanding that the model prices is very much skills with the size of the model as well. So the bigger the model, the more expensive it is, generally have higher quality. The smaller model are less expensive, but generally they're not quite as powerful, but they tend to have a lower latency as well. So you need to sort of think about the price performance and of course price is always another consideration. How much is this use case worse for you, right? For some of the use cases where we think about financial services, where they have their analysts actually looking up highly value information, since there's information. For that case, they are actually willing to pay for a lot of money to make sure that the data is accurate and it's comprehensive. For some of the other use cases, not so much, right? I'm not willing to pay that much for a chat bot on my website for certain use cases. And so you want to sort of think about, what is the price performance you're trying to hit? And then sort of picking the right places, right price points. The thing is we offer sort of variety of kind of options for customer and allowing them to choose. And then going with that, other thing a customer always ask about to say, with all the different models, how do I actually evaluate these models? Which one's better for me, right? So this is where our model evaluation framework come in. And so this is something we obviously do day in, day out basis. And we're putting this out there for customer to allow them to be able to use our model evaluation tooling to be able to compare two different models or two different sizes and decide what's best for their use cases. So they have a taking a much more systematic approach to this model selection versus just like, hey, I feel like this is better or I feel that is better. There's gonna be a lot of models that are gonna be working together with each other. It's got a power law. We got a big research piece out on the power law of AI models. Some are small but powerful. Some are big and robust. Context, retrieval, search. All this is going on in AI, big part of the refactoring, a lot to unpack. And Jim will certainly do more content. This is gonna be a topic we're gonna continue to hit. Really appreciate you taking the time to come on our SuperCloud 4 event, focusing on general AI. And again, congratulations. Quick minute we have left. What's your goals up until Google Nexus is coming right around the corner in Vegas. And you got about a six to eight months away from what your goals for your group. Yeah, I mean, I think we've been very busy over the course of the past year and continue to deliver new capabilities and so forth. So we've taken the entire suite, it's now GA. And I think there's multiple sort of things that's gonna be happening. Obviously, from a model perspective, we'll continue to upgrading our models and making them more capable and introducing new variety of models and a new modality of models. Number two, I would say, it's really about what I mentioned earlier, augmentation of these models, right? With extensions, with connectors, with grounding and so forth. So you'll see us bring more partnerships and coming into this to allow you to be able to connect to other data sources with the enterprise more easily and using the different tools more easily. And the third I would say is really around what we talked about, taking the power of search capability and conversational AI and LNM together and really kind of making that a powerful trifecta over there and more and more integrations are gonna come and making that process easier. So you can be able to leverage the power of search and power LN much more seamlessly. So those are kind of things that we're working towards and it's really a continuation and journey we've been down already. And you're not a stranger for data, machine learning, AI and next generation cloud, super cloud is here. Thank you for your commentary and appreciate you taking the time to join us. Yeah, thank you, it's great to be here. Okay, super cloud for JNAI will be back at the short break. I'm John Furrier, thanks for watching.