 Welcome back to Las Vegas everybody. We're here at the Caesar's Forum. The venue here is exploding, so about 13,000 people here. So the keynotes are at Caesar's Palace. All the exhibits are here at Caesar's Forum. You're watching theCUBE, I'm Dave Vellante with George Gilbert. We go out to the events, we extract the signal from the noise. Christine Kleinerman is back. And the big news last night was the partnership between NVIDIA and Snowflake. Manuvir Das is here. He's the vice president of Enterprise Computing at NVIDIA, two hot companies. Wow, get together, magic happens. There you go. AI, LLMs, generative AI, it is like magic. You were just saying that. So you guys have been on fire, it's amazing. The AI is hot, don't touch it. Why is this partnership so relevant? I think Dave, because it's a very natural partnership because everybody's got the sense of what generative AI can do because of the chat GPT moment, the iPhone moment of AI. But for enterprise companies, really, if you think about it, every company is sitting on a corpus of data in which is hidden all of the intelligence of that company. And the question is, how can that enterprise extract that intelligence out of the corpus of data that belongs to them, right? So it's taking LLMs to the next level. I think a really good way of thinking about this, and Jensen has alluded to it, when you have a foundation model, a general purpose LLM, it's like somebody who just graduated from college, you know, a super smart new hire, but now you want to take that person and what if they had been at your company for 20 years and they had all the experience and knowledge of working in your company for 20 years, wouldn't they be a much more productive employee? And that's the difference between just a regular foundation model, but when you take the same thing and you make a custom model for your own company with your company's data, your company's intelligence, and where is that data? It's on Snowflake, right? And so Snowflake has the data, Nvidia has built the intelligence, the engine to extract that intelligence and create these custom models, and so we are bringing that to the data, to Snowflake's club. So I generally have put the reactions to this whole trend over the last six, seven months into three camps. There's obviously a lot of hype, okay, we get that. There's skepticism, you know, some of the old timers, ah, no, no, nothing new, I've seen this before, it's over hype, and there's fear. What's the right way to think about this? I would say all three are valid. The excitement and the noise, it's legitimate. We've all seen demos that are completely mind blowing when we were talking earlier, some experiences are magical, so that's true. The skepticism, I get it, there have been many other technologies emerging in the last few years that they're like, oh, everything's going to change. I'll name names, Web 3.0, not quite. Now it didn't turn out to be what it was. And the fear is real because this technology is going to disrupt, I would say, every industry, every use case. Not completely replaced, but in some way it's going to affect the types of results and the experiences you have. It's only that we need to let the dust settle between all of this. I think out of the excitement and the hype, a lot of it is going to cool down, it's going to convince the skeptics, and it's going to hopefully provide clarity for the fearsome. So explain what you guys have actually done. Have you essentially containerized the NVIDIA stack inside of Snowflake? Yeah, so the NVIDIA software, right? The best way to think of it is this, Dave, that NVIDIA's platform hardware and software is designed from day one to run anywhere, okay? All the software has been containerized from the beginning, right? And as you know, Kubernetes platforms of various ilks are available everywhere, right? So the integration path is very natural because what Snowflake has done, has been working on, is creating a containerized environment where if you've containerized your application, you can run it within Snowflake, and so that's why we were able to move so fast. Now with that said, that's sort of like the day one thing, but there's a lot more integration in really making it a great experience for the customer, so it's seamless, right? And it has to be enterprise-grade, right? So you asked about, just to mix two questions, you asked about skepticism versus stakes, right? Yeah, please, I'd love to be in that one. I would describe it this way. In the early days of the cloud, right? So many enterprise companies were skeptical about the cloud. Well, what has happened, right? Because the journey has been, the cloud has essentially become enterprise-grade. Things like compliance, security, et cetera, have been brought into the cloud, right? It's become an awesome destination. I think LLMs are on the same path, right? They have to become enterprise-grade, and so a lot of the work between the engineering teams of NVIDIA and Snowflake is about taking that containerized model where all our stack runs, but really making it an enterprise-grade experience for enterprises. Is it secure? Where is the data for the training coming from, right? The model that is resulting, it's a copy for that customer. The data that from that customer improves the model is not going back into the base model, right? These are the kinds of things that really matter. When the model is generated, where is it kept? Is it efficiently serving the applications that are done running within Snowflake? There's a whole pipeline here that has to be well-optimized and well-engineered, right? So that's where a lot of the work is, really. So maybe let's drill into that first, but maybe some context settings so that customers understand the level of customization, starting with prompting and the guardrails you can put around that, then fine-tuning and when they would do that and what it looks like, and then initializing a model with random weights starting from scratch. Yeah, so you said it really well, but if you don't mind, I'll say it in the reverse order. Okay. So you can always start from the ground up, right? And you can take, train the model, right? With all your tokens and like you said, the weights and biases will adjust and you'll get your fully trained model. That's an expensive exercise, right? And if you think about what you're doing in that exercise is you're trying to endow your model with a lot of general knowledge about how the world works and generic skills and there's no need to repeat that every time, right? So the first thing that NVIDIA NEMO brings to the table which is part of this integration is we have pre-trained a suite of foundation models where a lot of the training, the months and millions of dollars of training to get a general knowledge and generic skills has already been done, okay? Then once you've got these base foundation models, then you can do fine-tuning and in fine-tuning what you're doing is you're actually doing more training on the model, right? So the model, the weights are being adjusted as you go, but the data you're bringing in now is specific data, okay? And in this process you can think of it as one level is I'm training for a domain, okay? So let's say HR is a domain. So there's a number of snowflake customers who are in the business of HR and now you can start with not just the completely generic foundation model, but the less generic model that understands HR. But still doesn't understand one company versus another, right? That's the first level of fine-tuning. Then the next level of still fine-tuning is here's the databases of my company's data. I want to start with the HR model, but I want to further fine-tune with my data, okay? But you're still adjusting the model. Then from there, as you said, you can go to prompt tuning where while you're using the model, you add prompts that are specific information about your company. You can do information retrieval, which is it's a combination. I will use the large language model, but then I will query my database, get responses from my database and feed that back in into the prompt, into my LLM, right? And then finally, you have the reinforcement learning, which is as you're using the model in your applications, the humans who are on the other side can evaluate and say this response is a good response or this response is a bad response and that gets fed back in the model to improve the model. So there's a whole process here. And that prompt tuning is a GPT, it could be chat GPT actually doing the prompt? Well, I mean, engineering? I think it's a general concept that is being applied widely, right? So pretty much most LLMs that are out there now, you can feed them prompts and what they vary in is how big the prompt can be and how effective the prompt can be. We're talking human prompts, or not necessarily. No, no, the prompts can be generated by anything. They can be generated by machine, but the art that is still a bit of an art, Dave, is the prompts you put in really sort of impact the output that you get, right? So you're seeing an army of people springing up who are just trained in how to ask the right prompts. It's a bit of an art and it has to turn into a science. Right, really. So that's firm is being used a lot, prompt engineering. Let's talk about the interface between Snowflake and the NVIDIA stack. Now, might you have, say, a Streamlit UI where you could either have the end user prompting as part of a broader experience that's presented through Streamlit, or could there be some programmatic interface where you're curating the data that you're fine-tuning the model? What would that look like? So, you know, marrying the Snowflake data to your tool chain? Yeah, I mean, that's a great question. Okay, so firstly, there's the base case which anybody can do, which is all of these capabilities that I just talked about with NVIDIA Nemo, they're all on APIs. Okay, so with the Nemo stack already deployed that, if you've got an application that you're building, you can just call into those APIs. Okay, so think of it as a development step where you build the models, where there are APIs for fine-tuning and all of that, and then once the model is hosted, then there's APIs for basically using the model. For example, there's predict V2, which is a pretty commonly used API now to actually query the model, right? So that's sort of the base case through the API. And to add, any of those APIs are serviceable from a container running in Snowpark container services into a functioning Snowflake. And now you can call it from a number of use cases, a SQL query or a pipeline or even Snowpark. So that's base case. That's the base case. And then as we go forward, what you'll see more and more with the integration is, as we discover sort of interesting use cases for Snowflake's customers, that's where we can do much more curated, deeper integrations where the customer is not actually going to see the LLMs or play with them at all, right? And they're just going to do a higher level workflow and it's going to invoke the LLMs, right? I mean, this is the thing about AI, right? The true democratization of AI is when customers are using it and it's just sort of under the covers, right? So that's where we'll get to more and more for the use cases. So if I'm tying together last interview in this topic, we have these data engineering pipelines which are creating these curated data sets, heavily modeled and that becomes the context for either fine tuning or prompting. And in part of the flow of a Snowpark application. Yeah, that's right. So you can imagine these workflows, right? Where you're going through, it's always the data processing to produce the curated data, then the training of the models and then the application that is using the model, right? And so what you're talking about is that first part of the pipeline which is actually crucial for the AI process. We've mentioned it now many times in the last 48 hours at a conference, of course, many times before, which is the results you get with AI are a function of how good the data is. If you fine tune with bad data, you get bad results. So that's the beginning and you can tie it all together. Land data in Snowflake, ingest it, transform it. Once you're ready, now we want to go and call into Nemo and start doing prompting or tuning whatever you want. So let me take one application. Let's say in the past you had a deep learning model that was a recommender that was really complicated to train. It took data scientists to figure out what was going on. You trained it, you create an endpoint, you call it from your UI. Now you have something that's sort of pre-trained. You fine tune it with your sort of customer data, your product data, and then it's a much lower skill set to be able to create really sophisticated recommenders. That is the goal of all this work. Now you took recommenders as one example, of course, because anybody who's in online retail, for example, is doing a recommender system and you're right, you've got to hire just the right people to build your recommender system and we're sort of democratizing that because the barrier to entry is much lower. A lot of the data science has been done a priori by people like us to actually build the technology so that these models are being generated in the right way. But then the usage of the models is a lot simpler. That is really what's going on. And similarly, like the anomaly detection model might jumpstart someone to do really sophisticated security where before they didn't have the expertise to do that. Correct, and we're doing that already. There's actually a framework from Nvidia called Morpheus which basically does this. It does anomaly detection by doing AI and the difference is the state of the art has been that a human has to put in rules about what is good, what is bad, what's an anomaly. But with AI, it just observes the patterns of behavior and detects for itself what is normal versus abnormal. So that's another use case. Recommenders is one, securities another. But I think the important thing is that, if I may, the way we sort of simplify this all down into one concept is that the beauty of large language models is it's unsupervised learning. You put a lot of data in front of it. There's no human who's labeled every piece of data. And that's very, very powerful. But especially in enterprise applications, you then have to augment it with a relatively smaller amount of supervised training so that it really produces the best results. Right, and that's what this process is all about. So that's the setup for the question that we were talking about. If you're not going to ask it, I will. Let me set it up. Snowflake's not known historically for its ML AI tool chain. So your question, I'll give George credit for it. Can Snowflake use Gen AI essentially to leapfrog its position relative to those who have been using supervised learning? 100%. In some ways, you can say that Gen AI is resetting the race. Yeah. And when you have partners with this type of technology, you might start on the pole position like you're leading. You know, it's so interesting, Christian, because we've spent a lot of our energy yesterday and today talking about how together we're putting the engine in the hands of all Snowflake customers so that they can use LLMs. But the other thing that's so important to this partnership is for us working with Snowflake to do exactly what you talked about, right? To make Snowflake a better platform for its customers because of all the different ways that it's using Gen AI, right, in front of the data warehouse or as part of the data warehouse, right? Well, it's leapfrogging. I mean, look, you saw what Microsoft did. I mean, Microsoft wasn't really even into the AI discussion. And now they've, from a business model standpoint, it's leaped to number one. People say, no, well, Amazon's behind it. Adam Soliski has to defend that on Bloomberg. And he's changing what, in six months? Sighting, so a couple other quotes from last night. I got a share with the audience. Jensen said a large language model turns data into an application. The goal should be to build an AI application. Yes. He talked about it's the end of CPU scaling. He's talked about that a lot. Accelerated computing is now here. And in the last, the killer, the mic drop was we are going to turbo charge the living daylights out of snowflake. Yeah. And what he means is, so, I can give you a way of thinking about all those things in one go, Dave, which is that at the end of the day, who's Nvidia? Okay, what Nvidia's whole approach is that we've created this new style of computing, accelerated computing, where you run your workload on GPUs. It's dramatically faster. And because of that, it's dramatically cheaper, right? It doesn't apply to every kind of computation, but workload by workload, you can do accelerated computing. That's been our whole mission. Data processing, deep learning, inferencing, generative AI. And so when he says, turbo charge the living daylights out of this, what he says is, if I look at snowflakes platform as a workload, I know that if it runs on GPUs, it runs much faster. And then if I look at snowflakes, customers who want to do build AI applications, if they use large language models and they use the GPUs that snowflake is bringing into their platform, all the applications will run faster. And then the final part of turbo charging is the part about data to AI applications. It used to be that if you have a new idea for a new application, you've got to write code. But now what it is is, you just write the text for the questions your application would ask and you're done. So you can write applications faster and they run faster. Buy more, save more. You know, the business case is strong. And you're going to get people hooked. And then you're still going to get hammered for price. But that's the way our industry is, right? Deliver value, that's all you got to worry about. By the way, even in that one, we're very aligned. Price performance, performance is what matters, right? Exactly. You've heard us say it for forever. Absolutely. There's a price per unit and then there's a price per workload. It's got to be a denominator. I've done a lot of this work in my life. Yeah, no question about it. One really quick question, which is from the admin's point of view, Snowflake has always done a really good job of hiding the infrastructure, not worrying about provisioning it. Does the same carry through to the NVIDIA stack in the containerized service? Does Snowflake hide all that, you know, the deployment and spinning it up from the admin, from the Snowflake admin? Yeah, so our stack, right, can be used at two levels. One level is where the infrastructure is very explicit. You decide, okay, I want four GPUs and I'm going to run this and that. And then we've got stack at the level where you just submit a job, right? This is the thing I want done. The inferencing is just an API, okay? Every time there's a call, the model is served. So it really, it's the choice of the customer. Some people like to operate at a very low level but I think one of the benefits of this partnership is Snowflake understood very early the power of the serverless model, right? And so we're very keen to provide that kind of part of the customers, right? So, yeah. All right guys, we got to go. Thanks so much, Christopher. Thank you for having us. It's been great to see you as always. Absolutely a pleasure. And congratulations on the partnership and a lot of buzz here. It's going, AI, AI, AI. Very excited. All right, keep it right there. Lisa Martin and I will be back to wrap up. We've got AirBike coming on. They're a disruptor of the ELT market. Keep it right there. You're watching theCUBE live from Las Vegas. Snowflake Summit 2023.