 Welcome back to SuperCloud 6. I'm the guest host here, how we shoot and my longtime Silicon Valley executive in the AI and the data space. I cannot be more excited to welcome my two distinguished panelists and I wanted to met whole from NVIDIA and a bearish from Snowflakes. These two companies are the Silicon Valley darlings of the last few years, and we are going to talk about AI and the data. So I want to introduce out to our audience a little bit about yourself, and in particular, how did you get into the AI space? When and how did you get to the AI space first? Maybe we can start with Matt. Yeah. So thanks for having me. I'm thrilled to be here. I'm Matt Hall. I look after our global AI solutions go to market in NVIDIA. I've been in NVIDIA about six years. How I got an AI, a long journey through many different twists and turns in the IT infrastructure space. Different avenues around AI led to me culminating at NVIDIA over the past six years, and it's been quite the ride to really be at the center of the storm of AI and watch it grow. Bearish. My name is Bearish. I run the AI and ML product teams at Snowflake. My journey into AI started more than a decade ago. When I was at Google and then we built a early version of the Google Assistant, and I ran the Google Assistant product teams there, and then did a startup which got acquired by Snowflake, and now I'm core at the heart of the enterprise AI. And the center of the universe now. So speak of that. Last December, CNBC ran an article. The title of that is, along the line of 2023 is a great year for AI. Everyone knows about it, but then that was a year of hefty profits for NVIDIA, lofty experiments for the rest. So I would like you to comment on this one. Obviously, that's pretty precise for 2023, but we are anticipating a new year 2024. Is 2024 going to be a very different year? Is there going to be a lot more than just the experiments other than lofty profits for NVIDIA? Maybe you can start with NVIDIA, where the company just broke the record again and again in the last few quarters. Tell us more about both the profit and then hopefully more, how and when are we going to get out of the experiment stage? Yeah, so I think this year is going to be a massive sea change. Obviously, the big explosion was chat GPT really woke up everyone, every enterprise, every individual, every researcher out there is to what was possible with AI. Over the past year and a half, we have seen a lot of experimentation. We've seen a lot of progress. The progress has been around the consumer generative AI applications. We've seen some enterprise generative AI, large language model builders that are off doing wonderful things. But what happened with chat GPT is it really spurred the enterprises to start moving in the direction of deploying AI. And yeah, it was a lot of experimentation at the beginning. They have to figure out how they're going to implement AI. And what we're starting to see are the real green shoots of true enterprises, financial services companies, pharma companies, auto manufacturers that are looking to deploy AI at scale. They've realized the advantages to their business of deploying AI. And the interesting piece is not just one use case, they realized that they can implement AI into many different parts of their business. So I'd like to think that this year will be the year of enterprise AI. So just along this line, what did you observe last year that that was the bottleneck for people to put that into production or to run that at scale or to generate the real revenue? Why that's going to be changed in 2024? Yeah, I think there's a couple of things. And surprisingly, one of the biggest is the culture within organizations. AI is a very new thing. It's not traditional IT. And it requires connected tissue between various personas within a company that may not have existed in the past. You have the line of business folks that realize that they need some sort of business outcomes. They're working with data scientists, which is an entire new position within companies. They're working with the IT folks to figure out how they get access to data and to compute. So I think the organizational structures and the connected tissue within organizations is going to be a big part of it. I think there's going to be shared learnings. I mean, there already is a lot of shared learnings out there. And next week, I know you're going to be at GTC, which is our big conference. And what we do at that conference is we allow folks to share best practices and learnings. And we're really starting to see that snowball. People are starting to figure out, no pun intended with Snowflake, but they're starting to figure out what works and they're sharing that. And other people are germinating on that. And it's becoming this really neat flywheel of improvement in the way enterprises are going to. Interesting. So from your point of view, it's not just the technology, but also the culture, the best practice. But hopefully a year passed by, we are in a much better position, right? Absolutely. Parish. So very similar. Last year, it was all about experimentation. We had a lot of our customers starting to think about all the different use cases that are in enterprise AI. And this year, all of those are moving into production. And a lot of our customers had to figure out, how do I make sure that I trust the system that's built? How do I make sure that I have my data and the governance in the right state before I kind of take that data out? So all of these frictions that are introduced are now being figured out. And a lot of our customers who have hundreds of use cases moving into production systems. So you're seeing a massive move from the experiment to the production this year? Absolutely. A massive move. And then with that move, of course, there's a lot more thought into and how do I manage the system? Is the cost the way I want it? Is the quality the way I wanted? How do I manage the hallucinations that are kind of inherent in these systems? So a lot of moving from a proof of concept to production is kind of sweating that remaining 20% that takes majority of the time. So both of you are very bullish about moving from the experiment to the enterprise real adoption 2024. Absolutely. Cool. So speaking of that, there's also a technology aspect. And Jensen Huang has been on many, many talks in the recent past. And then I heard him say about Moore's Law quite a bit. Moore's Law was dead. And to me, as a person who has counted the CPU cycles for 10 years a VMware days, that's something. But at the same time, Jensen also said that if you look at the last 10 years, the AI technology, the hardware technology has moved 10 times 100 times. I don't think he's quite saying that the hardware technology is dead. He's just saying that Moore's Law is dead. So I wanted to understand a little bit more, like what does he mean by Moore's Law was dead? Is that or is that going strong? I think that's a multifaceted question. So Moore's Law, as you know very well, is all about the number of transistors on a CPU doubling every two years. And that held true for quite some time. What we're seeing in the CPU space, though, is you can no longer continue at that pace. And even Moore himself predicted, I think it was in 2006, that Moore's Law would be coming to an end. CPUs are still very necessary in the great realm of things. All of our accelerated computing has some sort of CPU in it. But the CPU innovation just isn't there. We're seeing companies that are accelerating or extending their depreciation cycles of CPU servers from four years to six years because they're not getting the gains. What we're doing with the GPU is because of the parallel processing power, the increases that you see generation to generation are massive. And we're also able to create new generations of products every 24, 36 months. So we're really breaking the barriers of what Moore's Law was set out to say around the CPU. But beyond the chip itself, how you mentioned, it's the entire infrastructure. So obviously at NVIDIA, in the industry, we're very concerned and excited about what the computing power of the chip level can do. But if you don't have the right networking, if you don't have the right data movement, you don't have the right storage, you don't have the right software stack, you don't have the right data center set up, it all doesn't matter. It needs to be one cohesive system that works together to provide the best outputs. So we obviously look at the output at the chip level. But for us, it's really the output at what Jensen likes to call the AI factory, the full data center level. And how you bring all the components together in harmony to produce very quick and accurate results. So perhaps what Jensen really meant is the Moore's Law on the CPU level may be dead. However, at a data center level, at a cluster level, at a much bigger sort of the large scale level, there is still a very strong Moore's Law or whatever the law, right? Strong Moore's Law, and even at the GPU level. I mean, if you look at our V100 chips to our A100 chips, we went from about 20 billion transistors to 50 billion with our H100 or about 80 billion transistors. So we are increasing the transistors, but that's just one piece of the overall equation that's allowing us to get to this million times improvement over the past 10 years that Jensen likes to talk about. Right, right, right. You mentioned the 50 billion transistors. Intel CEO Pat Gessinger actually recently said, by 2030 he wanted to put trillion transistors on the chip. So from my point of view, it sounds like there's a strong hardware evolution going on at both chip level as well as at a wider, probably more important at a data center level. Absolutely, absolutely. More importantly at the data center level, if you don't bring it all together, then you can't produce results. And that's the interesting thing that we're seeing in the market today. Everybody talks about this chip shortage. Can't get enough chips, but the chips quite frankly, aren't the long pole in the tent a lot of the time. It's the networking gear or it's the data center or it's the data center power. And it's cooling, right? And it's really thinking about the entire ecosystem of what you need to do your work. And that's where Nvidia is very focused today. So we're not a chip company. We're not a networking company. We make chips. We make networking products. We make software. We're not a software company. We're bringing it all together to provide full AI solutions. And that's the way you get the benefits and the gains. Yeah, I just saw the news that the DGX is going to be cooled by liquid. So good stuff. So let's come back to the software a little bit, right? Barish, you came to Snowflake through the acquisition. And as you mentioned in your own journey, you did a lot of consumer stuff from Google days. And when you and I chatted before, you mentioned that you actually looked at AI technology. You thought, wow, that's actually more applicable, interesting to the consumer space. Back in the day. Back in the days, what changed? And then what's the new realization? What made you to have a new realization? So before GPT happened, before these large language models, all of the AI systems were very much hand tuned in the sense that you would have to map any input to very specific things and hand curate every experience. With large language models, all of that changed. Now you have one general purpose model rather than having the engineering team and to know how to build very specific experiences, you could just write instructions in English. And then that would get you a long way there. So I think what changed is over the last year, year and a half, now we have these very capable models that don't require these kind of research organizations to build very specific systems. Instead, you can start with a large language model and then kind of fine tune it to get what exactly what you want. The quality increased and also it now became a lot more available for everyone in the industry to build. So you are basically saying that AI technology is more democratized, right? So that you can leverage the AI technology through maybe problem engineering or those things so that you don't need PhDs or sophisticated data science knowledge in order to get something going. That's correct. So from that point of view, you feel like this time around the AI technology is going to make a massive impact to the enterprise space more than just the consumer space. Absolutely. And I'll also add that the kinds of use cases that are now possible with large language models are very specifically applicable to businesses. When I talk to customers, every one of these customers have many, many use cases that can just increase the efficiency substantially that can create these new use cases from marketing to sales to increasing, you know, efficiency. Can you share with us some exciting use cases that you thought, wow, this is exciting. I mean, there's a wide spectrum. We have the most common use case that we see is kind of increasing the efficiency of internal teams. So one of our customers, you know, large financial bank wants to empower their sales teams to easily access all the documents that they have in their system to be able to do that like that in a very fast way while they're on the phone with a customer who's asking about a company, that's now possible. And other use cases to be able to, you know, generate very custom, you know, marketing content across your many, many, you know, customer base, right? So creating efficiency, but still bringing in personalization is a core new. So those are the pretty tailored smaller use cases, you know, very custom to a specific company, right? That's correct. And a lot of these companies, what I see happening right now is a lot of these companies are now saying, hey, these large language models are available to everyone. How do I differentiate myself? And the way they differentiate themselves is with their own data. So they can take a large language model, fine tune it with their own data for their specific needs. That increases the cost effectiveness of that experience, increases the quality and allows them to differentiate against somebody else who doesn't have that data that they do have. Very nice. Matt, do you want to share some interesting, exciting use case you have seen recently about GenAI? You know, we used to be able to pick out one or two use cases and highlight them and talk about them for hours. I'll tell you, the use cases are endless right now. From chatbots to recommender systems to customer service to routing, there are so many use cases. And what we used to see were enterprises that said, I'm going to pick one use case and work with that. We're now seeing enterprises come in to have 20 different use cases. How can I get access to the right tools that allows me to engage in all those various use cases? We did a success story with Domino's Pizza last year when you start talking about pizza delivery. The use cases are endless. It's absolutely stunning. I think what people are beginning to realize is that the integration of AI into every application and to every team is really something that's possible. And it's a sharing of these use cases. It's a general knowledge across the industry as to what the benefits and returns on the investment will be. And then you'll just see things continue to move faster and faster as people realize what's possible. Very nice. So this session, you know, the panel is about AI cloud, right? How should people think about what is the AI cloud? Because when people think about cloud, there are big threes, right? AWS, Azure, or GCP. But when it comes to AI, what does that mean? Of course, you can consume the cloud or AI service from the big three for sure, right? But then, you know, how does the snowflake and then Nvidia come into the picture? Because there seem to be many layers of the cake. So, can you? I can get started. For a lot of snowflake customers, data governance is a very core need. So what we want to enable is we want to bring compute to where the data is. Because our customers establish all these access controls and the data governance bringing all their data together in one place. And they'd like to allow easy access to large language models from within that data governance access. So for snowflake, what this means is having large language models running inside this parameter, having easy access to search, which is a core part of building these chatbots, easily accessible from within snowflake. So when we think of all the toolset that's required, the goal is to bring the compute to where the data is so that we break all these data silos that are otherwise going to be created when you take the data to one vendor for one thing, another vendor for another thing. So we'd like to consolidate all the data in one place and bring all the LLM functionality to there. So in that context, so as a customer, how do I think about it? I use snowflake data cloud, along with the big three or... So we give a lot of... All of those options are available. For a snowflake customer who wants to keep their data inside snowflake, we bring all the large language models that are very capable into snowflake and all the tooling around it. So you don't have to go elsewhere for your needs. However, we also make it very easy for you to take that data out if that's what the customer wants. There are plenty of options. That's right, cool. Nat? Love the comment, bring the compute to the data. I mean, AI is all about data. It's taking lots of data and processing it quickly. Moving data is very hard. The cloud, I think, is often misunderstood. People think about the cloud as some public cloud in AWS, a GCP, and those are absolutely clouds. But at its core, cloud is a set of shared resources. You can have a private cloud. You can have a sovereign cloud. So I think you're going to see a lot of various options for clouds pop up. Obviously, we're partnered very closely with Snowflake for their cloud with AWS, with GCP. We're going to see enterprises that build their own private clouds, shared resources across their organization. We're going to see governments, countries, regions that build their own sovereign clouds to serve out their constituents. You're going to see universities that build their own clouds. So I think the normal thinking about cloud being one of the big three is absolutely going to be a huge growth for AI. But you're going to see it in other places as well. And it's going to be all about, as I said earlier, as Jensen talks about building AI factories. And these AI factories are going to be a quote, unquote, cloud of shared resources that can be shared in various manners, whether it's rent by the hour through AWS, whether it's a bill that you pay to serve your own constituents. But it's going to be a very new world as it relates to cloud. So Snowflake is obviously leveraging the video, right? But where does it happen? Can you explain to the audience, what does the layer of the cake look like, right? There's a Snowflake data cloud on top of, some big three on top of what? NVIDIA cloud or NVIDIA cloud? How does it work? How does it look like? Yeah, so our goal is to democratize artificial intelligence. So we sell anything to everyone. We power all of the major public cloud providers. They're using our compute resources, our networking, our software. We're working with a lot of the regional cloud providers, the sovereign clouds, that are looking to build their own clouds. We're providing technology. We're providing enterprises with full sets of solutions that they can run in their own data center or that they can run in co-location facilities. And we're also coalescing all of the cloud resources together in something we call DGX cloud. So what we're doing is working with our cloud provider partners to become their customer and deploy our cloud, our software stack, all of the goodness of NVIDIA on top of their infrastructure as a service. So to answer your question, there's lots of different ways to consume our technology, to consume it easily, and that is 100% by design. So it's not just one way to engage it. On prime, in the big three, DGX cloud, public cloud, right. So one more question I have is about not just big companies, but also for all the startups out there, any opportunities you see for the startup, or what is the why space, or there's no why space, right? Because it's AI is too hard or whatnot. So any comment, and maybe we can start with Barish, you came from the startup world. I think this question is dear to your heart. It is dear to my heart. And so I'll give a couple of examples. Much of the investment we see investment so far has gone into foundation models. And a company like Mistral, for instance, with just 20 people within 10 months was able to create as good a model as GPT-4. And I think there is still a lot of innovation waiting to be unlocked in that foundation model layer. And it is very resource intensive, and therefore a lot of investment is going there. But beyond that, we're starting to see application layer develop, and then there's a lot of tooling that's necessary. I expect over the next two years, kind of new consumer startups will start to emerge beyond enterprise. But also within enterprise, we're seeing a lot of specialization. There is going to be a ton of opportunities to nail the marketing use case, to nail the legal use case, and to nail the healthcare use case. Huge opportunities. You don't see the winners are obvious today yet. I do believe that there is going to be billion dollar companies in each of these categories. Very nice. Matt? So I share the optimism. I think there's going to be a massive influx of new companies. There are going to be the next big names out there that are doing a lot of different things. It's not just building the large generative AI models anymore. There's going to be domain specific parts that people are building. My advice to the startups are they have the advantage of being nimble. They have the advantage of not having to worry about the organizational constructs that I talked about earlier in a large organization. And they have the advantage of having access to things that large enterprises don't. NVIDIA, Snowflake, every other company out there has programs for startups. We like to call it inception partners, people that we want to work with and help them grow. And the benefits of working with these types of companies and using our tool sets are things are readily available, easy to use. We're huge fans. We work with many, many startups. The whole developer community. I think we have tens of thousands of developers in our program today for startups. And we're going to continue to incubate the startup community because it's just going to be massive over the next few years. My last question is actually about the resource, right? GPU shortage, as we all know, right? The, you mentioned that a startup would have a lot of resource even to NVIDIA. Is that really true? That's one. But the other thing is really, where are we on the, for the cycle, right? There's clearly some GPU shortage to a degree, depending on which angle you're looking at it. And but where are we on this cycle, right? Because if I think about the internet early days, right? My friend, Martin Casado, and I had some Twitter exchange last week. He said, hey, this GPU shortage looks a little bit like fiber shortage. We saw 20 years ago, but then in a few years, we saw the fiber glut, right? I mean, it's natural, right? Comment ago. When are we going to see GPU glut? Do you see that happen sometime soon? Maybe. Yeah, I don't think we'll see a glut. So you're absolutely right. We've been in a constrained mode for some time to demand has exceeded supply. The demand has exceeded supply, not just for GPUs though. As we talk about the entire ecosystem, you have to have enough diesel generators to power data centers, eating enough coolers, chillers to power the infrastructure and keep it cool. You need networking gear. We've seen constraints across the board. The GPU just happens to get a lot of the attention, right? But once we have enough supply for GPUs, there's going to be other pieces that we're going to need to worry about. We're going to make sure that networking cables are there, that we have the right folks and the resources to stand this stuff up and manage it. So there's a lot of pieces that come into it. In terms of GPU shortages, we saw a massive swell in demand, not surprisingly, it's been in the news. And we believe we can serve all of the demand in the future. One of the things that people forget about is we make these chips and these systems much more powerful. You need less of them to do the same amount of work. So we have next generation technologies that we'll be announcing as we normally do in the future. Next week. Maybe next week, we'll see. And we'll be talking about the performance gains. So I think we'll be in a good spot that over the next year, you're really going to start to see the supply come in the line. One of the things that we've been very purposeful about is not just flooding the market with GPUs. We're very careful to make sure that we're giving GPUs, providing GPUs to the customers that need them the most and are most ready to do the work. We don't want inventory of GPUs sitting in data centers, inventory of GPUs sitting in boxes. That's how you wind up in the glut. So we've been very purposeful about watching where our inventory is going, making sure it's all being used. It's a holistic story. Holistic story, absolutely. Barish, any final thoughts around this line? And then maybe, do you see the shortage or glut? Or how do you see the end game in the next few years? I'll go with the optimism, which is right now everyone is moving as fast as possible. I've never seen the industry invigorated as much as it is right now. We have the compute available and we're empowering many customers to build amazing things. So the shortages from a customer perspective is not on the compute resources. It is on how do we make everything super simple? How do we empower all these customers to build great products over the next year? And it is all coming. So I'm very optimistic and positive that the next year is super exciting. Thank you, my two distinguished panelists. Clearly, two of you are very, very optimistic. 2024 hopefully, and they look like a year that we are going to be full of not just experiments and then also a lot of the production, the real adoption. And then as both of you mentioned, right? This is a holistic game, right? Not just one part of the technology, GPU. There's many, many different parts, software, best practice, data center wide. And ultimately, this AI is going to change the world in a holistic fashion. Thank you for being with us. Thanks for your evidence.