 Welcome back to theCUBE's coverage of AWS Reinvent 2021. We're here joined by Ian Buck, General Manager and Vice President of Accelerated Computing at NVIDIA. I'm John Furrier, your host of theCUBE. Ian, thanks for coming on. You're welcome, thanks for having me. So NVIDIA, obviously great brand, congratulations on all your continued success, everyone. Who does anything in graphics, knows the GPUs are hot and you guys have a great brand, great success in the company. But AI and machine learning was seeing that trend significantly being powered by the GPUs and other systems. So it's a key part of everything. So what's the trends that you're seeing in ML and AI that's accelerating computing to the cloud? Yeah, I mean, AI is kind of driving breakthroughs and innovations across so many segments, so many different use cases. We see it showing up with things like credit card fraud prevention and product and content recommendations. Really, it's the new engine behind search engines is AI. People are applying AI to things like meeting transcriptions, virtual calls like this, using AI to actually capture what was said. And that gets applied in personal-person interactions. We also see it in intelligent assistance for our contact center innovation, our chat bots, medical imaging, and intelligent stores and warehouses and everywhere. It's really amazing what AI has been demonstrating what it can do and it's new use cases showing up all the time. You know, Ian, I'd love to get your thoughts on how the world's evolved just in the past few years alone with cloud and certainly the pandemic's proven it. You had this whole kind of full stack mindset initially and now you're seeing more of a horizontal scale but yet enabling this vertical specialization in applications. I mean, you mentioned some of those apps. The new enabler is this kind of the horizontal play with enablement for specialization with data. This is a huge shift that's going on. It's been happening. What's your reaction to that? Yeah, the innovation is on two fronts. There's a horizontal front, which is basically the different kinds of neural networks or AIs as well as machine learning techniques that are just being invented by researchers or in the community at large, including Amazon. You know, it started with these convolutional neural networks which are great for image processing but it's expanded more recently into recurrent neural networks, transformer models which are great for language and language understanding. And then the new hot topic graph neural networks where the actual graph now is trained as a neural network. You have this underpinning of great AI technologies that are being invented around the world. Nvidia's role is to try to productize that and provide a platform for people to do that innovation and then take the next step and innovate vertically. Take it and apply it to a particular field like healthcare, medical imaging, applying AI so that radiologists can have an AI assistant with them and highlight different parts of the scan that may be troublesome or worrying and requires more than investigation. Using it for robotics, building virtual worlds where robots can be trained in a virtual environment, their AI being constantly trained to reinforce and learn how to do certain activities and techniques so that the first time it's ever downloaded into a real robot, it works right out of the box. To do, to activate that, we are creating different vertical solutions, vertical stacks, vertical products that talk the languages of those businesses of those users and medical imaging, it's processing medical data, which is obviously a very complicated, large format data, often three-dimensional boxels. In robotics, it's building, combining both our graphics and simulation technologies, along with the AI training capabilities and difference capabilities in order to run in real time. Those are just too simple. I mean, it's just so cutting edge, it's so relevant. I mean, I think one of the things you mentioned about the neural networks, specifically the graph neural networks, I mean, we saw, I mean, just go back to the late 2000s, you know, how unstructured data or object store created. A lot of people realized about the value out of that. Now you got graph value, you got graph network effect, you got all kinds of new patterns. You guys have this notion of graph neural networks that's out there. What is a graph neural network? And what does it actually mean from a deep learning and an AI perspective? Yeah, I mean, a graph is exactly what it sounds like. You have points that are connected to each other that establish relationships. In the example of Amazon.com, you might have buyers, distributors, sellers, and all of them are buying or recommending or selling different products and they're represented in a graph. If I buy something from you and from you, I'm connected to those endpoints and likewise more deeply across a supply chain or warehouse or other buyers and sellers across the network. What's new right now is that those connections now can be treated and trained like a neural network. Understanding the relationship, how strong is that connection between that buyer and seller or that distributor and supplier? And then build up a network to figure out and understand patterns across them. For example, what products I may like because I have this connection in my graph, what other products may meet those requirements or also identifying things like fraud. When patterns and buying patterns don't match what a graph neural network should say would be the typical kind of graph connectivity, the different kind of weights and connections between the two captured by the frequency of how I buy things or how I rate them or give them stars or other such use cases. This application graph neural networks which is basically capturing the connections of all things with all people, especially in the world of e-commerce is very exciting to a new application of applying AI to optimizing business, to reducing fraud and letting us get access to the products that we want the products that they have our recommendations to be things that excite us and want us to buy things. But more. That's a great setup for the real conversation that's going on here at Reinvent which is new kinds of workloads are changing the game. People are refactoring their business with not just replatforming but actually using this to identify value. And I'll see cloud scale allows you to have the compute power to look at a node in an arc and actually code that it's all science, all computer science, all at scale. So with that, that brings up the whole AWS relationship. Can you tell us how you're working with AWS specifically? Yeah, AWS has been a great partner and one of the first cloud providers to ever provide GPUs to the cloud. We most more recently, we've announced two new instances, the G5 instance, which is based on the RA10G GPU which supports the NVIDIA RTX technology or rendering technology for real-time ray tracing and graphics and game streaming. This is our highest performance graphics and hands that will allow us for those high performance graphics applications to be directly hosting the cloud. And of course, runs everything else as well, including our has access to our AI technology, runs all of our AI stacks. We also announced with AWS the G5G instance. This is exciting because it's the first Graviton or ARM based processor connected to a GPU that's accessible in the cloud. This makes the focus here is Android gaming and machine learning inference. And we're excited to see the advancements that Amazon is making or AWS is making with ARM in the cloud and we're glad to be part of that journey. Well, congratulations. I remember I was just watching my interview with James Hamilton from AWS 2013 and 2014. He was teasing this out that they're going to build their own, get in there and build their own connections to take that latency down and do other things. This is kind of the harvest of all that as you start looking at these new interfaces and then new servers, new technology that you guys are doing, you're enabling applications. What do you see this enabling as this new capability comes out, new speed, more performance, but also now it's enabling more capabilities so that new workloads can be realized. What would you say to the folks who want to ask that question? Well, so first off, I think ARM is here to stay and you can see the growth and explosion of ARM led of course by Graviton and AWS and many others. And by bringing all of NVIDIA's rendering, graphics, machine learning and AI technology to ARM, we can help bring that innovation that ARM allows that open innovation because there's an open architecture to the entire ecosystem. We can help bring it forward to the state of the art in AI machine learning and graphics. We, all of our software that we release is both supported both on x86 and on ARM equally and including all of our AI stacks. So most notably for inference, the deployment of AI models, we have the NVIDIA Triton inference server. This is our inference serving software where after you've trained a model you want to deploy it at scale on any CPU or GPU instance, for that matter. So we support both CPUs and GPUs with Triton. It's natively integrated with SageMaker and provides the benefit of all those performance optimizations, all the things like features like dynamic batching, it supports all the different AI frameworks from PyTorch to TensorFlow, even the generalized Python code. We're activating and help activating the ARM ecosystem as well as bringing all those new AI use cases and all those different performance levels with our partnership with AWS and all the different cloud instances. You're making it really easy for people to use the technology. That brings up the next kind of question I want to ask you. I mean, a lot of people are really going in, jumping in the big time into this. They're adopting AI, either they're moving in from prototype to production. There's always some gaps, whether it's knowledge, skills gaps or whatever, but people are accelerating into the AI and leaning into it hard. What advancements have NVIDIA made to make it more accessible for people to move faster through the system, through the process? Yeah, it's one of the biggest challenges, the promise of the AI, all the publications that are coming, all the great research, how can we make it more accessible or easy to use by more people, rather than just being an AI researcher, which is obviously a very challenging and interesting field, but not one that's directly connected to business. NVIDIA is trying to provide a full stack approach to AI. So as we make, discover or see these AI technologies come available, we produce SDKs to help activate them or connect them with developers around the world. We have over 150 different SDKs at this point, serving industries from gaming to design to life sciences to earth sciences. We even have stuff to help simulate quantum computing. And of course, all the work we're doing with AI, 5G and robotics. So we actually just introduced about 65 new updates just this past month on all those SDKs. Some of the newer stuff that's really exciting is the large language models. People are building some amazing AI that's capable of understanding the corpus of like human understanding, these language models that are trained on literally the content of the internet to provide general purpose or open domain chatbots so that customers can have a new kind of experience with the computer or with the cloud. We're offering large language, those large language models as well as AI frameworks to help companies take advantage of this new kind of technology. You know, Ian, every time I do an interview with NVIDIA or talk about NVIDIA, my kids and friends, first thing they say, you can get me a good graphics card. Yeah, I want the best thing in their rig. Obviously the gaming market's hot and known for that. But there's a huge software team behind NVIDIA. This is well-known. Your CEO is always talking about on his keynotes. You're in the software business and you do have hardware. You're integrating with Graviton and other things. But it's a software practice, it's software. This is all about software. Can you share kind of more about how NVIDIA culture and their cloud culture and specifically around the scale? I mean, you hit every use case. So what's the software culture there at NVIDIA? Yeah, NVIDIA is actually a bigger, we have more software people than hardware people. People don't often realize this. And in fact, it's because of we create, it just starts with the chip and obviously building great silicon is necessary to provide that level of innovation but it's expanded dramatically from there. Not just the silicon and the GPU but the server designs themselves. We actually do entire server designs ourselves to help build out this infrastructure. We consume it and use it ourselves and build our own supercomputers to use AI to improve our products. And then all that software that we build on top we make available as I mentioned before as containers on our NGC container store, container registry, which is accessible from AWS to connect to those vertical markets. Instead of just opening up the hardware and then the ecosystem can develop on it, they can with the low level and programmatic stacks we provide with CUDA. We believe that those vertical stacks are the ways we can help accelerate and advance AI and that's why we make it so available. And programmable software is so much easier. I want to get that plug in for, I think it's worth noting that you guys are heavy, hardcore, especially on the AI side and it's worth calling out. Getting back to the customers who were bridging that gap and getting out there, what are the metrics they should consider as they're deploying AI? What are success metrics? What does success look like? Can you share any insight into what they should be thinking about and looking at how they're doing? Yeah, for training, it's all about time to solution. It's not the hardware that's the cost, it's the opportunity that AI can provide to your business and the productivity of those data scientists which are developing it, which are not easy to come by. So what we hear from customers is they need a fast time to solution to allow people to prototype very quickly to train a model to convergence to get into production quickly and of course move on to the next or continue to refine it. So in training, it's time to solution. For an inference, it's about your ability to deploy at scale. Often people need to have real-time requirements. They want to run in a certain amount of latency, a certain amount of time. And typically most companies don't have a single AI model. They have a collection of them. They want to run for a single service or across multiple services. That's where you can aggregate some of your infrastructure. Leveraging the trade and inference server I mentioned before can actually run multiple models on a single GPU, saving costs, optimizing for efficiency, yet still meeting the requirements for latency and the real-time experience so that your customers have a good interaction with the AI. Awesome, great. Let's get into the customer examples. You guys have obviously great customers. Can you share some of the use cases, examples with customers, notable customers? Yeah, I want one great part about working in video as a technology company. You get to engage with such amazing customers across many verticals. Some of the ones that are pretty exciting right now, Netflix is using the G4 instances to do video effects and animation content and for many we're in the world in the cloud as a cloud creation content platform. We work in the energy field. Siemens Energy is actually using AI combined with simulation to do a predictive maintenance on their energy plants and preventing or optimizing on-site inspection activities and eliminating downtime, which is saving a lot of money for the energy industry. We have worked with Oxford University, which is Oxford University, actually has over 20 million artifacts and specimens and collections across its gardens and museums and libraries. They're actually using convenient GPUs and Amazon to do enhanced image recognition to classify all these things which would take literally years with going through manually each of these artifacts using AI, we can quickly catalog all of them and connect them with their users. Great stories across graphics, about across industries, across research, that it's just so exciting to see what people are doing with our technology together with Amazon. And thank you so much for coming on theCUBE. I really appreciate it. A lot of great content there. We're probably good to go another hour. All the great stuff going on in the video. Any closing remarks you want to share as we wrap this last minute up? You know, really what NVIDIA is about is accelerating cloud computing, whether it be AI, machine learning, graphics, or high performance community simulation. AWS was one of the first with us in the beginning and they continue to bring out great instances to help connect the cloud and accelerate computing with all the different opportunities. Integrations with EC2, with SageMaker, with EKS, and ECS. The new instances with G5 and G5G, very excited to see all the work that we're doing together. Ian Buck, general manager and vice president of accelerated computing. I mean, and I'll give you not love that title. We want more power, more faster. Come on, more computing. No one's going to complain with more computing. Ian, thanks for coming on. Thank you. Appreciate it. I'm John Furrier, host of theCUBE. You're watching Amazon Coverage ReInvent 2021. Thanks for watching.