 Welcome back everyone to the live SuperCloud 4 episode here in Palo Alto, I'm John Furrier with Dave Vellante, my co-host, all day and all day tomorrow coverage around what is Genevieve AI? That's the topic of this episode. Every quarter we unpack the most important topics. I'm sure AI Genevieve AI Dave will be around for a couple more episodes. We'll weave in cybersecurity, other key things. But as it's going on, this is a major shift. You're hearing all the reports come in. This is a moment in time where a new generation is coming in for the dawn of a new era of computing, software, data. User experience, everything. Our next guest is from Intel, Arun Subramanian, vice president of cloud and AI at Intel, formerly the AWS, HPC background, quantum computing, guru running AI over there at Intel with Pat Gelsinger. Thanks for coming on SuperCloud. Thanks a lot for having me. So we were talking before we came on camera as we were getting set up here, just almost pinching ourselves with excitement around how cool the current environment is right now with AI because all the generations and decades of work around AI and machine learning and intelligence around data has kind of had this potential energy has now been released with the current cloud, next-gen cloud and with the advent of the promotion of chat GPT open AI educates and consumerizes the user experience. And then now a bunch of technologies hit. So we are now in an era where it's now known that this is a new path in front of us. Like the web, hey, the browser, okay. Nathan, more things are coming. What is the key thing that you guys see at Intel because the compute's a big conversation, it's not just about computes, but why is it happening now? Why is AI so perfect for the shift? So everything you mentioned up to now, right? So the advent of the cloud, the massive amounts of data we've been able to collect, all of the data transformation efforts in the industry and enterprises that's gone on over the last decade, right? Now if you ask anybody what is the ROI that you got out of that, like based on any number of different studies, it's about eight to 10% up to now, right? So when people say they've been doing AML forever, that's the ROI they've been able to get. And I'd like to tell you a story that one of the board members I was talking to from United Airlines, where his point to me, like this was a couple of years ago, was look, AML is great, you got some great technology, but I just don't have the high-end data scientists for me to transform all of this into production. I do not have that many data scientists that I could go hire. The single biggest difference between that and now, after the Gen A advent really is, you still need data scientists, but you don't need them in armies. You can combine your business expertise, your business users, along with a large language model that is either from an open source community or something that you build together along with the community that you can then put to use that would fundamentally shift your time to value, right? That's the significant difference we're seeing now that you were not able to tap into before. Of course, it is not possible without doing all the investment on data that's been going on for the last decade or so, right? So most of the AI today in the enterprise is still an evaluation phase, but everybody's evaluating it. So we know that. So is the premise then by your statements that the ROI this time around will be for the roof? It's faster and through the roof. I'll actually give you a quote, a study that just concluded recently in the couple of weeks ago. It was between MIT and Warton and Boston Consulting Group where they went and surveyed more than 600 different enterprises in the US and the world around. And the questions they asked was, are you of course experimenting with JNAI? But they also asked in your view, are you already moving closer to production? And they actually defined what production was, which is you need to have at least two projects that have gone through the phases of multiple departments saying yes, and you're actually putting it into a critical application. That's a higher bar for enterprises to adopt. Surprisingly, 11% of the companies came back and said yes to that answer. And I just want to give you a metric. If you go look at McKinsey reporter, BCG reporter, any number of different reports, it's about eight to 9% ROI on the traditional AI ML after a decade of adoption. Here we're talking six months, nine months, 12 months. That's an astounding number. Now of course we'll have to see if it sustains and continues to grow, but everything we're seeing in enterprises, it's not a, it's both the bottoms up and top down push in terms of adopting. So on the numbers, you said so multiple years they get eight, nine percent ROI from the autobahn. So this is the eight to 10% of the projects actually get to the ROI that they promised in the beginning of the project. That's anything AI ML and data transformation related. I mean, there are multiple reports that are published, whether it's IDC or McKinsey or BCG. How do you see that changing with GenDevi AI? So with GenDevi AI, we're already seeing adoption north of that. And we are less than a year into enterprise adoptions for that. Now the final solution to getting outcome back really would come from our more and more business users using it, right? And today you can see that that shift has already happened, right? So the number of times a business user actually says, you know what, I didn't use my business data, but then I used a dummy data set with something open source to do it. I need access for this insight. What are they doing with, what does your data suggest they're doing with? So I would put that into two large buckets, right? So the first one is what I would put is knowledge discovery as a bucket. And what I mean by knowledge discovery is more than search. So you have a bunch of documents, whatever the document could be. Could be a text based PDFs, Word documents, PowerPoint, whatever you have. You want to extract information, but then it's more than just searching for keywords. It's about saying, I have a problem, give me a nuanced answer based on the documents I have. That's what I put under knowledge discovery. And a nuanced answer would have retrieval and then generation, right? So it's both of those they need to do. And they're probably leveraging vector databases for the embeddings. Some of them are, some of them are directly hitting models. Going from a model, by the way, models are a means to an end. They're not an end in themselves. So you need to put in a lot more infrastructure to get that done. But that's one bucket of use case. The second bucket of use cases is what I would put in the bucket of augmented services. Think of it as co-pilots or design augmentation services that you think of. The reason they're slightly different is you need to still retrieve data. But in addition to that, you also need to add in a business knowledge or a domain expertise along with it. And then do something. I'll give you a specific example, right? So you look at the financial services world. You can extract information from anything that is public. You get the information out. Now, if you want to do risk analysis, now that's a field that is well evolved. They have their own mathematics defined. You don't need to reinvent it. You can build a model for that, but why? They already have models that they trust. So take the data out using LLMs. Feed that into an existing model. Now, whatever results comes out, you still have to process it with LLMs to give a user an output. That's meaningful, right? So that's what I would put in the second bucket of information. Sounds like middleware to me. More or less, right? So it depends on what you would call middleware, right? Because, yeah. What about code generation? Would you... So code generation today is in the first bucket, which is like just extract and generate. But where you would get more information in the panel before us was talking to that topic, saying, I just don't want a dumb generation. I wanted to know who I am. I want to know my context. And you wouldn't... Where would you put like chatbots and chat? LLM powered chat... Would you exclude that from that? Not at all. So LLM powered chatbots pop into both those use cases, right? It's one manifestation of how you give the user the ability to interact with it. I think we've all been conditioned so well with the open source tools that we all expect to chatbot out there. Whether it's text-based or audio-based or image-based or video-based, that's just something that is a reality that we live with. So we've got two buckets. Is there another bucket or is it the main two? I think those would be two broad buckets. You can kind of put most use cases into one or the other. Okay, so this comes up to the point about agents. We heard about, okay, chatbots, let's put that aside because that's an entry level, in my opinion. Then you got co-pilot-like augmentation, humans plus AI. Then you got predictive and other AI-native-like things. That's right. So the second bucket is more predictive, right? Because the predictive nature comes not only from the individual generative AI models that you're bringing in, but you're also bringing in a domain, specific model that'll allow you to make the predictions with a lot more grounding, right? The reason I say that is even today, the best models out there are pretty terrible at counting. Try a fifth grade, say, math Olympiad problem and it will run around in circles. Once in a while it'll get the answer right, but that's because it's guessing. So how about this grounding feature? Because I like what you said because we had problems to power law, we put up a graph earlier in the session around the LLMs from size and smaller models are big. It's almost as if the words are just changing in the past, just in the past year. Open AI was considered a proprietary model. They're actually open, okay? Because it's kind of open web. Small data sets that are proprietary, intellectual property is called proprietary. You have walled gardens or closed data sets, which you want to protect because that's IP, you don't want leakage. So you have almost a whole nomenclature and the notion of memory, by the way. So this could be memory, physical memory or memory for retrieval. What is this telling us? So here's some other metrics for you, right? So the best models out there with all the controversy about data and all of that stuff, there may be seen 5% of the world's data. If you're being generous, maybe 6%. The rest of the 95% is still dark and it'll continue to stay dark. The companies are not going to open up their data. Even if they build models, they're not going to be able to open up the models. And those are in their own premises. In their own premises, their own enterprise data, their own data that they've spent decades generating, for example. So queue up the power law, if you would, because I'd love to get your thoughts on this. So it's, we won't go into too much of you, but you know power law is sort of, took a little liberty with the definition, but the point being, you got the cloud guys up in the top there, that cluster, and then you've got a long tail and then you've got on-prem and of course edge is a whole another beast that we'd love to talk about. So you're basically saying that dark data would be largely on-prem. This is not by the way dollar-based. The dollars are huge in that long tail. But so are you seeing a customer saying, we want to do this on-prem with our own data? And how is that playing out? Got it. So the notion of on-prem for us really is we are living in a hybrid world. It'll continue to be hybrid, right? So what you call on-prem versus what you call private cloud versus what you call public cloud are kind of morphing. But it's kind of where the data is. Data is, so most of the data is on-prem today, right? And that's kind of the growth story that has been told about the cloud itself, right? I'm kind of getting a, I guess, I guess. I'm getting a lot of heat. If you exclude, if you exclude video, most of the data is on-prem. Do you agree on that? I've been getting a lot of heat on this one against your reaction. I've been saying publicly, and everyone knows I'm a cloud person. I love cloud, I love Amazon, I love the work you've done over there and Andy and team. But if you look at what the trends are pointing to power law, there's a shift. Not repatriation. There is data on-premise. So what's happening is the word data center is coming back. Again, the words are changing. Proprietary, walled garden. Data center's not a bad thing anymore because if you have data on your premise, and that's- If you take use of it, make advantage of it. The operating model could be cloud operations. It could be cloud operations. It could be cloud native versus cloud adjacent, you know? Data centers are back then and they've never went away, Dave. They never actually- Data centers are booming. So I have a slightly different take on that, right? So your exponential curve- Yeah, bring that up again. What my take on that is you're going to see that you're going to go and change your axes on them where this curve is going to get dwarfed based on what's going to happen in the specialized AI world. So take even, for example, one industry. Take the pharmaceutical industry, for example. They are collectively sitting on data that's about 100x more than all the data that was used to build the largest models today. Even if one of those companies attempt to go and build the model for themselves, it will dwarf that curve, right? Now the notion of, oh, why would they? I'll give you another analogy. The best model out there today, I would equate it with no disrespect to a mediocre high school student, okay? And the reason for that is if you go look at the cognitive abilities of that model, it's way lower than a high school student, but it's seen more data than any high school student could ever do. So that's why I'm putting that as a mediocre high school student. Now domain-specific models that understand specific domains. Illucinations, I can see why. Illucinations come in, Dave. Exactly. That's college. Yeah, that's college. So a domain-specific model, like, say, the Bloomberg GPT, I would put that as a decent college student. They understand a particular domain, but then they only understand that they've not seen any real-world usage, right? The models that the companies are going to be building and continuing to build on their expertise are the expert models. Now, they can be small. They can be very, very large, depending on what models they're building. And those models have not even started yet. This is to your point about the model. But your point is that size of the model on that long tail is actually gonna... It's gonna be changed. It's gonna force us to change that. So back to the model. So the neck and belly will increase. The tail will still be there because you're gonna have small language models that are highly accurate. Highly specialized, highly accurate for specific use cases. And especially your edge, right? So that brings back to, say, Intel's message. You started the question of what is Intel doing? Really, it's about AI everywhere and it's not just about AI in the data center. What's changing with respect to data center is workloads are now data center wide, sometimes cross data center. So if we are looking at data center as a compute unit, the same way you would look at an edge device as a compute unit. Okay, but so bring that back one more time. You're suggesting, Arun, that where it says specialized AI, it's gonna be like a snake swallowing a basketball. That's right. It's gonna go up, the red line. It could go up. Your red lines are going to dwarf. Because I mean, if you want an analogy, just look at the cost of sequencing a genome. It's a camel. Right? Now the cost of sequencing a genome went down, but imagine if every human being walks into a doctor's office and in five minutes you need to sequence a genome. Just the volume of genome sequencing would dwarf everything else that's been done up to now. Notwithstanding the training and cost involved for the pharma company to do that, but that would be hurtfully end. But still, assume tomorrow's law continues, all the good stuff happens. I got to get your take on that, because we agree with you, by the way, 100% validates our model. I actually have a comment on that. Sorry to interrupt you. Go ahead. So a pharma company takes anywhere from two to five billion dollars to build a new drug. I'm using the word built. Spending 10, 15, 20, 100 million to build a model. No brainer. Yeah. It's a no brainer. They just not gotten there yet. The ROI. But they will. Yes. And you're starting to see these little specialized models. Moral starting. Security models. Google's got one. Yes. A health one for Amazon. Yes. B2B, linguistics, Q. Yes. We get detect jargon. Do you think, just take that, because one of the areas of focus of the super cloud four is industries, do you think the incumbents, the big pharma, have an advantage there or is it going to be disruptors in that industry? It's interesting. It's you said that it's actually both, right? So the disruptors absolutely have an incentive to go do that. Unfortunately, they don't have access to the data sets. Ah, okay. Whereas the incumbents have everything to lose. So they're like kind of slow to get started, but they also don't have the expertise to go do this. So M&A is going to happen. So they'll have to do some kind of partnership. It's going to be a whole nother market dynamic. It's interesting in the power dynamics there. So on that point, one of the things we've been saying, I go back to the cube tapes probably 2013, Dave when we first postulated this notion of horizontally scalable cloud, but vertically specialized domain expertise in the app, which we were kind of teasing out would be AI ones, you know, of course, we had it right early on, self promotion plug there. But now you're seeing, now you're seeing, okay, we were right as usual. But now you see all the vertical action in the industry. So that, cause that's what the domain expertise is, the data. So what, what has changed now? We're hearing other things like compute's going to be horizontal, but data's got to also be horizontally scalable too. So you have two issues. How do you make scalable freely available horizontally while maintaining the specialism of the vertical focus? So I think data being freely available might be a notion that's okay inside an enterprise, right? But I'll just try to enterprises from sharing data with each other, right? So we maybe are walking towards a world where there is a need for data exchanges that is more than what is today. But we're not even cracked the, the problem of enterprises themselves sharing data within themselves. There's so many silos inside enterprises, right? So different business units have their own data sets. They don't necessarily want to share. So those silos you're starting to see forming cracks is where I will put. But AI, the premise of AI is ultimately more data it sees. The better it becomes or can infer more cognition, more reasoning. That's right. So the more data it sees, definitely you can build better models. Now, whether that extends to cognition or reasoning is still left to be seen. Like I'm off the camp that cognition and reasoning and AGI, for example, you're far, far away from that, right? I think right now the AI wrapper, stuff like things that we do, for instance, we have our own proprietary data set with CUBE transcripts. Other people will take their core data and feed it into open AI and have that round trip back in as an app. That's kind of like a state-of-the-art, easy app. Yeah, exactly. So you can... Like a website. Yeah, so you can send it to open AI, you can send it to another proprietary model, or you can take a model and then deploy it for yourself so that you don't necessarily have to work with your data leaving your premises. But what you do with the data is like three steps is what I see, right? So the first step is don't build any models. Use existing models, whatever might be your model provider. See how quickly you can augment yourself closer to a business outcome. Get to an application. The second one is find out applications that you cannot solve with somebody else's model. Do whatever required fine-tuning is for a particular task. After you've exhausted both of them, then you will have to go build a model because those are things that you'll know that if you invest, you'll get the ROI. And just one quick follow-up, Jay, is that that's where the potential business models could change where you subscribe to a service that says, okay, and buy or build. It's a buy or build model model. Exactly. Okay, so you said your AGI you think is a ways off, a ways being decades, and how about full self-driving? You know, that's probably closer than AGI maybe. Yeah, so full self-driving doesn't quite require AGI, right? So AGI... It doesn't require a learning system, but... That's right, but... Okay, so you would agree on the spectrum. Full self-driving before AGI, full self-driving maybe a decade away? My hope is it's sooner than that. I mean, having worked with it, but yeah, my hope is sooner than that. Okay, so AGI, what are you thinking? I mean, because it's scary to a lot of people, me included. So I'm now walking off the... This is just my personal opinion, right? Yeah, nobody knows. Nobody knows, right? But then let me give you a thought experiment, okay? Let's assume that AGI will happen in the next two or three years, or AGI has already happened. Let's make it an experiment, okay? Now, if and when it happens, if it is truly AGI, won't you think that it will know that the natural intelligence beings would get freaked out by the AGI? Yeah, indeed. If that is the case, it would act in ways that will make us believe that... Would act dumb. Dumb. So... It would hallucinate. If you follow the thought experiment, we either are already living in a world where AGI has been around, or it's so far out there... All I'm thinking about right now is the matrix, neuro-pumping my... Exactly. But that's why I posted this, that this is a personal opinion, but just follow the thought experiment and we'll be in the circular logic of if it will happen, it's probably already happened. If it ever happens, we'll be the last ones to know. Yeah, so... Well, we are living in the matrix, it feels like... Ouch! But I think people need to understand Andy Grove, we use quote Andy Grove a lot on theCUBE, let chaos reign and then reign in the chaos. So I think we're in a stage now where, I don't even think it's even first in it. I think it's pre-game, what we say in theCUBE. So it's still, but the computer science integration into all these conversations, all have a C, S, or comp side theme to them. We just talked on the other panel how media and musicians are all going to be tech-driven. So technology's now driving everything, augmentation of human talent, and even Google just said more creativity. We've been saying a creative class is going to emerge in the tech world, never seen before, but that's knowledge work. In an Arun's thought exercise, you can't reign in the chaos, it'll never happen, but anyway. Sorry for that. I also have a different take on that from it. So when email was introduced to us, like those of us who are old enough to see that. Wow! Everybody said it's going to increase productivity. Now, study after study told us it's worse, and none of us would actually agree to that statement today. But would any of us be able to live without email? I mean, you can change it to different forms of communication, but it's still electronic. The same way you and I are going to assume that GenAI is going to be there and everything we do. If it's not there, we're going to find out. Do you believe that AI is a new way that changes everything in applications and infrastructure? So from an application infrastructure world, definitely, because the insatiable need for computing has just become even more insatiable. And of course, from being in the computing business, it's great for us, but from a perspective of how quickly people expect things to change, I was fascinated by the previous panel where they said, oh, but I don't want a dump co-pilot. Just think of that sentence. Six months ago, you didn't have a co-pilot. No, you don't want a dump co-pilot. Back to your productivity gains. We had a guest earlier today that said that computing is like oxygen, it should be free. You don't pay for air. Well, the fence will be on Mars. Whoa, wait a minute. Okay, but what about, go ahead. So the way it's changing also is you're going from, people not necessarily understanding how much compute is required to do a lot of these things to saying, well, we need real-time agents. We need things that can be done not so real-time and we also need things that look at our data when everybody's sleeping. But across the spectrum, you need more compute and you need compute that's dedicated for doing this from the edge all the way to the cloud. So how are you thinking about the edge in particular? Because it seems like there's gonna be so much AI inferencing already happening at the edge that it's going to explode. How are you thinking about it? So we are actually ahead when it comes to thinking about this in the edge and on computers, for example, Pat announced the AI PC and in our conference a month ago, it's really around adding the right kind of software and hardware acceleration across the board. Even if you're looking at a low power, low cost device, it also has a specialized AI acceleration added onto it. And we're looking at this now from years of research and years of development into it. We are ahead of the curve. Pretty much every Intel PC you're gonna have out there is also going to be an AI PC. Okay, what about the embedded edge? Yes, so same thing with the embedded edge where we are looking at software deployments that make it easy for customers to take a large model and deploy it onto their edge without actually worrying about how do I reduce it? How do I quantize it and things of that nature? And also when I get to update that model, how do I make sure that the life cycle is actually properly done? Is that an x86 or something different? So it is x86 plus, but then it's x86 plus acceleration that's also added onto x86. For example, if you look at our latest high-end data center CPUs, the Saffron Rapids, it has an AI acceleration that's embedded into the CPU itself. Data centers are not going away, Dave, as we said in theCUBE, clouds growing like crazy, edges developing. It sounds like distributed computing to me. Here is another controversial statement for you. The cloud is the data center and data center is the cloud. The data center on premise is one big edge. Yes. That's what Scott McNally told us years ago and he said, I just sort of called it cloud. And network is computer. Thank you very much for coming in and taking the time to contribute to our SuperCloud 4 Gen AI episode. We really appreciate it. Let's follow up more. We want to keep track of your work and obviously the work at Intel, obviously Pat Gelsinger, CUBE alumni as well. Thanks for coming on. Of course, thanks a lot for having me. Thank you. We'll be back with more SuperCloud 4 coverage after this short break. Remember, tomorrow all day as well, a lot of content, a lot of payload being dropped here on theCUBE. We'll be back after this short break with some close, two closing sessions. We'll be right back.