 Well, welcome back to SuperCloud 4, Genive AI. This is the focus. I'm John Furhose of theCUBE. We're here for our studio event that we got two great guests to unpack Genive AI, the role of compute and how AI is working together. Okay, big from Joshi founder and president CTO. Thanks for coming on theCUBE. Appreciate it. Joel Inman, CEO of compute.ai. First of all, love the domain name. So got to love that right out of the gate. Good to see you guys. Thank you. Thank you for having us. Joel, we've been on before on theCUBE. Three, I believe you were on. I think it was a three, was it three? Yep. Three, about cloud scale. Just a few months go. This is the focus as a Genive AI. The hype is off the charts and you're seeing kind of two schools of thought. You got the old school systems and entrepreneurs and thinkers who are all over this. Do they see a whole nother, complete paradigm shift around how computing, distributed computing is going to be implemented with kind of the AI angle. And then the next generation, kind of the young guns coming up, kind of like a generational shift. Clearly a major inflection point, some of saying it's as big as the PC revolution, the web revolution and then mobile. I think it's all three combined in my opinion. We see this as a big wave. It's a generational thing, but it's also has the infrastructure side super important. Everyone's talking about the apps, but under the covers, whether it's platform engineer or cloud, the Genive AI is going to be a big part of their narrative going forward. You guys are in the middle of it. We're going to get into the compute relationship to all this, but how do you see that app and infrastructure piece? Well, first of all, I'll talk to the hype cycle, question that you've posed. I think we're at the peak of the hype cycle, maybe even a little bit beyond the hype cycle and we're going to go into the trough of disillusionment, but for those who have a vision that is focused and understand the impact, it's not there for no reason. We're going to have a wave for the next 10 years of implementing, adopting enterprise AI in ways that improve our productivity. And I think the McKinsey study proves that we're going to have $4 trillion a year in extra economic productivity due to AI as we implement it, as we figure out what it even means. Vikram, as the founder, real quick, what was the motivation, what was the vision around this? Because your background, first explain your background, they'll get to the founding. Yeah, so I'm a system software engineer. I write code and an entrepreneur. So this is my fourth startup and my past lives prior to being an entrepreneur, I guess a good part of my life has been doing companies and going after chasing big ideas and dreams. I started my career at Sun Microsystems, was fortunate to have worked with some other founders. Back at Sun brought up some early sun machines to life, later Silicon Graphics, so played with 3D graphics before GPUs were born. So I changed domains rapidly and then later Oracle, which is I think more pertinent and relevant to the conversation which is going to have, relational compute databases and had something to do with exadata. So that's pretty much been my background. So you, we've seen the movie, you've seen the waves, you saw how the telecom and connectivity, compute power, many computers to then that whole interconnect, the networking, distributed computing, revolution. I mean, many ways of innovation. Joel, you guys, I mean, you've talked about the thoughts around liberation of compute from data warehouses. And right now, I'm going to be covering big data going back to the early days of the cube 13 years ago, the Hadoop days and the vision was great, but it just never happened. But yet data warehouses now move to the cloud. And now everyone says, move the compute to the data and you get the edge. But with AI, we see a whole nother power dynamic. It's almost as if AI is a gift that dropped into the market at this time that's going to change the landscape and liberate the market and specifically the role of compute from data and data management systems because all we hear about today is, I want to do more AI, but I need more GPUs and only got the compute costs are off the charts. What's your thoughts on this liberation of separating compute from the database, data warehouse, and the database management systems of old? Great question. So first of all, you know, when we think about AI and the adoption in the enterprise, it is going to drive a thousand times more demand for complex compute. And that complex compute is going to be in the form of machine generated SQL. I mean, that's the lingua franca of enterprise AI. That's what's proliferating everywhere. You can see it in business intelligence applications today, generating more and more complex SQL for compute. So the, so what that means is we need to get our data story together. You know, we need to come together and figure out how do we shore up our infrastructure? How do we drive a lot more efficiency out of our compute? And the first thing that needs to happen is we need to liberate it from our data warehouses. So just as storage was liberated from compute, we need compute to be liberated from our relational database applications. And once we do that, we're in a place where we can spread compute everywhere. And we're also in a position to go open. You know, the hyperscaler market boom because of the cloud architecture. Vikram, this is interesting with data. It's almost as data has this new inflection point where generations of thinking might have to be thrown out the window because AI works great with more data, but data is also where everyone wants to steal. That's the security challenge. And you've got the distributed computing challenge with latency. And even database theory doesn't make sense if you start to think that way. But so what does it mean for people to think about this? This isn't a huge concept, except creating the compute data management. What does it mean? Yeah, yeah, I think I'm so glad that you put the spotlight on that. And just so that we get the lingo here and the terminology right, it's not about separation of storage from compute. That problem was solved, whether it was map reduce or the oracles of the world, whether the sand and the nests, independently scale storage from compute does not matter. This is about pulling compute out of databases, out of data warehouses, liberating it, making it available like oxygen, like the ether that's ever-present, omnipresent, whatever omnipotent out there, because data is everywhere. And if you look at the clouds, let's take AWS EC2, for example, it's the entire compute layer. And then they have the storage layer. Every cloud has its own equivalent parallel. The ability to super recruit large numbers of cores and compute without having to think in terms of a database silo. I need to put my data into a table, into a database and a data warehouse to be able to type SQL. I think that's passé that day has gone. So we are upon a new future that looks very different. Even if you look at what's going on with the BI applications today, Tableau, Power BI, Looker, they generate at least 10x more SQL than all human-generated SQL. What's the future? The future is more autonomous sources of SQL generation, more AI ML-driven dashboards, no one's going to sit out there and use their editing tools and say, let me type, do a little bit of ELT, you're going to throw 50,000 joints. What's that, Dave Vellante, who talked about the computer's not here for doing those 50,000 joints. Well, let's talk about 100,000 or a million and make that cost effective. So semantic layer compute is going to go out not to humans, but it's going to be AI ML-driven and also the low-code and the no-code applications. So when we talk about AI ML, often the first thought that comes to our mind is, let me start to use those GPUs and run some generative AI and LLM models. This is a bit different than that. This is the impact of AI at an application level that is now going to go out and touch into, it's going to tap into these data stores, these semantic knowledge and this universal index of information to feed these applications all done in an autonomous and AI ML fashion. And the compute for that is going to be, what, 1,000 times, if anyone gets 10,000 times more than the SQL that we have today and leave aside that today's machinery is not even efficient for doing what we can do. People are talking about data processing costs. So that's sort of- It's like the caveman invented the wheel and now we got to move into the modern era. So what you're saying is interesting, is that that's going to require, you mentioned the lingua franca SQL, large language models, these were language. So SQL is the language, it's machine to machine talking. Exactly. If you will. So this feels like a neural net meets large language models. Are we looking at a different system? I mean, almost like an operating system for AI. It's like, if you take that forward, you have all this compute, what happens next? Because you can't run it on the old infrastructure or maybe you have to modify, abstract away the infrastructure. How should people think about this unlimited compute or dynamic compute or elastic compute? I mean, what do you call it? I mean, because if you have compute everywhere, it's oxygen. Yeah. Well, we like to call it abundant compute. So the concept is, you know, you should be able to breathe it in. Your application should just have it available wherever it is needed. When we break down data silos, we also have to think about breaking down compute silos. It does us no good. If we move the data everywhere, we have a data center enterprise, but our compute is still stuck in silos here, there, and the other dictated by the applications. We need to spread the love all over the data center. And, you know, the way we do that is it's a technical questions and you're right to point it out. It requires reinventing the vertical stack from the very top to the very firmware that we're operating our hardware with. It sounds complicated. Simplify it for me. What is the bottom line? How would you describe it? Because it sounds too good to be true. It sounds like, because you're setting the table for data being addressable and secure. Well, I think this is, this is not something new. I'm just going to roll back a little bit here. The separation, the concept of separating, pulling compute out of data management or databases has been present for quite some time. I mean, let's go back to then these ideas. What did they do? They took compute and they pushed it towards data, which was going to be better for certain workloads rather than move terabytes of data towards compute. You take compute and move it towards data. Later, Exadata followed, you know, as inspired by Netesa and others. So, playing with storage and compute separation, independently scaling them, the ability to take compute and move that around, these concepts have been present. A credit where it's due, the work which has been done by the Spark and the Presto community especially, has actually done a lot of, has made a huge dent in the separation of the data management and the data layer. For example, if you look at the lake house, right? What is the lake house? It's just a bunch of Parquet files and your compute is just your type in SQL. You don't think you're not confined to a data warehouse. You're not confined to one of those silos. So, I believe that the precedent is here. The problems that need to be solved have to do with compute efficiency and making it cost-effective. And the final frontier, I think, for data warehouses and databases is concurrency. Data warehouses and concurrency don't go together. So, when you start to look at the new kinds of workloads and applications that are going to be coming out and hitting these databases. We're talking about this machine-generated SQL out here. It's going to be so much in quantity, right? We talked about many orders of magnitude more and complexity too. A machine generates pretty damn complex SQL, right? And as the complexity, so complexity along with concurrency exacerbates the whole problem. You want to make the compute for that efficient. And to set the stage for what's happening, let's take a, you know, like what's the state of the union today? If you look at the compute efficiency of data warehouses, databases in general, today, especially for elastic compute, we are using three out of 10 codes, right? That's a 30% CPU utilization. And maybe even that's a generous number, especially if you start to look at elastic clusters. So, we are leaving 70 cents on the dollar here behind on the table. And obviously, if you hear about the cloud data warehouse compute cost, I talk to customers all the time and say, help me here, you know, we love what we have. We like having 2000 connectors. The cloud data problem has been solved, but my compute costs are very high. And that's, by the way, that's a validation for what we're hearing in the marketplace as well. Cost is the bottleneck. Joel, that's a great point about the late house. I want to get your thoughts on this because it's still a relatively new concept. It's clear data warehouses, it's not as last leg. It's old, old way, not the new way, but it's still installed everywhere. What does a customer need to be aware of when considering migrating, say a lake house, where you can start tapping into setting an architecture up for this kind of new compute separation? Yeah, well, I think that the warehouse market has been given a life extension by some of the leading vendors. And that life extension is from, they made it very easy to use. So there's a single JDBC endpoint. It's in the cloud. It's easy to access. You don't need to hire a team of DevOps engineers. And it's absolutely reliable, 100%. Those are really compelling things for people in the enterprise who don't want to dip their toe in the lake, so to speak, or the lake house, right? And when we're talking to customers, many of them are saying, I want to build a lake house. I know I have to. The data is ephemeral. It's open format. I need to go there to solidify my infrastructure, future-proof my infrastructure, but do I need to hire a team of DevOps engineers? Is it going to fail on me if I run out of memory? So these are problems that still haven't been solved. And the giants in the industry are solving these problems right now. It's a really exciting thing to see. Yeah, if I were to jump in and add to that, that's right on. The spirit is there. People want to go out open. They want to spell their tables in the data warehouses onto open parquet and iceberg or Delta or whatever format. And then the general sort of thinking that is, I do want to go to an open standard, an open dialect of SQL such as Spark and Presto, right? And I don't want to have to deal with throwing DevOps engineers and other stuff, right? It's like buying my Tesla along with the whole shop of technicians, right? I don't want to have to do that. And I think credit goes to the cloud data warehouse companies out there, the pioneers who made it super easy. That's some single JDBC or SQL endpoint. I just look at that. I don't have a deal with that. And stuff just happens. When's that going to happen? And the second other issue you touched upon at the risk of repetition here, John and Joel here, which is I have to provision for my worst case memory. Every once in a while, the compressions and the rare factions of my workload require massive amounts of memory. Provisioning for the worst case means because, you know, a lot for in-memory system. Nothing wrong with that. Stuff is supposed to run, but there's more data. There's more complexity, there's more concurrency. And you're not going to be able to throw large amounts of memory at it. And the trade-offs are harsh. You start to place this memory to disk, right? Spell to disk and the performance is going to be slow. So provisioning for the worst case means lack of efficiency. So these are the kinds of issues that are the customers that we talked to. I want to just complete the thought there for you. Or maybe add another point. Awesome. You know, the data lake market is growing at twice the speed of the warehouse market. It's growing at 20% Kagger through 2028 is one estimate that I saw versus 10% Kagger. So lakes are already, lake houses, I should say, are already growing at twice that pace. Can you imagine the rate of adoption that we'd see if it was as easy to use as a cloud data warehouse? Or, you know, as simple as connecting your single endpoint and it was as reliable. It was enterprise grade ready to go off the shelf. It would be massive. No out of memory failures, right? No omkills. Well, that's a good point. Well, first of all, the word memory is interesting now because it comes up in two use cases. Memory as in physical memory in memory for storing stuff. And then memory from a recall perspective, retrieval is a big hot topic in generative AI. So I want to kind of connect that to the super cloud theme, which is across environments, whether it's two public clouds or on-premises edge, because you don't talk about moving compute. The edge was really where the first conversation started around the conversation of moving compute around. So we all know moving data is expensive. So when you look at data and its addressability, how do you look at the multi-environment piece? Where is it a semantic layer? Is it just one big data pool where you have the intelligence built in with AI now, with compute kind of programmed into it? I mean, I'm just kind of riffing on this, but like how do you see this? Because I can see the benefits of data links over data where I'll check. But also the data warehouses in a public cloud are also constrained in their cloud. So the easy button here would be one big pool and managing multiple environments. What's your, am I over the top here? Or am I fantasizing too much here? No, I'm going to let you all address this. I didn't want to talk to you. What decade am I in? 1855? That might be beyond my pay grade as well. Just be that one big company. I'll offer a thought, right? So the big Fortune 500 companies that we've been in conversations with want to go hybrid. They want to go on-prem and cloud and multi-cloud. They want to have their cake and eat it too. When you start talking about federal contractors and military applications, they want edge devices. They don't want to wait for their data to be centralized and some repository. There's security issues around that. And so fast forward here, as we see the evolution of the super cloud, it's going to exist at the edge. It's going to exist in the fog. It's going to exist in the data center and the cloud itself, right? And our small piece of this party at Compute AI is driving compute efficiency in all of those places. It's really the interoperability between our pieces of hardware and making sure we get the most out of that CPU, the most out of our memory and the most out of our applications all the way up the vertical stack. Yeah, absolutely, yeah, yeah. Absolutely push the limits of CPU memory utilization, dissipate fewer watts. What would the world look like if we could dissipate five X fewer watts? You talked about the edge. We do talk to the government and DOD and stuff like that. And on the edge, battery power is a huge issue, right? So you want to do federated compute, right? I mean, that's the model that even Google. In real time, tactical edge, for example, with military is all about real time, having the data, flow latency, battery. Someone's walking there with a battery pocket back or there's a robot doing that. So compute efficiency is going to be very critical. And of course, GPUs are important. You're going to have to make those decisions out in the field without having to be able to tap it or even have a connection back to the central. I mean, basically this comes down to the use case. You're talking about the Tesla. I mean, you have to be optimized for the use case or the application. So here we come into the vertical versus horizontal. You want the scalability, but if you're on the edge, say military, you need to also talk back to the central data, but also maybe replicate. It's all kinds of use case scenarios for that application. And that may not be used by anybody else. So you need to have the compute. So is this an advantage where it's separating the compute makes that better? Is that an example of my thinking the right way there? Absolutely. Though I'd say that while John, you point out there are so many problems there to be solved, the one that we think is quite fundamental and maybe has the potential of being the type that lifts all boards is the one of compute efficiency. And when you look at compute efficiency, it's really compute and memory. They're two sides of the same coin. When the core to memory ratios haven't gotten any better, they're actually getting worse. As we know, processor speeds and memory speeds have stagnated for the last 15 years or so, more than well over a decade, right? So things haven't got many faster, much faster. We are doping more transistors into processors, which means what do you do with these things? So either you're going to put more cores in there or you're going to put more caches. And now with in-memory systems with large amounts of transient ephemeral data, which is in passing, I need to hit that data really hard while it's under my course and while it's in the memory, all that is causing massive CPU stalls. So when Joel and I look at these systems and we see what is the root cause of the lack of CPU utilization? Why is it that we're seeing only 30% CPU utilization? And why is it that when you look at elastic clusters and distributed systems, the CPU utilization is even more. And when you put the economics of that back into the picture, for example, cloud costs, right? I mean, we know we're talking about the edge here too. The same thing almost applies to the edge. So if you look at cloud costs, they're directly a function of the amount of memory, which is the most expensive component of the whole thing. And memory and CPU are literally two sites of the same coin. They mean money. If your CPUs are stalled, you're not doing work. And if your memory is not sufficient, you know, paging to disk, you're going to get IO weights and that means less utilization. So you're paging just to complete the task. Do interrupt me though. So I just want to get the whole thought out here. So whether you're getting the data from a lower tier of storage, which is a spelted disk, right? In the past, we had to take a coffee break when you started to page. Remember the Linux operating system? Or you're going to get that data from over the network. It's going to cost IO weights. These are some of the issues having to do with efficiency in compute. And aside from liberation. So this is where the action is because Dave and I were interviewing motherhead guys that tell Jeff Clark and the marketing people like, don't we talk about solutions? Not about speeds and feeds. We're in a speeds and feed market right now. All the conversations, how fast can the silicon go? I need more chips. I need more power. We're in a renaissance of systems architecture on a global distributed basis scale. And now with AI as the gift that's going to give the hyperscalers more power. Really in a perfect storm opportunity, it means pre-game is not even first inning. This is where the opportunity is had a heading. Where's compute AI fit in? What's the vision? Where are you taking the company? Cause right now, you know, all the future scenarios put aside, people just want to generate AI working. They want to be positioned to leverage the current situation with headroom. So they don't foreclose the opportunities ahead of them. They don't want to misfire. So I won't say baby steps. I'll call it maybe kindergarten, maybe play with some blocks here, you know, get on the rug, you know, play with it, generate AI. So people are experimenting. They just got to get going. Where are you taking compute AI? Well, compute AI is a fundamental building block for next generation architectures. And it really is providing the scale that is needed to address that extra workload that AI is going to bring, the thousand X workload that AI is going to bring. If we don't fix our data infrastructure now, and by fix, I mean making it compute efficient, making it infinitely concurrent and scalable, making it, you know, appropriate to feed a machine that is just hungry and just eating and eating and eating all this processing power, if we don't fix that down to the very lowest level with CPU utilization, you know, then the costs are going to be out of control. And I'll give you, let's bring it back down to earth for a second, right? Our favorite, you know, leading, this is just one example, a cloud data warehouse company, right? If you add a ninth user, your cloud costs doubles, your cluster size doubles on the backend in the cloud, right? Every ninth user, so the concurrency of nine, right? Or concurrency of eight. We're talking about the need for a concurrency of thousands or hundreds of thousands of joins, as Rick mentioned before. The concurrency is huge, and this is again, that's why it said, this is so disruptive to the database world, because the theory of databases was constrained to state of the art at that time. Yeah, yeah, exactly. And also if you see, which is the reason why we've had oil TP-style workloads, which is an ATM-style bank transactions, million per second or whatever, and then you had warehousing, which was more data mining, which was DSS-style workloads. And traditionally, that has been, let me just run through billions of rows in a column, shred through them rapidly, that's what data mining was. Let me do an aggregate and give you the average of median and median or whatever, right? From there, what has happened, and I think it's important that we talk about this, especially when we talk about efficiency of compute, and what does that really mean? When you look at the modern-day workloads and the modern-day complex compute, it's not exercising our columnar stores, even though the data is in parquet or on disk. It comes into, if you're just talking open source language, it comes into memory as arrow format, that's still columnar, but the days of columnar databases, especially if you subject them to modern-day workloads are gone because it's no longer columnar work. So let me give you an example. It's post-modern, it's old-modern. We got to re-modernize it, basically. Right, right, there's a better, better. Super cloud them. There you go. So what the pattern that we are seeing for some of the more modern compute, especially when it's generated through auto-generated, through one of these autonomous AI-generated SQL sources, is row column, row column. So you're going through large numbers of columns, doing aggregates, and now watch this. You take these aggregates and use them as join keys. For other columns, same or different tables. That's super complex. You cannot now have the benefit of that linear compression that you get in memory. You have to go row column, row column, and that's where you have to over-o provision memory for the worst case. And that's where omkills happen. That's where you run a lot of memory failures. So this row column paradigm puts a huge burden on the memory infrastructure and as a result on the compute, and now the same problem. So AI as a gift on one hand is also a challenge, and this is where inflection points really kind of show. There's always going to be kind of a new way to kind of have some friction that you fix. This is kind of a worry. Massive levels of complexity is going to come in. I mean, if you look at the SQL generated by Tableau, unless it's stupidly simple, select statements because my table was denormalized, you cannot de-normalize summary tables. Maybe I'm taking you guys into the field, but generating simple, we don't mind. like statements is not always possible because data changes, new tables come in, more joinery is needed, more group-wise are needed. SQL stands for structured query language. Let's say language, LLMs, large language models. It could be the lingua franca. Final question and conclusion. First of all, great masterclass. I love the deep dyes. Really good to get into. Cause it shows where the action is. Yeah, kind of. That's where it's happening. You got to go into the deep, into the stacks. Okay, that's the problem. It's a few hard problems. And it will be fixed. This is going to, has to, and it will be because the opportunities are so great. Final question for you guys. What is the future vision of compute? Cause I like this idea of separate and that's constant. We'll continue to talk about, cause it makes sense. But what's the future vision of compute? So, I'm sure you, you know, as a skipper, you know, you have something to say, but I didn't want to do another database company here. Don't want to be the 10th search engine and don't have something significant to offer. Think Google, right? So, the whole challenge that's ahead of us, which is lack of infrastructure efficiency. We are throwing massive amounts of infrastructure, CPU memory at the problem. Costs are higher as a result of that. There's an opportunity there. Given the problem of AIML autonomous sources generation of SQL, which is huge amounts of SQL coming with high complexity and high concurrency. So solving those problems is super exciting from a technical perspective. And we can see there is ample business opportunity here in value. There are today's problems that need to be solved. Help me, my data warehouse costs are high. Well, how about can I do to offload my compute? I'd love to go to the lake. So there is an opportunity here for us to address some of these customer pain points today and future proof these customers needs for the future that I think has gone up. Joel, it's a new paradigm. What's your thoughts and we'll close it out. So, our mission is to make compute abundant and infinitely scale. Well, I kind of said that a couple of times, like oxygen, like breathing, right? Yeah, it's a dream scenario. And, well, the future is closer than you think. You know, what we're seeing in real world workloads and situations is a 50X price performance improvement in early use cases. And we're building on that. And I kind of alluded to this earlier, but if you can stick something into the environment that is very open, that has a single endpoint, very easy to use and no DevOps requirements, that's where we're entering the market. So we're taking these data lake infrastructures and we're building upon, we're standing on the shoulders of giants and we're saying, maybe we can add a little bit here. We can make it easy to use, open, infinitely scalable and reinvent the wheel without even reinventing the wheel so that people can simply consume it. Well, if you can enable companies to build better, faster, generative AI apps, which is data-driven, data's enable, it's native, sometimes a bolt-on, sometimes it's an abstraction, however you look at it, it will be critical. Yeah, I mean, early use cases are data transformations and pain points where the costs are high, but I can't tell you how many conversations we're having about AI and how do I operationalize this and how do I get it out of my, after I've trained it, how do I deploy it? Vikram, great to see you on this new venture and big idea, Joel, great, great company, I love the URL, compute.ai. Let's get the backstory on how you got that amazing view. So we don't have the cube AI yet, so someone else got it, beat us to the punch, but thanks for coming on. John, great meeting today, and thank you for having us over. Thank you. All right, bringing out all the actions of SuperCloud 4, generative AI, the infrastructure, what it takes to make it happen to enable developers and applications to be AI-native with generative AI. That's what this focus is all about on SuperCloud 4. Thanks for watching.