 Hello, and welcome back to theCUBE's coverage of AWS's re-invent, the annual user commerce theCUBE's 11th year since 2013. I'm John Furrier, host Dave Vellantes here in the ANOS meeting, and we've got all our analysts here. We're breaking down, this is just where the action is. And we've got the executives, we've got the entrepreneurs all weighing in on theCUBE as they always do, extracting the signature noise that we got. Famous CUBE alumni here, Madhu Sudekar, who's the CEO and co-founder of Sarah, company he founded, one of the many companies, serial entrepreneur who's been in the forefront of generative AI, it was maybe called ML loss before. Madhu, great to see you, thanks for coming on. You always share great insight, thanks, great to see you. John, thanks for inviting me here and great to be with you after Thanksgiving. I've known you now, I think maybe eight years, maybe 10, I can't remember, almost a decade. Decade. We've followed your trials and tribulations and your endeavors. You've always had your finger on the pulse of all the action, social, how that's platforms evolving, get ahead of that, sell that, build the company, sell it, get another exit on your belt. You've got a new company now, and I remember a conversation a couple of years ago, we were talking about AIOps, MLOps, AIOps at the time, that was kind of the category. A lot's changed, I've seen generatives come on the scene, generative AI, role of data's still important. Chatbots been around for a while, but now they're a little bit different. I mean, chatGBT is technically a chatbot from an application perspective, that's what they call its category, but it's not chatbots, it's actually generating content. So again, people see that and they get excited, but right now it's a world of confidence. Excitement is great, confidence means it's working. So this, we're in this zone. What's your take on the generative AI market? We'll get into some of the Amazon stuff, we'll see what you're doing, but it's transitioning, but it's kind of the same game. What's different? Give me your perspective. Yeah, no, no, thank you, first of all. I started what I started in 2018, when I left service now to start this, so it's been five, six years we've been doing this. Back then we used to call it Converse in AI, AIOps. It was GPT 1.0 or P1.0, remember it was all the BERT models, NLUNLP. We went through GPT 1.0, 2.0, 3.0, now we are in 4.0. I think this evolution is happening a nice way. Back then we used to call language model, became large language models. As you said, the conference is really giving the, to both the consumers and businesses, I never saw two trends hitting at the same time. Like if you look at the internet is probably the only time when you had both the consumer and enterprise happen, even with the cloud it was only enterprise movement. So I think this time that Jenae is both impacting the consumers and enterprise, and with real applications and real use cases where people are seeing the value. So I'm seeing the really tailwind on our business, since I mean there's a lot positive I will give credit to still to open AI, they're like my heroes, right? They kind of put what you call oxygen, steroids in this business, so it's a great use case for Jenae right now. I think open AI and chat GPT has been that consumerization moment where they kind of did have their Netscape moment in a way, because Tim Altman debacle, Mark Andreessen lost Netscape when other people took it over, had he stayed and run it, it might still be around, but Netscape was the dominant browser, history Microsoft beat them. But that's interesting, I think I agree with you, and in fact I talked to Adam Sileski about this, I didn't put it in my stories, I didn't want to, I was covered long as it is, but we talked specifically about the comparison to the web and the internet, the internet, remember the internet was not the web, the internet was telecom, prior to the worldwide web, which was the standard HTML, HTTP, there was something called the information superhighway or the internet, and then you had online service providers like CompuServe and AOL, so these were proprietary online services that had content service providers. Okay, sounds a lot like open AI with all the- That's right. So open versus closed, I mean some people call it closed AI, because it's not necessarily that open. So this brings up the question, if you believe, and I do believe it is, where the AI is like the web trajectory or internet, web, proprietary versus open, open always, open was the key to success with the web, the internet, the web as we know it, was what it is, web apps, so does AI have to be open? And by the way, is there an AI company? There never was an internet company, I think some, I think one of the companies called this little internet company, but the internet wasn't a company. Right, the site. No, you bring up all good points, look I think first is let's go back to your point, I think open versus closed is going to be a debate, we'll continue, and you're starting that, so I think when the dust settles, there'll be a few open source models, it's not like you'll have 10 open source, and similarly there are a few closed source, like Microsoft Edge or Microsoft browser versus Netscape, right, I think you'll have both, but it won't be like 10 open source models, again the open source has evolved a lot with Elastic, with what happened with Hashicarp and MongoDB, so I think there'll be one or two open source will be there, but there'll be also value for closed source, and both will be, you need both of them to do each up level, right, each will have a value, but that is still at a foundation model John, what I see is the next value will be on LLMs, it's not the foundation model where the value will be, it'll be more on the domain-specific, I call them what do you call LLMs, which are domain-specific, which are small, could be medium, I think the word that I heard Microsoft call it SLM, which is very small, large, small language models, which are very domain-specific will be, I'll have one for IT, one for HR, one for finance, so one for each enterprise, I could have one for John for theCUBE, CUBE will have all the LLMs, you've created one, right? SLM, that's it, yeah. And that's proprietary, so now again the question is, leaking that IP, because if you consider that intellectual property, that's proprietary. That's right. So it's kind of bizarre how the words are flipping out, open AI is considered open, but they're proprietary large language model. That's right, so I think that's why there's, I call it new, I'm using the word called traps, maybe not the right way to say it, which is the trust, responsible AI, security and privacy, I think you need that to control your LLMs, because this LLM should be in, the CUBE LLM should be in the CUBE environment in your VPC, you don't want to put it outside, it's your IP, how do you protect that, and also how do you make sure that is socially responsible? I think there's a lot of new areas to make protective and security layer for enterprises. And I think the cloud is perfectly positioned with APIs to manage the interplay between data models. So for example, let's just say that we have a large language model, a small language model of the CUBE, and just we say, hey, it's so valuable, it's very small, but high quality, linguistic word combinations, we could actually be the training module for other people to use, you can hit us on an API and say, oh, I don't want to reinvent the wheel, the CUBE's already got, I'll hit that up. We might want to say, hey, I don't want to reinvent the wheel, I'll hit open AI or anthropic. So there's going to be alchemy with models, integrating of the models. So if that's the, you believe that to be true, right? Okay, so if that's the middleware, because what you're bringing up is if LLMs continue to be that layer, then you're going to have foundation and LLM models, we're multi-models in this abstraction middle layer, middleware, basically software, powering the applications on top and the hardware's the cloud. Right, so I call it like if in infrastructure world, you have infrastructure as a service, pass as a service, SAS, the LLMs will become the pass. The application will be where you create the embedded AIs, your chat boards, your universal board, your AI co-pilot, your GPTs will be the application that are created. So you have the middle layer, which is LLMs, you have the foundation layer, which will be your infrastructure for AI, and then you have the application that you can create. And application could be the one that can interact with you or a universal board as well. Great, great observation, by the way, that's consistent with what Adam's going to present and the keynote, the three layers I covered in my post on silkename.com, my exclusive interview, check it out if you're listening, you want to check that out. But this brings up the next question, if this happens and it is happening, so just say we're right, because I think we are. This changes the game on data management. It flips the script, because now, where's the data set? Do you sit below the LLMs or top of the LLMs or inside the, are the LLMs a bit data? And what about data pipeline? What about data interaction? What controls the data plane? And how does that render itself up to be an effective generator of content or application? Because remember, generative AI generates content. That's right. So it needs data. No, you bring up a good point. So both at the foundation model layer, you need the data with the knowledge graph. LLM also, you need the data. Now this will be domain specific data, so but you're right, the data management will change, where do you keep the data stored? How will the people with the largest amount of the original data will always win the game? And again, the data belongs to the consumers. Data belongs to the business. So even if I'm a business, the data belongs to you, if you're my customer. I can never leverage it. So this is where I see some of the vendors trying to learn one customer data to other customer data. You cannot do that, right? That's like a cardinal sin. So your data should belong to you. I should give an LLM to you, but I can't take your training data and apply it to somebody without your permission. I think that's the other thing that you will see play out really nicely. There might be a marketplace opportunity, and it's hard to tell, but I think that's going to be almost like page rank was for Google algorithm. You have some sort of referential integrity around data. How do you manage the data and having that trust relationship will come up? It's going to be very fascinating. And again, this is a stack change. It is. So the next question is, okay, Amazon's got faster chips. You're going to hear inference and training. Obviously, Graviton is the general purpose compute. As the chips get smarter and understand how to be optimized from a cost and energy perspective and be more performant, that's going to kind of create that kind of performance step up, which makes the LLMs fast. So what do software people do in this performance? They go to the next level. And then they bog it down. I think the chip's got to get faster, kind of like the PC days. Remember, it's always the next processor. The cloud guys have an opportunity to make it better, but it's not monolithic. It's distributed computing. It's edge. You bring up good point. If you ask me right now, the movement is on the software and algorithms. It's time for semiconductor chips to actually catch up. So if I'm a software company, since I have the data, I have to now take all these things and semiconductor companies can run my pipeline to make my GPUs and CPUs better. It's almost like profiling your software. So today when we do it, we still see GPUs are not catching up to my algorithms. They're taking a lot of time. So if I'm Intel, if I'm Qualcomm, if I'm NVIDIA, I still have to get my GPUs better for the algorithms of gen AI. Remember, most of those are designed for image processing, for other applications. Now with the latest gen AI and with AI copilot and chat GPT, I would see companies like NVIDIA really going to work with vendors like us or partners like us to really drive the next generation of GPUs. I would call it a GPU for GPT and the GPU for AI copilot and for LLM. It's a fascinating conversation you're having to do because it's not just about the chips. So I want to bring in an interview I did with Andy Bechtonstein. Okay, you know him. He's my investor. He's an investor of yours, yes. Yes, he's my mentor too, I love him. So he's also known as the Rembrandt of Motherboards. He was a designer, well known in the chip business, legend, CUBE alumni 2018. And I said to him at VMware that day and he kind of was actually raised as I was. That's a good point. I said, isn't the cloud kind of like a motherboard? You got a processor and you got other systems around it. Chips, DRAM, Rememory, SSD. So we're seeing cloud the same way. It's like a PC, it's not, but it's everywhere. So it's a distributed system that has interplay. It's a system management challenge. It is not only a system and what Andy used to say, at the end you have the same motherboard of the same size. You have same 19 inches by 11 inches. So your real estate is also limited. So what you do with that real estate in terms of CPUs and GPUs and purpose built, I think it's a fascinating game to see. My real estate does not change, but the server size doesn't change. I can only stack up so many of those boards. So question is how do I effectively manage them with what type of GPUs and which vendor is the next game going to play either for during the training time or execution time or inference time? I was talking this morning here on theCUBE with Prasad, who's VP of Infrastructure Service at AWS. He brought the conversation turned to power and energy. You can only have so many GPUs running if you don't have the power on the rack. So you can only stack, you limit it to the power. And also how stack them and the airflow should happen and the cooling, so that's why it's important that the next generation GPUs are really designed to your point, designed for your functions, LLMs. So I would imagine one day there'll be an NVIDIA GPU for the cube LLM. Yeah, custom chips with models. It's like pairing wine with dinner. I got a nice bottle of red with my red meat. Fish I have white wine. And that gives you the best experience right there. Yeah, it's very much happy, tastes good. So models will match chips. They'll be custom chips. So I think the custom silicon is huge. That brings to the point of architecture. So now let's get into the game of what happens next and why people are here at re-invent. Where's the hype from reality? And if it is the shift that's happening, it's generational. So it's going to be 10 to 20 year run. Everything changes, the stack changes. Scripts get flipped for like data management. Got any more addressable? Governance built in from day one, security embedded. Developers now using data as part of their development process. How do you architect this? Like a motherboard, you got limitations. It's a system architecture. That's going to be the conversation that I'm watching because the cloud guys have an opportunity to do what normal people don't want to do. Like what Intel did. Intel made processors. No one wanted to be in a processor business. So Intel did that. I'm an entrepreneur out like Amazon. If you do all that heavy lifting, I'll pay you for it. I don't need that to put a hard top on that and just build open source software on top of it. So it's going to be using the power dynamics, the role of the entrepreneurs and the builders and the DevOps and the platform engineers. Where's the line between what companies do and what the cloud guys do? Right. No, I think you remember, you and me talked about this 20, 30, 10 years back. Same thing will happen. They'll take another 10 years for AI. I think the cloud guys will become infrastructure only. Or they'll do like AI infrastructure, AI services, but they'll leave applications to partner like ISRA. Because they have to, and that's something that Amazon do really well. They said they're infrastructure as a service, pass as a service, they provide the services so that the partners and customers could build on top of them. If you give that flexibility, then Amazon will grow, right? So they have to have that so that way they can invite the ecosystem of the players. Same thing will happen here and the areas they'll grow, initially would be John, will be the Jenae application that we saw, offering the chat GPDs for IT, HR, enterprises, or I call the AGI services. The artificial general services, which is where the people will take it and say, how can I use now Jenae to do mathematical inference? How can I do to solve my calculus problems? Or I call it the equations such as differential equations, the Maxwell's equations, the diffusion equations, the more complex mathematical formulas you can do with AGI. I think that's where the, there'll be two groups will be created within the cloud companies. One for Jenae application, one for AGI. That's awesome insight. I have to ask you about entrepreneurship and the role of the founders now being a serial entrepreneur yourself. You're seeing the 2023 be kind of the year of AI love and summer of love in San Francisco. A lot of startups coming back to the Bay Area. There was more meetups in the Bay Area, San Francisco Bay Area, than anywhere else around AI. New York was number two but still way below what was going on in Silicon Valley. The pace of play is faster. What's your observation around the entrepreneurial culture right now because entrepreneurs are great at opportunity recognition. They see opportunity, they go after it. They see a white space, I can take some space down and then maybe threaten an incumbent and take more share. I think there's gonna be a lot of opportunities with AI. You're right, you're nail it. Look, I think since the chat GPD announced about open AI, there's more businesses came back from across the US, across the world. So San Francisco is back to the hotbed again with Silicon Valley to this. And within that, I think the reason why entrepreneurs and startups will do is, at the end of the day, you're focusing on a product and a solution. You're delivering a solution to a customer problem with an experience. So it's not only the data. The people who build the best product, if you put 100, 200 people like AISRA, we are doing, we are now 250, 300 people, we are focused on offering AISRA, GPD, co-pilot for enterprise. We are so focused on delivering that use case, that problem to that customer. That becomes a Tiger team or that becomes a Navy SEALs team. And that makes it really interesting how we can deliver a good solution, good customer. And that entrepreneurship is, I think, back to thriving again with AI. And that will drive next 10 years of AI evolution. Final question. At Sam Altman's developer day, that was the Monday before he got fired. Actually, the Monday before. It was pretty spectacular. They offered GPTs. That was one of the things got him into hot water under the big board cube. It was just the momentum's there. All he did was identify it and tested it. It was a home run. It felt like an Apple event. They did a couple of key things. It was a big success. One of the things that came out of that dev day was, on the Ocean Air Circles was, people said, my company just got killed today. So you're starting to see the moves by the big guys that potentially threaten some of the early players. Is that a feature or a bug? Because I don't remember the web scene where Microsoft or Netscape killed companies. I guess they did if they were in the browser business. Is there fear? Where should startups, in your opinion, focus on? No, very good. At the end of the day, Amazon actually killed many companies if you remember back in 2008 when they came, including the storage company, the compute companies. Companies just focused on just even the ETL companies. So I think the same thing Sam Altman and Open AI is doing is, unless you have a big barrier and a moat around your company, you will get eliminated. So if you're an entrepreneur, I'm a startup company, I want to make sure I have enough IP, enough moat around my thing so that I can build on top of chat GPT, on top of an Open AI, and next to them. You want to ride that wave. What you don't want is you also want to remember when Amazon used to come with Lambda services, so many different services, people who are focused on analytics, they all went away. Unless you are a large enough company like Tableau, you didn't survive on analytics game. Same movement will happen with AI is like, people who build with a full product, full solution to a customer and use case will thrive with a good moat. Other guys will be eliminated very fast. If they're a feature, they're not a company. Madhu, thank you so much for coming. We're getting the hook here. We got to call it a day here at Unlocation. Appreciate your time coming on theCUBE. As always, insightful masterclass on what's going on, on the latest trend of generality. It is hype, but the reality is matching with real momentum, real new opportunities. Again, enthusiasm is ultra high, confidence is coming on board. People starting to get more confident with their experiments and moving into production. Of course, theCUBE is all the action. Thank you for watching our coverage from our SuperCloud 5 special program streaming from Palo Alto, California. I'm John Furrier here on location with Dave Vellante and all of our analysts getting all the action here at Reinvent. Thanks for watching.