 Hello and welcome to the super cloud for the fourth edition our quarterly segment. We do this every quarter We're gonna unpack the hottest trends in enterprise tech this fourth episode is about generative AI It's the hottest thing for the past year We've been talking about it going way back to November and every quarter. We're gonna unpack it. We have a great lineup We've got AWS. We've got Google. We got Salesforce Dell AI ethics with sass sass CTO AI 21 labs all the hottest startups enterprises and and guests here two days of packed coverage leaders in the enterprise And we're excited to kick it off. We're gonna do an intro. We're gonna do a little analysis I'm gonna get the program going. I'm here with Dave Vellante my co-host and Rob Streche with the Q research guys This is our fourth installment every quarter Dave. We started this out a year ago and I can't believe in a year We got three in this year four for the 12 months It's fun We bring a lot of content to the table and it's just been amazing to bring the people together in the community And the the caliber of talent that are weighing in and contributing to the program has been phenomenal So just want to say thanks to you guys for all the work you guys do and research Rob and welcome to the team Yeah, hey first of all place looks great I got I got the tie the tie memo to match the the background and yeah, this is the fourth super cloud You remember like last year we set up the concept of super cloud we brought in the community We brought in technical experts. Is this technically viable? We definitely confirmed it was we tightened up the definition And then since then we've been knocking down security. Obviously AI is front and center. We had great sessions on data There's just so much. I'm still unpacking from the first cloud Have a AWS keynoting. We got brought in south who's going to come on VP. We got AI 21 labs on as well We've also talked with hugging face in the past. We got founders and startups. They're focused on AI We've got analysts segments. We get CEO side with the enterprise leaders Microsoft Google Salesforce Palo Alto Networks Dell's new AI leader and just tons of content snowflake amplitude Walmart PwC Google public sector health care experts that every single industry is impacted. Yeah I think that's the that's the underlying thread is that this is changing how? organizations are really looking at their data and turning into Conversations with their customers using that data and having their customers actually engage with them through the AI It's really exciting stuff guys I want to start the kickoff and just get into the program as I mentioned some of the names We have we're gonna hear from the leaders the big themes that are jumping out from the from the pre interviews and what we're gonna hear And the program is clearly this three ideas going on the chatbot. We've seen that done that That's moved on to co-pilot like human augmentation and then predictive all that's got data This is kind of foundational kind of like low-hanging fruit And then the big question is role of open source and the power law Dave that we've published around what the language models are What the foundation model is big better than small? What's in production? What's not in production Rob kind of what's the reality? We're gonna hear a lot of that this week and this show What's real what's hype and where the hype matches at Dave? What's what do you guys see out there in terms of as we look at this program? What's jumping out as the core theme? I mean, obviously, we're very excited As is everybody about AI and generative AI But I think it's important to your point is you got to separate the hype from from reality And when you look at reality you look at it spending it spending is not going up dramatically. It's funny I saw a note from Gartner saying it spending in 2024 is going to grow by eight and a half percent There is currently no evidence of that in any of the spending data, you know right now It's sort of very pretty tepid and so I think you know given the geopolitical uncertainties And when you talk to customers and you do the surveys It's it's much much lower than and now it could change and AI could be that big driver But right now what's happening John and Rob is I think you see this in the data that AI initiatives are stealing from other budgets The AI is by far now the number one sector that that with in terms of spending momentum And it's stealing from other areas and most of the AI is still experimental now The good news is and we've talked about this John is it the AI Experimentation is spending is actually matching the hype, but still it hasn't translated yet into a productivity boost which Eric Brynjalsson MIT professor and economist said he would be disappointed if productivity didn't globally didn't increase from its current 1.2 percent to as much as 4% or even more if that happens look out That's that's huge news Rob. What's your take on the role of data? You've been tracking this part of the cube research team What's what's the data aspect data angle here? Yeah, I think it's everybody is trying to figure out where is all their data There so there's a big infrastructure push to Do I consolidate it all into one data platform? Do I have multiple data platforms? How do I have a mesh over them? Where do I do? Transformation there's a lot still going on at that data layer as people talk about building data products And I think what we're seeing is that really people are focused on those You know segmented language models or smaller language models and being focused like I'm using it for HR I'm using it for finance and bringing that data together because they own that and that's their IP But there's big security concerns and how do I have a moat between that and make sure that it's not going back into the Models so there's a lot still going on with that data infrastructure And I know we have some great guests on that we're going to talk pretty deep about that guys I want to get your thoughts. We obviously we have AWS keynoting We also got an enterprise panel headed up by one of our community members how we shoe whose heads up AI and at machine learning at Palo Alto Networks, he's going to run a panel with Google Salesforce and Microsoft Okay, that's the end of I call the enterprise leader that perspective Then we have a startup panel that I'm going to moderate with a bunch of startup CEOs We're trying to get to that B round Rob a B round financing In this market and then so we have that kind of developer and we have a founder panel Okay going on so it's going to be multiple perspectives there, but we also had an in conversation with snowflake Box CTO Reggie Townsend AI ethics person at SAS and the CTO Brian Harris at SAS So we got conservative approach SAS is kind of a conservative company. They got a lot of install base So they're not rushing and saying buy the new shiny new toy with AI But clearly you're leaning in they've had data and machine learning but not generate AI now They're bringing that to the table so that the spectrum of consumption and usage and Experimentation is across the board. There's no one Playbook, it's not like oh you got to be a pioneer or an innovator every single company from conservative to cutting-edge is Leaning into AI. What does that tell us? Yeah? I mean, I think to Dave's point I think this spending kind of tells us that people are experimenting still and I think again We're you know, you had the first ones who got out there with chat GPT like Bringing that in and using that to get some efficiencies and things of that nature But the long tail from a power law perspective is really where we're starting to see People start to make those plans and I think that's where the real investment happens. I mean again It's I think a big tell you can can we cue the power law? Yeah, actually talk about that a little bit So this is something that Rob and John and I Developed and the vertical axis so we took that concept of a power law We kind of took some liberties with it and applied at the gen AI the vertical axis is the size of the model the horizontal is the model Specificity that's where all the industry action is happening and that orange line is basically an example of The music industry where there were like four companies four labels that dominated the music industry of the past And that's why you see that hard right angle what we're saying here and John Furrier You you've talked a lot about this those red arrows. That's open source and third party pulling the torso up So we think it's going to be a smoother line and then so you got the big guys the big cloud guys and Nvidia and open AI With those large language models the size of model is huge But then as you go down to the right a huge long tail both on premises would specialize the AI and then at the edge The telco edge we have we have a guest who's going to be talking about that a system on a chip And that's really where all the action is the last thing I want to say John And I'd love to get your thoughts on this as you help develop this model In fact, you you were the seed the catalyst of this this is not spending this is most of the spending I personally think is going to be on that long tail. There's me a lot of money made You know by the big cloud giants, but they'll be a lot of action on that long tail. Yeah Well, also you mentioned the music it put that slide back up again and show that again The the tail there was no there's no neck and no torso no stomach basically straight line down a straight tail This not always a music industry, but if you remember when the web started search looked like this top queries were you know Well known and then what happened is as the population of websites came out you saw that expand in that torso You saw kind of affiliate marketing kick and then you still had the long tail of keywords So search was an interesting power law as well now It's ironic Rob that and Dave is that search is one of the key Problems that they I saw co-pilot retrieval was getting into some of that data It's a data graph and if you look at the models again, I'll put that back up again The the the dynamic also includes open source. So the third-party open source also plays in the tail as well So what's happening now and we're going to hear this and this is going to be the premise We're going to unpack over the course of the next six months after this super cloud is as these models develop You're seeing a flipping of the script some of them are proprietary data sets Rob I use the word proprietary will live in the long tail, right? They'll be smaller high quality data And so we're seeing a trend where today they call the the chat open AI and proprietary models They're actually more open because they crawl the web the proprietary models will be the proprietary data sets Which is the initial property that's going to be the value of the company And then what's going to happen is people going to leverage the training of the bigger models to reduce their cost and focus On inference so a lot of model integration is going to come down and this kind of comes down to the cloud native World and you're going to see a lot of changes in this in this long tail the tail will get most maintain the torso and belly will grow and The model will define who you are as a company because that's where your IP will be and that's will be the engine of the application So that's going to be one of our things we're going to look at and I think that's Notable cutting-edge Dynamic and we're seeing it play out with open source There are two other really important points there the the big blockers to implementing Gen AI in production our privacy data privacy security Compliance and governance and so that's where these specialized models are going to have to shine and the second point I want to make and we're going to hear this from CMA AI that telco edge That's going to be all about power per watt, you know low-cost high performance Very very efficient and those are going to be arm-based processors and the I've thought for a while now that the economics at the edge Right now, you know, they're maybe not ready for mainstream enterprise even though we see it with Amazon and Graviton and others But that economic disruption that's going to come from the edge could be massive because it could create a whole new price Performance game in the enterprise. Yeah, I think that's really the key where the work actually gets done It's going to be closer to the consumer not all in a centralized cloud database You're going to have it where you train your model centrally you distribute out for him You know inference and it gets done the work gets done out there, especially when you talk about IOT and machine Actual machine data being you don't want to pull it all the way back into your cloud To be able to do all that you want actions to take place within seconds out in there or milliseconds And one of the things we talked about on a cube pod John is what's going to happen on-prem? Like that middle piece of that that that power law, you know, will the on-prem guys have that model? Optionality that's in the cloud today. You were definitely on the side of hey the lot of stuff is going to happen on Yeah, I mean I think the models models point to the fact that the data is back in the game again It's the data is where the action is and it's just in the word proprietary I use that on purpose because that's a bad word the other bad word that we used to the poo poo You know 20 years ago 15 years ago was walled gardens if you look at the models the ones that are successful The ones Rob are proprietary intellectual property of the company or the entrepreneur and they put a wall around it And then they interface with other models via API. So you're going to start to see these data sets look like walled gardens Because the high quality actually makes them better the bigger the model the more Dirty data or bad data you have the more hallucinations kick in so I think we're going to see kind of a neural network AI system emerge And this is what's going to be interesting to see how that app development kicks in So it's the classic cloud game infrastructure middleware and apps Dave in the middlewares the data and The apps are going to be either AI wrappers like we're seeing today or Other cloud cloud native AI native. So we're going to start to hear AI native more I don't really know what that means, but we get buzzword, you know But and and as well to your point when you talk to customers about where they're spending their time They're spending that time on getting their data quality right because we don't have the data quality right and vast Is coming in today and they talk about this concept of model collapse, right? They've Chris Miller had a pretty good article on that where when the large language models are actually creating more data There's derivative data, you know, it starts to get stupid Well, there's three this three this three areas I want to talk about you guys about and get your reaction because we've been seeing it in some of the interviews There's three areas of AI development right now One is I called the AI wrapper and that's where people take their data They call open AI or a big language model and wrap that around their data and produce a UI and we we do that with some of our cube data I think Bloomberg's doing it with their GPT That's that's a like a website to me That's like an instant app and I used to be kind of down on that but I actually see that as a very viable game plan and then there's going to be the cloud native AI native Proprietary app and then there's going to be some sort of infrastructure piece developed That's where the action is right now and right now everyone loves the AI wrappers because it's very easy to execute if you got the data And I think that's one category and the other one that's that's a wildcard right now is how do you actually build From the ground up modern application with AI native built in But I think you hit on a really good point in the Bloomberg as an example because IBM for instance for Watson X has gone out and trained one of the granite versions of Watson X for finance on the Bloomberg data So you start to look at where that data comes from and how it's actually utilized and Can it be used on-prem or in a smaller model in that very specific or segmented language model that can be used and That's bringing an opportunity for companies to monetize their data in a different way And I think we're going to start to see that as well as part of the economy is how do I get better data cleaner data That I know is known good for doing you know I want to produce a you know LLM for my CFO so that my 10 K's my 10 Q's get developed in the same way and they're Formatted that's a really good use case for it So you start to look at the use cases that get pulled along with this and the cleansing of the data the importance of that where it Is and how it gets refreshed okay? I don't want to push my 10 Q data my financial data into a model But I want to use a model on it So how do I do that and how do I do that wall to garden in this you know effective way and Rob You're talking mentioned some of the horses on the track. It's probably worth naming some names AWS with sage maker They were one of the leading companies you had Microsoft because of their deal with Databricks They were pretty prominent Google always had good AI They just didn't have the cloud momentum of the size of the cloud what happened was the open AI deal with Microsoft First of all open AI shot up to the mind-share lead Everybody's using you know their products whether it's chat GPT or other tools that they have Microsoft does that deal with them they shoot up Amazon still prominent but moved a little bit You know not as prominent just in terms of the the mind-share and the market share Google ticked up a little bit and then you had anthropic You know because they're everywhere coming out of nowhere Databricks still pretty prominent and you have IBM you mentioned IBM They went from kind of nowhere on the momentum and now they're popping back up and even Oracle of course is showing up So they're they're the incumbents that are playing and they have a smattering of other companies That are in there like data robots of the world and so it's a pretty crowded space right now But the big cloud guys are fairly dominant Well the cloud guys aren't going to go away and that's the dynamic but you bring up a good point And I think this is a nuanced point, but I'll bring it up because you mentioned those names All those companies are all saying the same thing. We've been doing AI for a long time Okay, that's like the classic line and they're right, but they weren't doing generative AI I think what's why we're focusing on the super cloud is generally AI is that that creates a new dynamic is when you generate Something that's new data and the question of whether it's good data or synthetic data as you generate Data concepts like vector databases and retrieval kick in so things like memory come up I've had a conversation Multiple times in the past month they were the word memory came up not like memory as in like hardware remembering remembering that in the retrieval vector embeds that this answer was good Because that could go away. So now there's a whole observability question Rob, which models which tweak knob that I push Changes the output if you look at a lot of these these AI models you type in the same question you get different answers Yeah, and so that's not remembering anything. So memory is a huge topic Yeah, and you can talk about data lineage and catalogs and how that all ties together I think when you get to it there is this hey We're still learning how to build these platforms to really be effective long term I mean everybody complained when chat GPT 3.5 started to you know give worse answers because they were segmenting the model down and how that was being retrieved But you start to look at how do you get at the right answers more consistently? Especially if you're building it yourself, and I think that's where people are really focused is how do I build the infrastructure out? Where do I build it out? Where is it most cost-effective? And I think one thing that we probably won't have time to get to is the sustainability of it because that is a huge issue when you look at I think it's six Chat GPT requests uses 16 ounces of water in Microsoft's data center in Iowa It's like it's a bottle of water for you know doing one prompt You know you're going in to figure something out and chat GPT that's huge Yeah, and that's where that long tail edge it is and efficient processors is going to well I mean not to bring up like the distributed computing but blockchain day We've been talking about this data sets being distributed in these walled garden little data sets You know I think that the end of the day the future when you look to the future the what will be written about our era My prediction is that we're going to look back and saying what we knew about data warehouses and data is completely going to change I think we're going to see a complete change over radical Transformation of how data is managed to distribute a date. Well just data in general I mean Rob and I riff on this all the time on the Cube It's like you know, you know snowflake and data bricks They could be extinct if this goes in a distributed way or they get bigger and better But certainly the old data silo is going to go away because unless it's integrated into the AI so again The old way of handling data management is going to be upside down But I think it's going to buy into the governance It is going to go distributed and those companies are going to have to pivot and and adopt and adapt to that distributed nature Well, I mean data bricks and an event they announced the open source They got parquet and iceberg now with just Everybody is that whole all those worlds are coming together, right? I mean they have deltaring and everything else I think put that lower the bar to get into the to scale side of the data. How do you scale the data? It's a disruptive force There's no question about it and those companies have to respond and it's going to be distributed I mean, there's no doubt they're all they all know it and I think it's a what segment of this do I do? I think what we're seeing is that a lot of features that have been Data platforms are coming together and you can't just be one piece of the data platform anymore You have to have multiple pieces to be viable going forward. Yeah, the buzzword is bring AI to the data Well, the data is going to be everywhere Yes, you can be bringing AI everywhere and that's where your comments about inference or critical Costs are driving it. Look at the silicon action. It's in David infrastructure data Absolutely your stack and I would just encourage people to go check out the latest breaking analysis We did some the ETR guys and their survey work, you know a lot of the you'll you'll see the momentum of the various companies And it's it's worth sort of putting in context. Well, we got a great program. We are here This is part of our super cloud for this is our part of our in studio live performance We're live on the cube here and we have guests coming into the studio folks who couldn't make it We're going to bring them in remotely to have a conversation here with the cube, but we got some great lineups here We got a founder panel coming up. We got a Executive panel with the analysts. We got gen AI startups geni enterprise leaders and of course, we got the big the big companies AWS We got AI 21 labs hot startup and and one of the large language models and just some great companies with their leaders We got Intel coming on as well Hopefully address all these concerns around processing and compute and we're going to hear some interesting ideas Around how to get these workloads into production. What's the bottleneck? What's the big trends? This is where we unpack January I thanks for watching and states with us more for the next keynote presentation from AWS