 From theCUBE Studios in Palo Alto in Boston, bringing you data-driven insights from theCUBE and ETR. This is Breaking Analysis with Dave Vellante. Our research indicates that the adoption of generative AI is occurring across a variety of frameworks with diverse deployment models and domain specificities. The potential of generative AI to transform industries is immense. And we believe in the narrative that AI will be brought to where the data lives. The power law of gen AI, developed by the CUBE Research Team, describes the adoption patterns that we see emerging in the cloud, on-premises, and at the edge, with a long tail of highly specific domain knowledge across virtually every industry. Hello and welcome to this week's CUBE Research Insights powered by ETR. In this Breaking Analysis, we revisit the gen AI power law, which describes how we see adoption happening within industries and organizations globally. And we introduce new survey data from ETR that looks at some of the vendors that are being adopted or considered with a special peak into the emerging technology sector. First, let's review the concept of a power law. A power law, also called a scaling law, is a relation of the type of the following. Y equals AX to the power of beta, where Y and X are variables, those are the ones of interest. Beta, it should be a power there, an exponential, it doesn't show in my slide here, but beta is called the power law exponent. And it is typically some constant, sort of some irrelevant constant. For instance, it's not irrelevant, it's important, but it's not relevant for this discussion. So here's the example. If X is multiplied by a factor of 10, then Y is multiplied by 10 to the power of beta. One could say that Y scales as X to the power of beta. Now, mathematics, an example of a power law that many of you will be familiar with is a Pareto distribution, otherwise known as the 80-20 rule. Now, in precise mathematical terms, the rule states that 80% of the outcomes are due to 20% of the causes. But for our purposes in this breaking analysis, we're not so much interested in the cause and effect relationship, rather we want to use the shape of the curve to describe the adoption trends and patterns that we see evolving in Gen AI and what it might mean for organizations, industries, and adoption models, and the application of those data models. Now, the power law of Gen AI, developed with our analyst team, myself, John Furrier and Rob Streche, describes the long tail of Gen AI adoption. So in this diagram, the Y-axis represents the size of the LLM and the X-axis shows the domain specificity of the models. The orange line there is an example of the historic music industry where only about four labels just completely dominated the record scene. And so the shape of the curve was a hard right angle as you see there. Now in Gen AI, we see a similar but somewhat different pattern in that first the cloud titans along with NVIDIA and open AI and a handful of other companies will be few in number, similar to that sort of orange line. They'll train giant LLMs, but most applied models are going to be much smaller, creating that long tail. And that long tail will occur not only in the cloud, where much if not most of the action around Gen AI is happening today, but will also be very domain specific within industries and result in on-prem deployments that stretch to the edge and to the far edge. AI inference at the edge, we believe will be the most dominant use case from a volume perspective. And inferencing will occur using smaller data sets in the cloud and on-prem as well. Now the red lines here pulling the torso up to the right are a function of a spate of third party and open source tools, LLMs that are emerging that are going to have market impact in our view. Now the other points are shown in the gray box in the upper right here. We believe that like many waves, the consumer adoption in this case of chat GPT catalyzes innovation that bleeds into the enterprise. And our research shows that enterprises are currently going through a cycle of evaluation and experimentation, then leading to production with very specific use cases. Most are still in the evaluation phase, possibly as high as 90%. But those in production that are successful are showing very fast ROI and sharing the gains to invest more. This is important because today as we've reported most Gen AI is being funded by stealing from other enterprise IT buckets. Now coming back to the conversation on volume economics in the upper right here, the consumer adoption will in our opinion result in dramatic shifts in the economics of running workloads, which will eventually disrupt traditional data center or even possibly cloud economic norms. Now we are particularly interested in the Silicon Wars taking place. To us, the arm model is most compelling because of its maturity, it's time to market, power per watt advantage or performance per watt in particular, but most importantly, the volume of arm wafers as compared to any other standard like that of x86. Arm wafers, wafer volumes are probably 10x those of x86. And the performance coming out of the arm ecosystem is progressing at a rate of two and a half to three times that of historical mores, law progressions when you combine the combinatorial effects of CPU, GPU, NPU and accelerators are growing at over 100% a year compared to let's call it 30% a year, maybe 35 in the heyday of Moore's law. So it's no coincidence that Apple got this all started with iPhone volumes and is TSM's biggest volume driver and is the number one enabler of TSM's Foundry leadership. But it's not only iPhone, Apple laptops, AWS, Nitro, Graviton, Inferentia, Tranium, Google and Microsoft following suit, Tesla, Oracle with Ampere and many others are driving this innovation. Oh yes, let's not forget NVIDIA which is an arm based platform and this is all related to cost. Arm is the low cost architecture and we believe will remain so because of the volume economics and rights law. Now we're well aware that the entire industry is trying to find alternatives to arm in the form of risk five open standards and alternatives to NVIDIA and GPUs but the experience curves for these companies around arm and NVIDIA they're years ahead of the pack in our view. And it's going to be very, very difficult for the industry broadly to catch up. And the reason we spend so much time thinking about this is we see the long tail of gen AI and the massive volumes of embedded systems spawning new data consumption models that will radically alter the economics of enterprise IT in similar ways that x86 won the game against risk back in the day. So let's review how these IT decision makers are thinking about which vendors of large language models they're adopting today for gen AI and how are they thinking about future adoption and partnerships. This data from a recent ETR drill down asks customers which products are currently in use for gen AI in which are planned for evaluation. This is part of a cloud survey so it's biased toward cloud but that's where the action is today for the most part. And you can see the dominance of open AI and Microsoft. Remember Microsoft said that gen AI provided a 300 basis point tailwind for Azure this past quarter. And they're the only company to have expressed meaningful revenue from gen AI of course other than Nvidia. And in the chart you can see vertex AI and Amazon in strong positions. AWS late last month announced bedrock was in GA so we'll likely see a fast uptick there based on this data and we reported that last week as well. As well also we talked about Watson X last week. IBM finally got it right in our opinion and as the data is showing has had a nice uptick in adoption but again we showed last week I'm not specifically showing it here but you can see it's got some potential. But the new call out of this data in this chart is the impact of hybrid AI. That notion of bringing AI to the data and we think that occurs in the cloud, on-prem and at the edge because that's where data lives. Oracle as well is the other call out on this chart as is other. As you no doubt are aware there are many other players and as we showed in the power law graphic so let's take a look at some of the names that are worth noting. Here's a chart from ETR's emerging technology survey. It's a relatively new addition to the mix of ETR surveys. Their new CEO decided to accelerate this. I believe they're investing at least three, maybe even four times per year. The ETS survey measures net sentiment on the vertical axis and mind share on the horizontal. And this is only for privately held companies. And ETR will from time to time throw in capture open source tools like you see here, TensorFlow. The vertical axis measures intent to engage in the horizontal measures awareness. And this is a mix of machine learning and AI tools so it's not just gen AI and there's other AI you didn't know besides gen AI. I'd be happy to explain the methodology in more detail if you ping me and want more info on that or I could put in touch with the ETR folks. What we want to really emphasize here is one, the market has many, many players. You can see how crowded it is. And two, open AI came on the scene and is dominated. You can see there in the upper right, the lines show the progression over time and the difference over the different survey periods. In the August you see the progression of hugging face. They made tremendous progress on both dimensions. And you see Cohere, Anthropic and Jasper AI which is a co-pilot for marketing teams. And you can see the mix of other ML and AI players which include the always prominent Databricks. But the one major force that hasn't shown up in the ETR dataset just yet is Meta's Llama 2. Meta introduced Llama early this year and in July announced Llama 2 in seven, 13 and 70 billion parameter versions. Llama 2 is an open sourced LLM that Meta is deploying internally and is gaining lots of attractions as an alternative to open AI. It's a free download. Now for context, Google's Palm is said to be built around 500 billion parameters and ChatGPT is estimated to be like, if you think over a trillion. So Meta is showing with Llama 2 that the massive potential of smaller language models. The other really important thing that we haven't talked about is one of the most significant barriers to gen AI adoption today is concerns over privacy, legal exposures and compliance. Now also coming back to the power law of gen AI, many companies want to deploy gen AI on-prem where much of their sensitive data still lives. Think financial services, healthcare and other related industries. And of course, the edge. There's no hard data on this, but in our conversations with various industry sources, it's indicated that over 50% of the Llama 2 deployments could very well be on-prem. At the very least, the majority of the downloads are from companies that have significant data center deployments, in-house data centers or colos. So this fits squarely into our thinking on the long tail of the application of smaller models. Now this doesn't mean you won't see smaller domain-specific models deployed in the cloud. You will, absolutely. The point is data, or these models are going to be data location dependent. Now the other caveat is the long tail is going to have little camel humps, maybe not so little along that curve where there will be organizations, for example, in high performance computing, super computing applications that use very large data sets. So keep that in mind as well. We'll be watching for that. Now there's a major industry movement around retrieval augmented generation or RAG. RAG dramatically simplifies and improves the deployments of Gen AI and will support this on-prem and out-to-the-edge thinking. And again, it'll happen in the cloud as well. There's the problem. The problem with LLMs is they're inconsistent and sometimes flaky. They hallucinate a lot and are often unreliable in a large part due to their very wide scope of knowledge from so many data sources. So organizations are using RAG, which is an AI framework to improve the specificity of data used in their LLMs. What RAGs do is they take the queries from the users and they vector in specific domain knowledge to complement the LLM using a vector database like a pine cone or a milvus. So the original user prompt is then augmented with the proprietary knowledge base and then sent to the foundation model resulting in better responses. So this approach is more accurate. It reduces hallucinations and it provides domain specific value that generalized LLM models like chat GPT don't in and of themselves. So it gives you better control over the data used by the LLM when formulating a response. So let's take a look at an example of a RAG that I'm actually quite familiar with, the CUBE AI. The CUBE AI is a good example of a domain specific model on that long tail of Gen AI and the power law that we showed you. We've ingested a corpus of around 35,000 videos from the CUBE and other data that we have. It allows our audience to ask very specific questions about companies, strategies and industry trends. It's an AI powered engagement platform that works as a virtual analyst to summarize video content, extract and share key insights and provide AI assisted search. You can check it out. Go to thecubeai.com, it's in private beta, sign up, John Furrier will let you in. So for example, I asked the CUBE AI here, what is Dell's AI strategy? And it comes back with a very high fidelity answer. It says Dell's approach to AI is based on a four part framework, AI in, AI on, AI for and AI with. So AI in focuses on embedding AI into Dell's offerings, AI on, that involves building infrastructure where customers and partners can run their AI and their LLMs and their machine learning workloads. On top of that example would be Project Helix and their reference architectures that they're building. Then the AI for dimension has two prongs. First, Dell applies AI capabilities and tools to improve their internal processes. And second, it shares, Dell shares their knowledge and best practice with customers to support their AI journey. And then AI with focuses on enabling ecosystem partnerships. And the AI, let me look at it, it gives me a relevant clips along with this really high fidelity narrative, which by the way, the clips were also made by AI. I was a little side here, there are a lot of learnings. One is relevant to the conversations about AI taking jobs. And I got to tell you the truth is that CUBE AI has actually eliminated at least one job. We used to have a human making clips manually but AI now does that with a push of a button and we can control the length of the clips, the context, the quality, which models we're using. But I had to tell you that the person that the AI replaced is not unhappy because that individual is like an air traffic controller staring at videos and clipping videos all day long. So the jobs thing is a two-edged sword. RPA is instructive here. Like RPA deployments, much of the AI is going to be replacing mundane tasks that people don't want to do. A longer term, it could have greater impacts on employment and no doubt, but we'll leave that for another day. The key to rags specifically in LLMs generally is the quality of the data. Now, because we have current data on Dell, the CUBE AI's answer was pristine. But here's an example that is not as great. I asked the CUBE AI, who is the CEO of ServiceNow? And it gives me John Donahoe, a CUBE alum, former CIO, and it references Frank Slutman, John Donahoe's predecessor. Now, even though we've talked about Bill McDermott on the CUBE, the new CEO of ServiceNow, the AI couldn't connect the dots. The data in the corpus just wasn't current enough and so the engineers, we notified them with a thumbs down and a comment and they'll train the system to really key off dates and maybe in this case put in a caveat that the answer is relevant as of a certain date. The point again is the currency and quality of the data is going to determine the relevance and usefulness of the results. So you won't be surprised by this ETR survey data asking customers where their focus is on Gen AI data and analytics priorities related to their Gen AI adoption. Look at improving data quality, improving data warehouses and data lakes and storage data, incorporating more data, improving data literacy, better data integration, revising data governance policies, improving data cataloging, better metadata management. Again, it's all about the data. A lot of data lives in the cloud. There's a ton of data on-prem, probably as much if not more than is in the cloud. I mean, estimates say I've heard 70, 80%. I'm not sure it's that high. I think it's probably more like 60% and trending downward. I think the cloud is growing faster. Cloud data is clearly growing faster. Anyway, there's a lot of data. There's more, there's tons of data at the edge. So this phrase that we use all the time, it's all about the data. What it means in this context is bringing AI to the data and improving the quality of data, putting the right data governance models in place. Data is going to be the key differentiator for those who act. So let's close with some final thoughts on expectations on Gen AI adoption. Top down, bottom up, middle out momentum. What do we mean by that? Clearly the C-suite is saying, hey, what are we doing about Gen AI? There's tons of shadow AI going on. And then of course you've got this middle out where middle management wants to drive productivity and drop money to the P and L, to the bottom line. So all constituents are really focusing in on trying to figure out where they can get more productivity out of Gen AI. Second point here is most organizations are still in the evaluation phase, but those that are in production are showing ROI and using the concept of gain sharing. And this is again important because AI budgets are still largely dilutive to other initiatives. Sure, there's some R and D budgets and some innovation budgets that are paying for some of the Gen AI experimentation, but generally speaking, IT budgets in general are not rising dramatically. So Gen AI is new and new means we've got to figure out where to get the budget from elsewhere and it's not paying off enough yet to justify itself except in those cases where companies have shown ROI and that's not the majority today. The next point here is the industry is scrambling for alternatives to Nvidia and they're looking for open source options like Lama 2 relative to open AI which some people joke is closed AI. You know, Nvidia is very expensive. Some people say, many people say that it's gouging today. And so people are really excited about alternatives like AMD's GPUs, like Intel's capabilities around AI. They may not be as functional. They certainly don't have the robustness of the CUDA software architecture. And that's why you hear people saying, well, that's closed, we're open. So they're competing. But as Andy Jassy says, there's no compression algorithm for experience that the case applies here. Okay, next point. It may be early innings, but tactical positions matter in this horse race. And the reason is having a lead is important when things are moving this fast because the early players are getting the best feedback on what works and what doesn't. And they're getting more data and training the models at a very fast pace. As well, they're learning about monetization models, what's working, what's not working, governance, best practices, and they're really driving thought leadership forward. Now admittedly, this is a two-edged sword. The pioneers a lot of times take the arrows, but the momentum is so strong that I'd rather be on the lead in this horse race than starting from the back of the pack and getting dirt kicked in my face. Okay, that's it for now. Thanks to Alex Meyerson and Ken Schiffman on production. Alex also manages the podcast, Kristen Martin and Cheryl Knight. Help get the word out on our social media and in our newsletters and Rob Hoef is our editor-in-chief over at SiliconAngle.com. He does some awesome editing. Thanks all. Remember, these episodes are available as podcasts. Wherever you listen, all you gotta do is search Breaking Analysis Podcast. I publish each week on wikibon.com and siliconangle.com. You can email me at david.volante at siliconangle.com or DM me at davilante or comment on my LinkedIn post and please check out etr.ai. They just keep getting better and better. Their data, their time series data, incredible outfit over there, great data scientists, love the partnership, best data in the enterprise tech business survey data. Okay, thanks for watching. This is Dave Vellante for theCUBE Insights, powered by ETR. Appreciate your time, your attention and we'll see you next time on Breaking Analysis.