 From theCUBE Studios in Palo Alto in Boston, bringing you data-driven insights from theCUBE and ETR. This is Breaking Analysis with Dave Vellante. This week, we spent the better part of a day in New York City reviewing AWS's AI strategy in progress with several AWS execs, including Matt Wood, who was the VP of AI at the company. We came away with a much better understanding of AWS's AI approach beyond what was laid out at Reinvent 2023. We also met separately with a senior technology leader at a large financial institution to gauge resonance with AWS's narrative. And while the stories from both of these camps were very positive, the data still shows that open AI and Microsoft continue to hold the AI momentum lead, that's a trophy, the pair you surf from historic time to market champion AWS. The question is, can AWS take back that lead? How will it attempt to do so and what external factors could propel or thwart its aspirations? Hello and welcome to this week's theCUBE Research Insights powered by ETR. In this Breaking Analysis, we review the takeaways from our AI field trip to NYC, we'll share survey data from ETR on GIN AI adoption barriers and place these in context to the recent scathing review of Microsoft's security practices by the government's Cyber Safety Review Board. Let's first talk about LLM diversity, open source and domain specificity to set some context. Early last year, the CUBE Research published the power law of GIN AI shown here. The basic concept is that while some industries have a handful of leaders and a long tail bit players, we see the GIN AI curve differently. On this chart, we show model size on the vertical axis and domain specificity on the horizontal plane. And while a few giants like the hyperscalers are gonna dominate the training space, a very large number of use cases will emerge and are emerging and will continue to do so with greater industry specialization. Open source models and well-funded third parties will pull that torso up to the right as shown in the red line. And they will support the premise of domain specific models by helping customers balance model size, complexity, cost and best fit for the use case. Now, I wasn't able to obtain the deck that Matt Wood shared with us. So I'm gonna revert to an annotated version of a slide that Adam Salipsky showed at re-invent last year to kick things off. This depicts AWS's three-layered GIN AI stack, which comprises core infrastructure for training foundation models and doing cost-effective inference. Building on top of that layer are tools to leverage LLMs, which is bedrock and the other tools inside of that, managed service and at the top of the stack queue, which is Amazon's effort to simplify the adoption of GIN AI. So let's talk about some of the key takeaways from each layer of that stack. First at the bottom layer, there are three main ones, AWS's history at ML and AI and two, it's custom silicon expertise and three, the compute optionality and of course, time to market with those EC2 instances. And we're going to take those in order. Amazon emphasized that it has been doing AI for a long time. A lot of companies do so, but Amazon has had a presence in ML with SageMaker, which is definitely has been a leader and that is true that they've been in that market for a long time. But SageMaker, while it's very powerful, is also complex. Getting the most out of SageMaker requires an understanding of really complicated ML workflows. You got to choose the right compute instance. You got to integrate into pipelines and IT processes and so forth. A large proportion of AI use cases can be addressed by SageMaker and GIN AI presents an opportunity to be an orchestration layer to simplify and widen the adoption of traditional ML tools like SageMaker. Now as well, AWS has a long history developing custom silicon with Graviton, Tranium and Inferentia. And in addition, it offers EC2 options that sometimes make your head spin, including GPUs from NVIDIA. AWS claims it was the first to ship H100s and it will be the first it says to market with Blackwell, which is NVIDIA's new super chip that they announced at GTC last month. Now, AWS's strategy at the core infrastructure layer can be boiled down to key building blocks like Nitro and internet working to support a wide range of XPU options with its own custom silicon that can do work at much lower costs. Now let's move up to the next layer on this chart. This is where much of the attention is placed because of this layer competes with open AI. We've got Amazon Bedrock here. It's the managed service platform by which customers access foundation models and tooling to ensure trusted AI. And we've superimposed on Adams chart several foundation models that AWS offers including AI 21 Labs, their Jurassic option, Amazon's own Titan model, Entropic Clawed, perhaps the most important of the group given AWS's $4 billion investment in the company, Cohere, Metaslama, Mistral AI with several options including its mixture of experts, MOE model and its Mistral large, which is its flagship. And it finally stability AI stable diffusion model. We expect to see more models in the future including maybe who knows even DVRX from Databricks. And of course, Amazon will be evolving its own foundation models. Last November, you may recall a story broke about Amazon's Olympus, which is reportedly a two trillion parameter model headed up by the former head of Amazon Alexa. And this will be one of many AWS barraisers in the LLM game. Finally, the top layer is Q an up the stack application layer designed to be the so-called easy button with out of the box GNI for specific use cases like Q for supply chain or Q for data with connectors to popular platforms like Slack and ServiceNow. Essentially a set of GNI assistance that AWS is building for customers that don't want to build their own. Now let's look at the reality in enterprises and in a March survey of almost 1400 IT decision makers nearly 70% said their firms have put some form of GNAI into production. And this chart shows from ETR the 431 that said they hadn't gone into production and asks them why. And the number one reason was of course they're still evaluating but the real tell is the degree to which data privacy, security, legal, regulatory and compliance concerns are key barriers to adoption. It's no surprise unlike the days of big data where deployments went unchecked most organizations are being much more careful with AI. Now given this concerns about privacy and security one can't help but reflect on the recent report initiated by the head of Homeland Security to investigate the hack traced to China one year ago on Microsoft that comprised the accounts of key government officials including the commerce secretary this report it's I don't know a 30 page report. I heard about in the news I got in early this morning and read it. I highlighted some key takeaways and it's astounding to the degree at which the government in this report scoriates Microsoft. I mean it eviscerated the company for prioritizing feature development over security using outdated security practices failing to close known gaps poorly communicating what happened, why it happened how it's going to be addressed still to this day there's uncertainty. This story was widely reported but it's worth noting in the context of AI adoption and near the few couple of call outs and poll quotes from that report quote the board finds that this intrusion was preventable and should never have occurred board also concludes that Microsoft security culture was inadequate and requires an overhaul. Interesting things the term was because based on reading to the report we haven't heard how it has changed. Second point call out throughout this review the board identified a series of Microsoft operational and strategic decisions that collectively point to a corporate culture that deep prioritize both enterprise security investments and rigorous risk management. The report also evaluated other cloud service providers and specifically called out in a positive light Google, AWS and Oracle and gave specific best practice examples of how they approach security. Now, why is this so relevant to AI other than the obvious? Look, the cloud has become the first line of defense and cybersecurity and there's a shared responsibility model that we have all heard about and they generally understand with cloud. And if you're a CEO, a CIO, a CISO a board member, a P and L manager and you're a Microsoft shop this breach, which by the way it wasn't even discovered by Microsoft it was discovered by its customer you're relying on Microsoft to do its job in the shared responsibility model and Microsoft is failing you. Your business is at risk and this is especially concerning because of the ubiquity of Microsoft and its presence in virtually every market. Satya Nadella, you saved the company from irrelevance when you took over from Steve Ballmer and initiated a cloud call to action. Microsoft however has violated the trust of its customers many of whom are now putting their AI strategies in your hands and you have to do better. You have the resources to do better and you have to change the culture is what this report said. Now, if you're a business technology executive this should be a wake up call. You need to think about hedging your bets. You need to think about your AI strategy. You need to think about reducing your risks. But look at the data and you'll see many customers are ignoring this threat. Here's data from the very latest ETR Technology Spending Intention Survey, TESIS, of more than 1800 accounts and I got permission from ETR to publish this ahead of their webinar for private clients. The vertical axis is spending momentum. We call that net score on a platform. This is just for the AI and ML space. The horizontal axis is presence in the data set. It's measured by the overlap with that platform has within those 1800 plus accounts. Red line at 40% indicates a highly elevated net score or spending velocity. And the table insert in the bottom right shows how the dots are plotted. Net score by the number of N, the number of mentions in the survey. The first point, open AI and Microsoft, look at them, they're off the charts literally in terms of account penetration and this underscores the risks that we just talked about. Point two is really interesting. AWS and Google within the AI sector are much closer than they are if we were looking at just the cloud segment. AWS is far ahead of Google when we cut the data on cloud accounts but Google appears to be closing the gap when it comes to ML and AI. We both have very strong net scores and very solid presence in the data set but the compression between these two names is notable. Point three, look at the moves that both the anthropic and Databricks have made in the ML AI segment. Anthropic in particular with a net score rivaling that of open AI, albeit with a much smaller N. But that is AWS's perhaps most important LLM partner as we talked about earlier. Databricks as well as moving up and to the right. Now my understanding is that ETR will be adding snowflake in this sector. Snowflake you may recall, essentially containerizes NVIDIA's AI stack as it's AI strategy or one of its AI strategies, a primary AI strategy. And so that's one of their main plays. It's going to be really interesting to see how they fare in the days ahead. Now the last point is in the last survey, Metislama was ahead of both anthropic and Databricks on the vertical axis. And it's interesting to note the degree to which they've swapped positions essentially. We'll see if that trend line continues. But coming back to the previous discussion and security and trust, this data is a wake up call to those exposed to Microsoft and we must hear from the company as to what its plan is to remediate this massive customer risk, especially in this age of AI. Okay, now coming back to AWS, who as you see from the previous chart is doing well, but if you believe that AI is the next new thing, which we do, then one, the game has changed and two, AWS has a lot of work to do. So what are the some of the themes that we heard this week from AWS that we can consider and think about the company's future in this market? Matt Wood laid out an eight step journey that they see from a customer AI perspective, customer AI's initiatives. And they're not really, really not a journey. They're not really linear steps, but he used that term, which I'm fine with, but I wanted to just point that out. It's not like these are sequential steps along the way. They're just certain things that customers are doing that AWS has been exposed to, that they're helping them do. Think of them as key milestones or objectives that the customers have. It starts with training. And we don't want to spend a lot of time here because most customers are not doing hardcore training. Rather, they start out with a pre-trained model from the likes of an anthropic or a mistrall or whatever. Step two is perhaps the most important, IP retention and confidentiality. And despite that ETR data that we just showed you, many folks have banned the use of open AI tools internally. But I know for a fact that developers, for example, find open AI tooling to be quite good and better. For most use cases or many use cases, I give an example of code assistants. I know Devs, whose company, their company has banned the use of chat GPT for coding and because of concerns over privacy, but rather than use, for example, code whisperer, which they could do, Amazon account. In many of the cases that I've evaluated, what they do is that because they find open AI tooling so much better, they download the iPhone app and they do it on their smartphone. And again, this should be a concern for ZeeSOS. Customers should be asking their AI provider if humans are reviewing the results, what type of encryption is used, how is security built into the managed service that you get access to the LLM through, how is training data protected, is it separated, can data be exfiltrated? If so, how, what are the exposures there? How is access to data that flows through the system? Is it being fenced off from the outside world? And even the cloud provider, can the cloud provider get to that data and how so and what controls are there? So these are key issues that the customers have to be thinking about that up fairly confident AWS has thought about from the ground up. Okay, step three is widely adopting gen AI to applying it to the entire business. And when you look at it, what they're really, most companies are doing, they're doing tech summarization, document summarization, maybe image generation, maybe code assistance, they're pretty standard things. The reality is customer use cases are piling up. The ETR survey data shows that 40% of customers are funding AI by stealing from other budget buckets. So the backlog is growing and there's a lot of experimentation going on. Now, interestingly, historically, AWS has been a great place to experiment from the cloud. But from the data we showed, open AI and Microsoft are getting a lot of that business, a lot of that experimentation is going on with open AI and Azure. AWS's contention is that other cloud providers are married to a limited number of models. They'll say things like, there won't be one model to rule them all, by the way, we agree. Clearly Google wants to use its own AI models, but it's got other choices. Microsoft prioritizes open AI, but at Ignite last year, it announced other models support. And this is one where only time will tell. In other words, does AWS have an advantage over other players with foundation model optionality? Or if it becomes an important criterion, can others expand their partnerships and add optionality? Now, they already have, but perhaps not to the degree that AWS has, the competitors I'm saying. So AWS tried to position its model diversity and optionality as a potential flywheel where some models can assist other models and play off of other models and maybe perhaps train other models or leverage data or the right tool for the right job. So we'll see. And the other thing is AWS has got like 400 instance AC2 instance types and will this be an advantage? Right tool, again, for the right job, for cost optimization, for example, or a better use case fit. Now, some would say they'd like gen AI to help to optimize for all those instance types and no doubt that's coming. And of course, we're seeing simplified rag models or expanding the adoption of gen AI, which gets to step four, which is that kind of getting that consistency and fine tuning those rag models as an example. Matt Wood talked about the Swiss cheese effect where if a rag has data, it's pretty good, but where it doesn't, it's like holes in the Swiss cheese. And then that's when models start to make things up. Step five gets deeper into industry problems. And again, I stress this is not like a sequential journey. And this is one, however, where people want to go beyond some of these, you know, more basic document summarization or even, you know, coding assistance. And they want to get into deeper industry problems, solving cancer, restructuring an entire business, drug discovery and the like. And so there are some advanced organizations doing that. Step six is cost optimization. I'll use that term. AWS didn't, but it's where, you know, AWS touts its custom silicon. You don't need necessarily NVIDIA GPUs like Blackwell to do inference, use inferencia. A lot of times you can train on training. You know, maybe you're not going to get like a hundred percent of what you get, well, not maybe, you won't get a hundred percent of what you might get from an H100 or Blackwell, but you might get 70, 80% of the way there and it's good enough. And it's, you know, who knows, half the cost, maybe even less. So that's something that AWS, I think generally as we've talked about, you know, has an advantage and a lead on much of the competition because it's been designing custom silicon back early last decade, maybe mid-last decade. Step seven is rolling out common use cases like document summarization, code assistance. Those are things that are very, very common. Step eight is simplifying with Q. Think out of the box, gen AI use cases. So those are the sort of a quick run through of the eight steps or goals, initiatives that customers are pushing AWS to help with. Now some other key points that we want to touch on. Bedrock adoption appears very strong with tens of thousands of customers claimed by AWS. We also met with industry experts at AWS in financial services and we had cross-industry pros that we were able to probe. We talked about numerous use cases and insurance, financial services, media, healthcare, you name it. And that's right in line with the power law that we discussed earlier. AWS is positioning itself as a platform to support scale. You know, one of the interesting examples was Adobe Firefly, Adobe Firefly was trained on AWS and we talked about this last week in our breaking analysis that Adobe is doing personalization at scale. And the last three on this chart, we've touched on a little Silicon and LLM diversity, ecosystem partners and companies like Adobe training on AWS with products like Firefly, security and up-to-stack applications with Q. My sense is Q is still a work in progress. Packaged apps are really not AWS' wheelhouse, but it's a start and perhaps GNAI makes it easier for them to enter upstream markets. Okay, we got a wrap. Let me leave you with a few points. Microsoft and OpenAI, they stole AWS' decade plus time to market advantage, so can they get it back? Thanks to watch, watch anthropic. Not only AWS' use and its customers' use of anthropic, can anthropic help AWS, for example, develop better silicon and low-cost silicon. How important is model choice and model diversity? That is a linchpin of AWS' strategy and of course watch AWS' own foundation models. And it's custom silicon, how fast that gets adopted and the ecosystem adoption. You know, another big question is, will models become commoditized? There's a real debate in the industry about this. So on the one hand, you've got folks saying, oh, foundation models, LLMs, they're gonna be commoditized. That's a race to the bottom. There's another school of thought that says, well, maybe not. Maybe it's a right tool for the right job and maybe the combinatorial effect of foundation models could actually deliver incremental value. Now, if that's the case, can competitors match AWS' diversity or will there be a moat? You know, one would think with open source models, there's actually not much of a moat there, but maybe through its silicon knowledge and other tooling that is tuned for these various models, maybe AWS can reduce the commoditization effect and take advantage of this. AI trust has to be a decision point, but will ease of doing business win out? And what about private AI and AI cloud alternatives? And speaking of AI cloud alternatives, like GPU clouds, our friends at Vast are doing very well in this space. At NVIDIA GTC, John Furry and I attended a lunch hosted by Vast with Genesis Cloud, which was very informative. These firms are really taking off and positioning themselves as a purpose-built AI cloud to compete with the likes of AWS. So I asked the folks at Vast that could give us a list of the top alternative clouds that they're working with in addition to Genesis, names like Core 42, Core Weave, Lambda, Nebul, are raising tons of money, they're gaining traction, and look, maybe they won't all make it, but some will to challenge the hyperscale leaders. Can AWS increase the adoption of AI with Gen AI as the orchestrator? In other words, can it take some of the complexity in its platform and use generative AI as the orchestrator and queue as a simplifying abstraction layer? In other words, can Gen AI accelerate AWS's entry into the application space, or will its strategy continue to be enabling its customers and ISV partners to compete up the stack? Maybe it's not an either or, maybe it's a both. Okay, that's it for now. What do you think? Can AWS's strategy resonate with you in AI? Are you concerned about Microsoft's security? Will it make you reconsider your IT bets? Let us know. All right, thanks to Alex Meyerson and Ken Schiffman on production, and Alex does our podcast as well. Kirsten Martin and Cheryl Knight help get the word out on social media and in our newsletters, and Rob Holt is our EIC over at siliconangle.com does some wonderful editing, thank you all. Remember, all these podcasts are available, all these episodes are available as podcasts wherever you listen. All your new search breaking analysis podcast, I publish each week on thecuresearch.com and siliconangle.com. You can email me at david.valante at siliconangle.com or DM me at dvalante. Or comment on our LinkedIn post, and please check out etr.ai. Their data just keeps getting better and better and more granular. They get the best survey data in the enterprise tech business. This is Dave Vellante for theCUBE Research Insights powered by ETR. Thank you for watching and we'll see you next time for some breaking analysis.