 Welcome back everyone to theCUBE's coverage here for AWS re-invent, 2023, I'm John Furrier with Dave Vellante extracting the signal from the noise with our studio pumping out the content in Palo Alto. We're here on location. 11th year with theCUBE here at re-invent, it's been a great year. And this year marks the big step up with generative AI and just overall price performance focused really bringing that next generation developer mindset and then feeding frenzy soon to be on these LLMs Foundation models. We've got a great guest here, Ed enough, chief product of DataStacks, CUBE alumni, great to see you. We've talked product management of the cloud for a long time on theCUBE with you. Great to have you back, have you back. Yeah, same, same. So big news, you guys, relationship with Bedrock, Astro, you guys got some news, let's get right to the hard news. Sure, so we are talking about here at the conference has been the fact that we've taken AstroDB, which is our cloud database built on top of Cassandra. So it brings you all the power and scale of Cassandra. It's the database that powers Netflix, Apple, Federal Express and all that. Earlier this year, we made it vector powered. So you have all that power, you have all that scale, but all the power of being able to do vector similarity search and vector is the native language of LLMs. So if you're building AI, you've got your LLMs talking to your data, which every business wants to do, it's a great match. But the key piece to that, the missing ingredient, was Bedrock letting you to have access to these models in a powerful, open way. And so we've built that integration and we're talking about it here at the show. So choice and open is always a good form of success. Definitely. They hit a home run with that, I'd say. Now, but the models have gone fast. Yes. Their job is to create a model scenario where developers are going to be in that, it's the layer two of their quote, gin and I stack, which we report on Silicon Angle and they presented. What's that mean for developers? What has to happen? Because to me, my first reaction was, oh, this is going to be a feeding frenzy, yes. People are going to be kicking the tires, but the data management model's going to be upside down because you want to have everything close to the data store, then you want to put stuff around it. Low latency packets, but also low latency time to value on answers. This is a big part of the new equation. Well, I think, you know, we've been saying it for a while, but it's been great to see it at the conference that everybody's caught up with this, that AI and gen AI, once you get past sort of the cool experiments and you know, we all love that stuff and chat GPT and so on. But the key piece for businesses, how do I make AI work with my data? And like you said, it's going to be low latency. You've got to be able to go and deal with the entirety of the data that runs your business, making it available to these language models. And you got to be able to capture all these interactions, to be able to build history with your users, to be able to detect hallucinations and correct them. So it's a pattern that's called RAG, Retrieval Augmented Generation. You probably have heard a lot about it at the conference. It's funny because it started off at the beginning of the year, this funny term and now, you know, you've got a lot. It's in the keynote. It's in the keynote. Exactly, exactly. How far have we come, damn. So when you, thinking about the LLM optionality that is obviously front and center at the show, how do you think about, and what are you finding with the different LLMs and what are the sort of dimensions and the critical sort of factors that you use to determine, you know, whether it's cost, whether it's speed, whether it's accuracy, and how are you adjudicating that in your system? So it's all the ones you listed and one very important one, which is cost. And so those are the trade-offs that we see our users or the developers, you know, at any level of scale, which is that right now you're trying to find that sort of Goldilocks zone of how do I get the best possible relevance at a cost that lets me go and bring this out to as many of my users, whether it's my employees in a call center application or whether it's something I'm putting on my website as a shopping assistant. But I need to be able to push a lot of data through this and these LLMs are not all created equal in terms of cost. And so people are experimenting with small models. There's a number of open models, a lot of stuff that we see on a hugging face and then a lot of the great foundation models that you saw Amazon talking about from partners and that they've developed and you know, whether it's anthropic or open AI, all of those are choices that people have. What we try to do from an architecture standpoint is let you go and easily switch between them in any situation and all those situations, you're bringing your data to the model but we're giving you that optionality to tune it so that it gives you the right relevancy, the right accuracy at the right cost that makes this possible for whatever it is you're trying to do with your business. So one of the concerns that customers had and sort of epiphanies when the cloud was kind of the wild, wild west is, oh wow, I got my bill. I didn't expect that. And so then we had a lot of emphasis on whether it was optimization or FinOps, et cetera. Is there a similar dynamic with the LLMs in other words? Can the AI help you choose the right LLM? That's a good question. There are some folks working on that. I think right now, the challenge is that every business, and I talked to Fortune 500, I talked to startups, every business is going and figuring out how do I inject the LLM into my core experience, into my business model and there's a ton of great ways to do that. And then they go and say, okay, can I get the technology right? Can I get it to work within my business model? And then what will it look like when I put this into production? And right now the reality is there's a ton of great stuff but a lot of it is in the early stages and as they get closer to going and putting it out that's when the cost starts to kick in, they're like, this is great, we built a fantastic demo to show to our investors or board of directors, everything and they get it and we're now green lighting it and now the question is, how do we put this into production? I don't know that it's a situation of going and saying, do I need a model to help me build the cost model? Although, as I said, people are doing that. I think it's more of a case of, is it aligned with the infrastructure that I already have and the cost models and the operational models and that's why for us, the stuff here, obviously AWS is, you know, the giant here and for most people that have their systems in production they're doing it on AWS, very important for us to make sure that we're tied in with that system because next year and the year after that we're moving from experiment into production and these are the systems that they're going to use to do that. Things we've seen from the spending data is still, most people are still in experimentation, although it's starting to uptake into some fast ROI but we've noticed that there was a very high correlation between a product going GA and then the adoption rising right after. We certainly saw that with JetGPT and OpenAI tools but you also saw it with Vertex AI and then Bedrock just recently went GA. Now you got obviously advanced access to it so have you seen that similar dynamic? What are you seeing from uptake because we totally expect that Bedrock is just going to explode? Well, I think that it's a really good question. It's one of the things that all of us within the industry are looking at because this has been one of the most exciting times by the way, I've been doing this for too long in Silicon Valley and just in terms of the pace of innovation this year it's been unlike any other time so we have shipped so much stuff and it's probably, somebody has a pretty in product management one of the most satisfying times because just, you know. The appetite. Exactly, the appetite because you've got to have the pull. You've got to have the pull. So the GA designation is a powerful signal to customers that this stuff is ready for prime time at least on our end. Now what's more important of course is are the customers or the enterprises or the businesses ready for prime time with what they're doing and they're still working on it. There are a whole bunch of Vanguard folks that are the early adopters and by the way in some of the largest companies we talked to there are a few of them, particularly in a few markets, retail. We've actually seen it in healthcare, believe it or not because there's so much content generation that's necessary in there. Folks that have gone into production this year but the lines share them are going to be over the next several quarters. You're going to see what people are getting this stuff out there. Thanks for coming on. We appreciate it now with times tight. But let's follow up. Congratulations on the news and success. Final question for you for the folks watching. Why is the data store and the vectors together a better scenario than keeping them separate? What's the net net here? Well look, ultimately your large language model unless you're just building an experiment, right? Unless you're just trying to show off how cool an LLM is, which by the way is an acceptable part of the learning journey which many people are at. But once you get down to it, it's only as powerful to your business. The impact to your business is going to be dependent on it working with the data, right? So, you know, for example, one of our customers' price line that went live, they are building an AI powered travel agent. They use the data, they use the travel history, they use your personal profile in order to have an assistant that can give you a personalized vacation, right? That's all the data that's in the systems. If you go and say I'm going to have a different system for the vectors and for my core data, now you're not going to be able to bring those things into a common experience. So, we see that as a natural convergence point. Great, and strategic relationship with AWS, co-selling, co-marketing, multi-year, congratulations. Yes, yeah. Thanks for coming on. Okay, two coverage, back to the studio we'll be back with after this short break.