 Welcome back everyone, the Cube's live coverage in Las Vegas, I'm John Furrier, host of theCUBE with Dave Vellante, we're here with SAS, at SAS Innovate 2024, we're back with two great guests, Brian Harris, Chief Technology Officer of SAS, Marinella Profi, Global Product Marketing Strategy for AI and Genet, both stage performers today on the keynote, Brian, Marilyn, thanks for coming on theCUBE again, great to see you. Thanks for having us. Great to be here. So, we were just at your event last year, we said, you said some things, put some things out there, this is what we're going to do with AI, Genet of AI, it was all the hot rage, chat GPT, really innovates on the user interface, check, we see that. Getting things done is a whole nother ballgame, you guys got some huge announcements, the models, I love it, let's just get into what's going on right now, what's the key things for your transformative technology around Genet AI? Let's just get to Genet of AI out of the gate, because everyone's talking about it, Huffin and Puffin, which is again, it's a really exciting technology, but it has to be put in context to actual business outcomes, and so, I think for me, I'd like to actually give it over to Marinella a little bit, because she's a day-to-day data scientist out there, and on the ground, meeting with customers, hearing the feedback, and so Marinella, why don't you talk about, why don't you talk about what you're seeing on the ground? Absolutely, thanks, Brian, yeah, so from my perspective, I'm a data scientist, and I think the number one lesson that I've learned, and there is a lesson that I've learned, well, the past year and a half has been, LLMs, Larging with Models alone do not solve business problems, like, this technology is so cool, and no matter how cool it is, it's still imperfect, and it's all going to come down to solving real-world business problems for specific industries and specific people, right? We're seeing a lot of companies just saying, we have this technology, let's go and build it, let's see, here's how we're going to do it, so a lot of it's around how do we build something, but that there is very little around why, why are we even building it, who's going to serve, but what's the value that's going to deliver, right? And so, for me, from my perspective, what I've been seeing is that, you know, Genevry.TVI and LLMs in this case are like the 20% of the entire value chain, if you consider a real-world use case, and I think, you know, what Brian shared today with Georgia Pacific was an incredible example of that, I don't know if you want to add more. Yeah, definitely, I mean, it creates some point, right, that, you know, Georgia Pacific is on stage talking about their use of a SAS Viya as part of our Genevry.TVI workflow, I mean, there is a lot of moving parts, and there is no more kind of validated feedback on a production use case at Genevry when they're actually trying to run their plant, like their plant operations is leveraging Genevry.TVI, and as part of that, they're getting corrective actions through this Genevry.TVI experience that is leveraging SAS Viya to orchestrate across LLMs, orchestrate across quantitative reasoning needs where we're interacting with enterprise systems, so all of this orchestration is happening ultimately to give a human language response back to an operator who is three, five years' experience in their job, and they're trying to convert them and upscale them to a seven, you know, five to seven year employee. There were a lot of components to that stack. Yes. And you think you get a picture of it here. You had Bedrock on there, you had SageMaker, you had SAS, you had another couple other pieces, and so you see Genevry.TVI as the orchestrator. Yeah, for us, at some point, you got to integrate all of the tech, and what you're really getting to is you're making a final decision to give a response back to a user. And as we know, a lot of these cases right now are very shallow. They're just prompt engineering to an LLM and come back. And yes, RAG architectures are there, but we got to get things done in sub-second response time here. So there's very, very challenging integration and expectations of responses from Genevry.TVI workflows that we have to get done. So for us, Viya sits at the center of that, and even Sam Coyne mentioned it today, that we really bring all of these things together. We can augment prompts, we can orchestrate against other traditional models and things like that and bring that data back in so that the LLM can reason over that and send back the response back to the user. An answer to your question, I think Viya is orchestrator here, not Genevry.TVI. Genevry.TVI is a piece that can augment existing processes and Viya, our data and AI platform functions as that orchestration of the entire positioning flow and that's exactly what we were announcing today. We've extended Viya to allow developers to use our API to govern, orchestrate, and build real world use cases with Genevry. And I think the point about Genevry.TVI is an element of the stack, the higher end of the stack. I have a picture, I want to get your reaction. Okay, you built on AWS, so got some hardware, got some cloud. There you go. Sass and intelligent decisioning, that's your layer, right? That's the- Decisioning system. Decisioning systems and the multiple, real time data knowledge bank process model. Okay, that's the meat and potatoes, that's the stuff that's the key jewels to make that thing work. And then you got the G&A on top, managing it, and then you got the outcomes. Yeah, and so like, yeah, so do you think of it as that something has to bring all these things together because right now, a lot of the reasoning capabilities in text for LLM is around, you know, textual fact-based reason responses. But a lot of the decisions in the enterprise are quantitative. So that's why we pride ourselves on being in the quantitative reasoning engine inside of Genevry.TVI workflows. And intelligent decisioning is such a flagship product for us because it allows us to, we can bring four or five different models to actually integrate across text, across the LLM itself, across the actual enterprise systems we're integrating with. I want to connect it to something you said in the media briefing, if I could, which is about ROI. I mean, if you look at how people are using Genevry today, it's very chat GPT-ish. Yes. You know, so it's not literally blowing away the cash flows. I mean, it's not really paying for itself. And so when you start talking about these industry-specific models, that's where you're going to get return. And people are pushing out their ROI expectations based on the surveys that we have, right? And so, you know, they're being a little bit more conservative because they're saying, okay, this is fun, this is cool, but we need real dollars. So when you open up a prompt to thousands of people in your organization, the quality of your questions is now your biggest concern, right? And now people are paying for this. So now, okay, well, that's why you saw us talking about stored prompts because our ability to say, all right, look, there's not an infinite set of questions like in the public domain with like chat GPT. In an enterprise, there's a finite set of questions. So why don't we understand those questions you want to ask, make an inventory of those, use Viya to kind of do the semantic analysis on this and then say, when three, four people ask a question, we're going to get the question that the right question you're asking. And then let that then drive, reduce the error to the LLM, reduce the error to the vector database, reduce the error to the quantitative reasoning with Viya as well. I thought that was a very nuanced point. I thought one of the things that jumped out at me was the stored prompts because Dr. Goodright was actually bragging about the compiler versus the interpreter on stage. If you think about that, as you get smarter with these stored prompts, you can learn, and that's like compiling away the LLM risk. What's funny about it is if you remember, what did database providers want to do? They wanted stored procedures. And stored procedures was their way to govern the way to build optimized queries for databases. So stored prompts are really just an extension of that entire model. People are doing this in all kinds of use cases to get smarter. You make a prompt, you get an answer, you come back, it's back down to the architecture, why I like that graph and why I called it out was, that's a production workload. The only thing that's outside there is the customer's environment in Amazon and your SaaS stuff. So that brings up the kind of the big picture here. You got training and inferences, the big discussion on how to deal with data, and then you got prompt response, and then you got reasoning in the AI paradigm, all contextual and behavioral. So you're essentially rolling all that up. So, okay, prompt response, that could be real like IO. Reasoning could take a little longer maybe, or so I got to run that somewhere. How do you see that applying to known end-to-end workloads? Because this is the theme that's popping into this AI world and it rises. I got known workloads. I can scope them, I can qualify them. I think the way it rolls up is that we're seeing the, definitely the agent design pattern is emerging. In fact, in our Viacopilot demo that Maranella was showing today, we're actually doing some iterative work where we have one generating the code, right? One part of the orchestration is generate the code underneath it, execute the code, if it fails, tell the L, if it failed, it'll make the corrective action on it and execute it again. And then we'll get the actual- No human. And then it's why you saw like a map on the screen or you saw some of the extra follow-on visuals or suggestions. So that's why this orchestration is so important because it really is an iterative process. It's not a single request, single response experience. And so for us, this reasoning thing, if you think of reasoning, reasoning has to span, you know, text-based, fact-based things, media, like images and video. And, but most importantly, in the enterprise, quantitative reasoning is really the majority of the workload challenge. It's not about just, you know, I mean, customer service, yes, I got a knowledge base, I can give you answers on it. But a lot of the enterprise-class problems are, how do I look at numbers, analyze numbers, and make a recommendation on the next best action? And that's why I asked that dumb question in the media thing. That was not a dumb question, by the way. That wasn't a dumb question. Well, I was trying to get it. I came out wrong. I've seen a lot dumber questions in there. I'm usually going to go for dumb questions. But the point is, customers today have to go to different models for different models. So they want one place. They want to have the reasoning and all this. It sounds complicated, but put it in one place. You can do vector embeds for unstructured data. I can bring a table in. I can bring in a DNA sequence. That's the mode, right? I mean, all kinds of data can be structured that way if it's in one place. I don't want to go five different places. And then do you have the technology, I think, that you can talk about, why don't you give the example of the banking one we talked about today? Correct, yeah. Where you can really talk about all the tasks that were required to really make it work. Yeah, so this is one example that we were working with a global bank where they had, they were doing complaints management, it's a simple use case. Like it's not revolutionary, but like the question was, how can I augment what I'm doing with generative AI with large language models? And the integration of the LLMs is not just easy as, oh, give child GPT the complaint and generate a summary. And then based on that, it'll ask to generate a reply to the customer and send that reply to the customer. Like that's not how it actually would work because you have so many other pieces in the middle. And in this case, the flow that the developers were able to be able to actually productionalize that use case involved other things like, okay, the summary is generated by the LLM, but how do you explain if that's accurate? How do you explain how you generate that summary? And so that's where natural language processing techniques, which is actually a method that has been proving to be really critical and fundamental in explaining some of these, the output generated by large language models. That's, and then how do you integrate your customer data within that summary? The LLM doesn't know if your customer has a high credit score, is a good payer, is not a good payer, like the credit, those are your enterprise data. The LLM, and unless you want to give that to the LLM and expose them to the LLM, which I don't think that's what our customers are asking to, you need that quantitative decision engine that augments that information and sends back that information for the LLM to actually generate that summary and to generate that email in a more reasoned and a way that makes sense. I mean, if you think about it, we compress all the complexity of the quantitative side into a much more consumable fact that could be presented back to the LLM so we can use the strength which is to generate natural language response. Which, so that might be a score, for example? Yeah, yeah. And then the LLM will take that score as opposed to trying to generate the score, which it's not good at. It's actually horrible at that. Exactly, that's horrible. You said it well, right? I would say it exactly that way, it's terrible. Dave, he's a SaaS user in our, he used to program in SaaS and that wasn't bad. We appreciate it. We appreciate it. I was born in that year. For those about the proc, we salute you. Yeah. And if your customer is complaining about your credit, your service, the LLM is not going to be able to generate the next back action for your specific customer because that's to know your customer, doesn't know what the next back action is. Since you're a customer, I want to get more use case information of what you guys see in the marketplace. As Dave and I always talk when Dave does some research on the monetization, that's really going to be the scoreboard. Are you, is it adopted in production when you guys are showing some examples of greater than others? And the monetization of AI, you see the early adopters, some are monetizing really well, so that's also a tell sign. Monetization, uptake, and then customer success. So, you mentioned you're doing well with the business performance. And you spent a billion dollars of investment and spending more on the tech. So you got tech investment and monetization. Good job. What do the unit economics look like? What's the use cases that are emergent? You can point to it and say, we're starting to see the things line up in the market, the fogs lifting in the enterprise. What is it out there and how is it being monetized? Beyond the chit-chat, right? Yeah, yeah, yeah, I'll get it there. I mean, I would say that we see a lot of customers worried about yet another metered challenge to the business. I mean, there's CFOs saying, look, we're trying to run a profitable business. And look, if you can tolerate metered billing when you know you've got enough room on the financials to tolerate it. But when you start to, remember, we're lowering the barrier to interaction for data and AI across the entire enterprise. When you do that, you're actually encouraging usage of these metered services, which is fine. But you have to make sure that the use cases are driving an ROI that can support those metered interactions. And then we have other customers are saying, can you take me off the meters? Can we just run? It's my data. I want to bulk upload my own data. Yeah, yeah, exactly. So we want to respect all of those, right? Many customers, we're partnered with Microsoft. We want to leverage their open AI services where possible. We're working with Bedrock and Amazon and AWS will do that. But there are customers who are like, hey, I want to run with Mistral. Like I want to go and I want to run that in a GPU that's hosted in their environment. So the same cloud spend conversation, it was really... They want optionality, but they want to have confidence on what the spend's going to look like. Yeah, absolutely. Have some scope. Yes, because what's happening is everyone, again, the FOMO that was happening about, just go run to this thing, is now being realized in the financial outcomes of these businesses. These are big customers. I mean, can they get GPUs? I mean, if you're not spending 10 million bucks on video, you have to get it. I mean, I think when you're training models, it's one thing, right? If you're just looking to go and do the, hosting them from scoring these models, I think there's options there, right? There's a lot of things coming down in size on it. Now we're just going to make sure that our stuff can react to any one of these scenarios. Like if you want to use a cloud service provider, we're happy to do it. Some of the customers just want, that's what they want to do. So we'll leverage it. But we're seeing others now starting to say, let's bring it locally. Is it tracking like DevOps did? Do you get a use case and you experiment with something? Or is it end-to-end workflows that are emerging? What is some of that? Or is it the data that's on? I think first and foremost, I think there's a little bit of a hammer and nail problem where everyone's saying, let's start with generative AI. When that really, generative AI is a subset of AI. It's not a superset of AI, which is a big problem because it's been presented in the market as if generative AI is this new thing. It's actually a subset of the larger AI umbrella. In fact, the problems being solved by the rest of the AI space are much larger than the problems being solved by the LLM and generative AI space. And some of the best models are small. Yeah, yeah, exactly. I think if that's going to be the future, where the smaller language models are, yeah, smaller language models, that would say almost auto fine-tune themself for specific industries and specific use cases are going to be the future. And that's because they're more accurate and faster? I think it's like anything else. If you go to the public, the large language models are out in the public are just, they're kind of generalists in these things. And they're powerful for many reasons. But they get vanilla. Right, exactly. If you want to use an example here, and I mean, Google was trying to, remember when Google was so huge, they wanted to get into the enterprise space, right? How could we put that Google engine? People were going home and they were having a Google experience for search and they would go into the business and say, why can't I have that same search experience in my business? The use case was different, right? Google's indexing engine was about finding information authorities. We, while in the business, it was about quantitative analysis. So they never could really penetrate that market. We're having the same challenge now with generative AI, which is that you can have a very public domain question and answer experience with LMS that are very powerful. And you want that capability on the enterprise, but there's more work to be done. And what people are realizing now is how much more work there has to be done. Especially for us, we're in regulated industries. So we can't be wrong. So I can't just go and run. Yeah, I can't wing it. Yeah, I can't wing it. We've got to go and work meticulously with them to understand here's what happens when you do this response. And if that response gives an answer and you need proof of it. Yeah, there's no space for hype. We need lineage. You need all the things that go behind it. No space for hype for us. And what I would add is also that, I mean, my question is, until what point this gigantic large language models providers are actually realistically going to have data to keep training these huge models? Yeah. Yeah, they're going to try it. Meaning they're going to run out of data. They are. It's called strip mining. I think so. And either it's because it's going to become a legal issue or a point where you can't take my data anymore to try and whatever you want to train. Or there's just no more data. So we have to go back to synthetic data. Can you believe we're saying that? I know. We're running out of data. Adobe is charging $7, I think, a minute for the video, right? Yeah, yeah, yeah. And that's why a data maker for us is a big announcement. Correct. So that's data maker for us. It allows us to attack the structure, the gap in structured data for synthetic data. And by being able to do that, we can go into regulated spaces and give proof to the statistical congruence of synthetic data that can bootstrap models that will allow them to run effectively on real-world data. You got to have explainable AI as table stakes and security built-in. Great to have you guys on. We're out of time. Final question. They got the hard stop. Oh, that moment. I was enjoying this. That's what I got. I'll get one more question in. Good. You got one more. Maybe two. So next, great step up from last event. Thank you very much for bringing all the goods to the table. Next year, when we're sitting here, what are we going to be talking about? What's going to be the biggest discussion with Brian? We'll share your thoughts with Brian. Yeah, I think that what we're about to achieve with that factory, right, we're still, we're building our own solutions with it, but the productivity of that thing is incredible. When we start wrapping some generative AI experiences on building applications that are, the treatment on that's going to be incredible. I think what you're going to see on our Viacopilots work that we've just done and demonstrated today, I mean, the ability for our Viacopilots to be a very practical approach to the AI lifecycle is, I think it's going to be world-class and second to none. That's where I think I'm right, we're at. What I am really excited about is to continue. So the strategy that we have taken over the past two years is to have purpose, like building something with a purpose, right? And I am really excited to come and see that come in action. Like we're not just a startup that is going around what looking for funds from a VC. We're building these things for actually being used in the world and creating real world value. And I can't wait for that to continue to be and have even more, like I want to come here next year and talk about other 10 customers that we've done and how all the models that we're going to sell. And so that to me, what I'm really excited about. That billion dollars in investment, you're monetizing well, which is proof. And you got production workloads. I mean, just keep it coming. Yes, and I would say too we will have a significant amount of progress on penetrating the market with selling our models, which we're really excited about. I got to ask one more question. We got time. Ask for forgiveness for you. You said in the media briefing, nobody's looking at the impact of AI the way SaaS is. And today it's very clear you're talking about very focused on solving a lot of business problems. How far out are you looking? I mean, What do you mean by that? In terms of the impact of AI, AGI. If we have AGI by the end of the decade, I mean, are you thinking that far out? And then if you think about exponential growth of computational power, mid-century. I mean, this, when you make that statement, I have to ask you like literally how far out are you thinking? Is it more? I think we'll have tools that we'll be able to evaluate. We'll have capabilities that can evaluate the reality of these models. I mean, some of these models are self-fulfilling validations where it's like there's no real, I mean, they're not deterministic, right? So there's like certain kind of workloads we put against them. We try to measure them on grading these things, but they're not measured on the enterprise's actual workloads. So how do we think about what does the answer look like in an enterprise? And how did that model react to it? And then being able to go and say that work, that didn't work, starts bringing some true in-customer validation endpoints to what are we doing with generative AI? So, I mean, I think that's the area where you'll start seeing what these things will have to be validated independently of the model creators. Well, the stars of the show, you're at high demand, you got to go, we're getting the hook, you got to leave it there. Okay, we're going to continue the conversation here on theCUBE, we'll be right back after this short break. I'm John Furrier, Dave Vellante, we'll be right back.