 Well, welcome back to SuperCloud for our episode, four of our series identifying the next generation cloud. This topic is on generative AI, how cloud technology, next gen cloud is evolving with generative AI. And guest here is Ori Goshen as the co-founder and co-CEO of AI 21 Labs. Cube alumni was on six months ago, and you know, six months, things change in AI. Ori, thank you for coming back on theCUBE for SuperCloud 4. Thank you for having me. You know, we had a really great chat six months ago on the startup showcase. You guys were highlighted with your partnership with AWS. I'll say one of the big hyperscalers in this now multi-cloud world. Your interview really resonated with folks, certainly in our audience, and we took out a bunch of highlights from that. Many were really around, what's now obvious, but six months ago, lots changed. You know, people are realizing the value of this wave, that the humans are in control, the creative classes emerging, and the infrastructure and tooling is needed. So a lot of those comments resonated. We're back at the table six months later. I have to, first question I have to ask you is, you know, with respect to the generative AI hype, there's a lot of reality matching the hype, and which you don't really see in these super hype markets where you got a lot of hype and then you get value matching pretty quickly. So much has changed in the six months. Just go back six months when you were last on theCUBE. What's changed? Yeah, a lot. And actually I think the whole, this whole space probably changed multiple times. But I think if we kind of zoom out for a second, I think, you know, nine months ago, six months ago, whether we were still at kind of sporadic exploration, people were trying to figure out what can be achieved with this technology. And there was some initial momentum behind testing and understanding the capacities and the limitations. I think this is just shifted towards massive experimentation. So there's almost no company, no enterprise on earth that is not looking in how this impacts the business and, you know, thinking at the board level. I think it really reached that kind of attention and visibility. And, you know, that drove a lot of interest and I think we're, you know, we're now still at the phase where most enterprises are still experimenting with the technology, doing some POCs, testing, and to kind of understand the capacities and the limitations. And the next six months, we'll probably see more and more production deployments, like gradually going to production and implementing some of these use cases and testing them in, you know, a scale kind of setup. And then we'll have probably another six months of enterprises understanding, well, really measuring the ROI on these use cases they've tested, like what's the impact on the business, is it really economical? And from what I'm seeing, and you know, I'm speaking here very much intuitively, I think some of the use cases will be maybe flashy and kind of really cool, but not necessarily prove themselves as in terms of ROI and others will be just traumatic ROI more and more than people have expected or anticipated before. So that's kind of the perspective I'm seeing from the last six months, kind of a year from now where we're heading. It's great, I love chatting with you, because you know, we were talking to everyone as much as possible in the same kind of trend we see just to validate your point is there's a lot of development going on and I want to get into that open source aspect of in a minute, but you're right, I think this is an important point. In every new market like this is a lot of experimentation and there's a lot of flashy demos, great, I mean, good products. I mean, if you have data, you can create a good app. The question then is what's the scale look like, right? And this comes back down to every early market. This is the same pattern. I remember with the web and some mobile apps, the demos were kick ass. And then the question was, I don't never scale. You got to be technical debt. This is pretty easy to work that down. Once you get a success point, this is the development playbook now. Okay, identify some use cases, less ambiguity the better and for the low hanging fruit. But it's okay to have a flashy demo if you got some working data, right? Then the next question is, what's the scale look like? And it's not so much tech stack. It's, or it could be cloud because we've got GPUs and need in TPUs, whatever, but it's what data is available. So take us through your thoughts on that sequence from flash in the pan demo, great demo, prove it, now produce a production ready scalable app. What does that look like? Yeah, I think there's several aspects. People are now struggling with kind of to describe these gap between a flashy demo and use case in production. And the gap, the aspects are, you know, there's the whole issue of security and how do you manage data in this regime where if you're trying to customize these models, then some of that data is becoming available through these models, actually could be extracted through this model. So how do you manage the whole security and data governance? I think this is a big theme. Another theme is costs, whatever works in a small space or small demo, what happens if you, you know, scale it deployed for your customers or for even your internal employees and, you know, it may not be economical at all. And another consideration is, or another aspect is reliability. I think this is the biggest one and hardest one to crack. And it goes back to what this technology is. I mean, these large language models are essentially, you know, they're stochastic systems. So, and they will, you know, these are statistical by nature. So they will make mistakes and they will hallucinate. So how do you overcome this challenge and how do you provide the level of transparency and the explainability and also kind of make sure that the real results and the outputs you're giving in, you know, mission critical environments are true because this could lead to, obviously, and this could have some serious consequences. So I think now that people are experimenting this technology and gradually deploying them and experimenting with actual usage, they start to face these challenges. And I think there's a path to move forward, but these are real challenges that will take some time to crack. Well, great, let's get into the conversation. Great riff there. For the folks watching, Ori is the co-founder and co-CEO of AI21 Labs. They've raised a total of $283 million, valuing the company over $1.4 billion. They came out of stealth in October, 2020 with the first AI writing tool, WordTune. They have a flagship product, AI21 Studio, pay-as-you-go developer platform for building custom text-based applications. Doing a great job. And again, six months ago, we featured them on our startup showcase with AWS, obviously one of the big clouds and super cloud players. In that conversation, you talked about a lot of things that want to shape into this conversation and this discussion. One was the role of open source. In our previous conversation, you highlighted that using existing foundation models, whether open source or proprietary, is what people are doing. How do you see the role of open source communities evolving in the development and democratization of the generative AI wave? I think open source is a super significant role in this space. I mean, more of the, I think a lot of the research and progress we're seeing right now in the accelerated phase of progress is due to the open source community. I do want to highlight that there's, there are several new ones when you speak about open source. Having the weights of the models released and open sourced and there are different types of licenses is one thing. I think if we really want to attribute full open source, you'd expect the researchers or the organizations to release the code for training and the data. So I think that's a full open source version. And there are some cases, I think there are some organizations that go to take that path of full open source, but most common practice we're seeing is actually that partial open source where they just release the weights. And the weights, you can think about these as binaries, like you can't, it's not like code. You cannot do a lot of things with it, but you can find you need, you can continue training the models and you can use them. So I'm not saying it's, I'm not saying it's unuseful, but still kind of, I think it's good to calibrate of what open source means. It means different things in this context. So, but broadly speaking, I think it had tremendous effect on the research and the pace of research we're seeing right now. And I also think in a couple of years from now, we won't be speaking about large language models. We'll be speaking about AI systems that encapsulate those and perhaps even use a portfolio of large language models. And then I think the open source we may have even a more significant role because you'd be able to, again, use these specialized models for different purposes, for different cases. And I guess that we'll see more energy and activity on the open source that will allow us to just proliferate it. So I'm a big believer in that direction. I do think there's a lot of value in the proprietary models just the way it's being trained and evaluated. And again, in the context of an AI system, I think it's really important to own that model piece or at least for some of the capabilities you want to own the model piece so you can have the performance guarantees that you need to make a reliable system. So- I think you're onto something that I want to just unpack a little bit as you brought it up. I like how you said AI system because one, it's AI in the word system. I've always been, you know, me I've been ranting about systems thinking as a skill set, you know, we've seen the evolution of iteration and design thinking, you know, cloud first. I think a data first or AI first thought process has to have a systems construct. And so, because AI is a system, it's got consequences when you make changes. It's an operating system. In a way, you're bringing up a good point about this nuance around open source and what's not open source. I mean, open source was built generationally on the old computing model of proprietary software and available hardware. Okay, I remember the days back in the PC revolution, software ran on hardware and hardware have an operating system, but they had chips. Chips were proprietary. Intel was probably the most proprietary product on the planet at that point, but no one really cared about Intel. They did great processors. In a way, we're seeing something similar with AI. If you overlay this super cloud concept around AI and say, for this thought exercise, what if open source wasn't old school? It was new school, meaning the data is open sourced and these other models can be proprietary if they're a hard and top or a system that works that makes the system work. You know what I'm saying? So, I mean, it's kind of a riff here, but I want to get your thoughts because this is a generational potential shift in what the operating model will look like for AI, which will impact the applications and ultimately the end value. Yeah, I think that there's a computational paradigm shift here because of these stochastic statistical systems that we call large language models or generative models. So I think one way to look at it is from an architectural point of view is that we have these models. We're trying to encode all the knowledge or maximum amount of knowledge inside of them and try to make the models better and more performant. I think that's one approach and it has its merits, but I think probably what we'll see if I had to estimate is actually a more decompositional approach where we won't have this one model to rule them all, we'll have a set of smaller models or combination of big ones and small ones, which is a portfolio and a very sophisticated orchestration layer that operates these models. And that orchestration layer is considering the statistical nature of the model, understanding that these could produce errors and how do we treat them, how do we deal with them, how do we minimize them would be part of the orchestration layer job. So I think this kind of describes what it means to have an AI system instead of a model. And I think you're right. I think the analogy where we will have we will have proprietary models, but open source systems could happen and it could be also a combination where we'll have open source models, proprietary models and open source systems that will be the main stream and be adopted by most of the developers. And we can also speculate that if data is the intellectual property and you talk a lot about this in the market, I've watched some of your interviews around how this AI systems can scale intellectual capabilities, knowledge workers, really a big creative class development could come out of this. And so you're seeing companies who have proprietary data, that's their own whether that's protected, that's now intellectual capital and property. So, okay, security and sharing that. So it brings up the question of integration of models. If you have a system, you obviously maybe have some proprietary and open. The fusion of a model from a finance department, maybe or sales or technology or machine log files could be part of a blended model. So the future of foundation models are really the discussion here. So the AI landscape is evolving. You're seeing specialized models and certainly enterprises will have those. So will people with a distinct advantage. You're going to have public models, open and proprietary. This is an emphasis on this power law. You got the big, fat models and then you got the more evergreen, more maybe custom, less costly, maybe smaller. The rise of smaller models is out there now. So, and then you got the integration questions. And if the cloud is API based and you brought this up in our last conversation, API is the lingua franco for the cloud. Are models going to be API based? Is there an, is our vector database is going to work together? What about embeddings? These are new questions. What's the integration and how do models work together? How do you construct an AI system if you have the benefits of all these potential models? It could be foundational or there could be NLP systems. Right. Yeah, I think it's a great architectural question. This is actually one of the areas we invest and we spend a lot of our research. And it fundamentally goes to why we started the company and the fundamental reason was that we believe in a kind of a neurosymbolic hybrid mode where some of the computational jobs are should be done symbolically. I mean, there's no need to kind of conduct operational, logical operations and neuraly, right? That it's not the neural strength. And you would, there are certain things you wouldn't want to apply strict rules. It's too brittle. So the question is how do you, how you married the two and how you mixed both of them. And that kind of, it also goes back to your integration question. How do you, how do we mix and match? How do you use different models for different purposes? And I think we'll see some really cool progress on that front in the next couple of months. So stay tuned. Is that on your side or just generally in the industry or both? I guess both. Okay, all right, cool. You know what I'm sharing. We're going to get the secrets out there too early. We'll definitely be following up on that. I definitely want to do more about this AI system. I think that is really a great vision. And I think that's the reality. We're going to move to a new operating model. It's very clear to the super cloud conversations we're having. And it's going to be completely different in our opinion. We don't yet know what it looks like, but it's going to just evolve. And once it pops into interview, everyone's going to jump on that bandwagon. The question I want to ask you that's kind of on everyone's mind is the infrastructure and scalability side. And you talked about it a little bit earlier in this conversation around scale. You know, everyone wants to know how much it's going to cost. But the bigger question is, as the apps start growing, what's the big knobs to turn for managing scale, not just from a cost perspective, just from just growth. GPUs are obviously the hot conversation. Google's got TPUs. You know, hardware and the silicon is going to be very important. What's going to be the most and crucial piece of infrastructure? You know, what advances or changes do you anticipate being crucial for sustaining the AI boom? Sure, I think maybe a couple of trends. So first you will have, so what we see in, you know, in like, I don't know, 99% of enterprise use cases are not about open-ended, you know, general purpose chat systems. It's not about that. It's about fairly narrow types of tasks that are interested, that the enterprise is interested in. And it's also typically deeply integrated within an application or a workflow that the enterprise, that the enterprise is interested in. And if that's the mode of operation, then I think, again, instead of going to, going after the large, very large models that are capable of doing many, many things and are extremely flexible, I think a more pragmatic, practical approach would be to have these specialized systems, right? That are really good at particular tasks. And taking that approach would allow you to, A, you know, run this on much more compact on basically smaller chips that are much cheaper to serve. And B, some of these chips are actually far more available than the, you know, large systems, the large and fairly new systems. So that, I think will create the advantage of both having the lower costs and the higher availability. And that's something that is definitely something a lot of companies face right now, the lack of availability of compute resources. So that's why I think it's this architectural decision is really important for getting your application or the idea from a small scale to a large scale. Great point. And that, you know, in this market, where there's demand and there's a growing market, the entrepreneurs will be creative and the big provider will figure it out. That's called innovation. Great point. The question I want to get into now is you've mentioned the human piece of it before, human loop, gonna be humans in the loop. And I think that resonated well. Also, a point you made in our last conversation I want to get your thoughts on is you mentioned that every industry is going to be disrupted from industrial to medical, every vertical is going to have, see, generally that's clearly the case. Almost every event we go to, it's like the same line. Every industry is going to be disrupted. Okay, great. You were right. Can you provide some examples or predictions of how Genevieve AI might revolutionize these non-tech sectors? Because I think what's happening is the non-techie industries are leaning in heavily. And even some regulated industries, because, by the way, regulated industries have all that labeling done because of compliance. So it's kind of like interesting point, right? You're right. The ambiguity is not really there. So it really doesn't really care. There's not going to hallucinate if it's got some policy and some fixed data. So we're seeing regulated industries leaning in more. So what's your thoughts? You're in the middle of it. What examples in the non-tech areas? What are some revolutionary things happening? Yeah, I actually think that the more traditional industries will benefit and will actually be surprising, but they'll be the ones who are massively adopting it first and yeah, just maybe to give you a couple of examples. So think about the pharmaceutical industry. We've been working with a few players on drug discovery use cases where using these language systems, now a company like a pharmaceutical company and really accelerate the drug discovery process because it typically has research of millions of documents, studies and things that were put for years. And when you use quite primitive search tools yes, you can extract info and find clues for interesting connections, but if you have a very strong generative system that's sort of retrieval based, then suddenly you can gain new insight, things, connections you haven't imagined like as a human being you haven't thought of before. So you suddenly start to make new discoveries and I think that's, it sounds small but that types of new insights and you can think about it also as a type of and hypothesis generator. This would totally disrupt the pharmaceutical industry. So I think it is just has a big impact and regulated industries, I think there will be impacted as well pretty extensively just because now they have so many cumbersome processes in place, like to make a decision or make an action, you sometimes need an army of people to make sure all the constraints are satisfied and you're working by the rules and everything but if you can accelerate that, if you can have machine augmenting humans and providing them the information that they need quicker and help them with some of the reasoning, I'm not saying it's going to give you the bottom line the end conclusion, but I think some of the reasoning could be offloaded to these types of systems. So I think this will enable these types of regulated organizations and it will enable their employees to focus on their jobs and have the assistive system to help them with figuring out all sorts of regulation issues. So I think both of these examples kind of show how this is actually going to impact almost any industry we know of. It's really interesting and some of these industries like pharmaceutical healthcare, where you have these experts at the top of the pyramid and skill set, you know, and is always had to be, you know, and were they almost like savant brilliant, right? And they were masters at their trade or grand masters in chess, for instance, in chess it's well documented. You know, D grand master in chess, there's only a very few people that were at that level, but when you introduce computers, humans plus computers, the number of grand masters increased, okay? And because the humans had augmentation with the machines and you add AI and it's well documented. Believe me, the chess world is highly docked vocal about the relationship between computers, man versus a computer, computer versus computer, humans and computers versus humans and humans plus computers plus. So if the trend is clear, humans at the top will be more knowledgeable. So in chess, the grand master in the league category, the computer AI assistants made someone who was almost a grand master or grand master because it gave him extra capabilities to think. So it created- It's like a thought partner. Yeah, it's a partner. And so this intellectual scaling step function is happening with AI. I want to get your thoughts on, you've talked about this, you've hinted on the last time you're on theCUBE, this idea of AI can scale intellect, which is data, which is in people's heads as well. So what's your thoughts on this? Yeah, I think, I mean, thinking about knowledge worker broadly, like every, you know, knowledge worker in every industry. So our jobs could be reduced to be described in a very kind of compact way is that we're consuming information. We're producing or synthesizing insights and during that process, we're conducting reasoning. And that's the main job that we're actually doing. So I think in all of these three parts that I mentioned, the AI assistants will just provide a very dramatic boost. Like from consuming information, this would be much more faster and productive than it's been done today. Like finding the right information and even doing compositional stuff. Like, you know, I want to compare between two different things and tell me all about it. And then, you know, when you click on a button, we'll get it. And then the reasoning piece is also interesting because we can start offloading problems to the system, not just of a way of surfacing relevant information, but we can actually combine the relevant information with problems we're facing and we'll use these systems to help us solve these problems or at least suggest a way to solve these problems. And then when thinking about writing and communication, of course these systems are really good at drafting and generating and ideating and all these types of capabilities will, again, augment us. So I think that, you know, day-to-day jobs, this will have a tremendous effect on our working environment. And may free, maybe I think that's one of the predictions I heard that it may help us free a lot of time. I kind of think everybody from a competitive standpoint it may not, but at least we as humans will devote much of our thinking and energies towards the things we're really good at. Yeah, it'll make our minds, it'll help us have more free time to think and do things either creatively. I mean, no one really sits on the beach all day. I mean, people, we're human, we do things. So I mean, basically what you're saying, I mean, so maybe next time when we talk, I'll plug the two transcripts from this interview into AI 21 lab studios and I'll ask it to do the interview for me. What do you think about that one? Are we there yet? Yeah. I actually think we're not that far, I think, but now with the question is what type of, you know, results are you expecting? At the end of the day, you may end up with pretty boring interviews. I'll throw out an architectural diagram of the AI system we just rift on, including which best vector database to use in the system. See what comes up. Yeah, that'll be interesting. Well, Ari, I really appreciate you. We're good rift, good keynote conversation here around SuperCloud 4. I think it was important we hit those architectural things. In the last minute we have left, put a plug in for what you guys are doing right now. What's the coolest thing you're working on? How do people engage with you guys and become a customer? What are some of the onboarding best practices? How do I jump in? If someone wants to jump in and use AI 21 lab studio, what's the playbook? Put a commercial in, put a plug. Yeah, so ai21.com, you go and you sign up for our studio platform. And one of the things we're making available there is our task specific systems that are optimized for specific tasks like summarization and grounded question answering and generating different types of content. And we have a solution architect team that works together with enterprises and it has a six weeks program where we build a solution together and provide deployed POC that people can actually touch and experiment and extract value. And then we take customers and go to production and again in the matter of weeks, not months. So we're very excited about this revolution and our mission currently is to take enterprises and help them in this journey and make them successful and gain value from the revolution we're experiencing. Ari, thank you so much. Ari Ghoshan, co-founder, co-CEO of AI 21 Labs here on SuperCloud General. Thanks for joining us, I appreciate your time. Thank you very much. Okay, we'll be right back for more SuperCloud 4 coverage after this short break.