 Hello, and welcome back to theCUBE. We're here in Palo Alto, California. I'm your host, John Furrier. We have a special guest here in the studio. As part of our Cloud Native SecurityCon coverage, we had the opportunity to bring in John Turro, who is the partner at Madrona Venture Partners, formerly with AWS, to talk about machine learning, foundational models, and how the future of AI is going to be impacted by some of the innovation around what's going on in the industry. ChatGPT has taken the world by storm, the million downloads, fastest to the million downloads there before summer saying it's just a gimmick, others saying it's a game changer. John's here to break it down, and great to have you on. Thanks for coming in. Thanks, John. Glad to be here. Thanks for coming on. So first of all, I'm glad you're here. First of all, because two things. One, you were formerly with AWS, got a lot of experience running projects at AWS. Now a partner at Madrona, great firm, doing great deals, and they had this future at modern application kind of thesis. Now you're putting out some content recently around foundational models. You're deep into computer vision. You were the IoT general manager at AWS, among other things, Greengrass. So you know a lot about data. You know a lot about some of this automation, some of the edge stuff. You've been in the middle of all these kind of areas that now seem to be the next wave coming. So I wanted to ask you what your thoughts are of how the machine learning and this new automation wave is coming in. This AI tools are coming out. Is it a platform? Is it going to be smarter? What feeds AI? What's your take on this whole foundational, big movement into AI? What's your general reaction to all this? So thanks John again for having me here. Really excited to talk about these things. AI has been coming for a long time. It's been kind of the next big thing, always just over the horizon for quite some time. And we've seen really compelling applications in generations before and until now. Amazon and AWS have introduced a lot of them. My firm Madrona Adventure Group has invested in some of those early players as well. But what we're seeing now is something categorically different. That's really exciting and feels like a durable change. I can try and explain what that is. We have these really large models that are useful in a general way. They can be applied to a lot of different tasks beyond the specific task that the designers envisioned. That makes them more flexible. That makes them more useful for building applications than what we've seen before. And so that, we can talk about the depth of it, but in a nutshell, that's why I think people are really excited. And I think one of the things that you wrote about that jumped out at me is that this seems to be this moment where there's been multiple decades of nerds and computer scientists and programmers and data thinkers around waiting for AI to blossom. And it's like, they're scratching that itch every year is going to be. And it's like, the bottleneck's always been compute power. And we've seen other areas, genome sequencing, all kinds of high computation things were required, high performance computing. But now there's no real bottleneck to compute. You got cloud. And so you start to see the emergence of a massive acceleration of where AI's been and where it needs to be go now. It's almost like it's got to reboot. It's almost a renaissance in the AI community with a whole nother macro environmental things happening, cloud, younger generation, applications proliferate from mobile to cloud native. It's the perfect storm for this kind of moment to switch over. Am I over reading that? Is that right? You're right. And it's been cooking for a cycle or two. And let me try and explain why that is. We have cloud and AWS launch in whatever it was 2006 and offered more compute to more people than really was possible before. Initially, that was about taking existing applications and running them more easily in a bigger scale. But in that period of time, what's also become possible is new kinds of computation that really weren't practical or even possible without that vast amount of compute. And so one result that came with that is something called the Transformer AI Model Architecture and Google came out with that, published a paper in 2017. And what that says is with a Transformer model, you can actually train an arbitrarily large amount of data into a model and see what happens. That's what Google demonstrated in 2017. The what happens is the really exciting part because when you do that, what you start to see, when models exceed a certain size that we had never really seen before, all of a sudden they get what we call emerging capabilities of complex reasoning and reasoning outside of domain and reasoning with data. The kinds of things that people describe as spooky when they play with something like chat GPT, that's the underlying term. We don't, as an industry, quite know why it happens or how it happens, but we can measure that it does. So cloud enables new kinds of math and science. New kinds of math and science allow new kinds of experimentation. And that experimentation has led to this new generation of models. So one of the debates we had on theCUBE at our super cloud event last month was what's the barriers to entry of say open AI, for instance? I'll see, and I weighed in aggressively and said the barriers for getting into cloud are high because of all the CAPX and how we shoe formerly VM where now at ZScale he's an AI machine learning guy. He was like, well, you can spend $100 million and replicate it. I saw a quote that said for 180,000 I can get this other package. What's the barriers to entry? Is chat GPT or open AI? Does it have sustainability? Is it easy to get into? What is the market like for AI? Because a lot of entrepreneurs are jumping in. I mean, I just read a story today. San Francisco's got more inbound migration because of the AI action happening. Seattle's booming. Boston with MIT's been working on neural networks for generations. That's what we found the answer. Get off the neural network bus and jump on the AI bus. So there's total excitement for this. People are enthusiastic around this area. You can think of an iPhone versus Android tension that's happening today. There's in the iPhone world, there are proprietary models from open AI who you might consider as the leader. There's Cohere, there's AL21, there's Anthropic. Google's gonna have their own and a few others. These are proprietary models that developers can build on top of, get started really quickly. They're measured to have the highest accuracy and the highest performance today. That's the proprietary side. On the other side, there is an open source part of the world. These are a proliferation of model architectures that developers and practitioners can take off the shelf and train themselves. Typically found in hugging face. What people seem to think is that the accuracy and performance of the open source models is something like 18 to 20 months behind the accuracy and performance of the proprietary models. But on the other hand, there's infinite flexibility for teams that are capable enough. So you're gonna see teams choose sides based on whether they want speed or flexibility. That's interesting and that brings up a point I was talking to a startup and the debate was do you abstract away from the hardware and be software defined or software led on the AI side and let the hardware side just extremely accelerate on its own because it's flywheel. So again, back to proprietary. That's with hardware kind of bundled in bolted on. Is it accelerated or is it bolted on? Is it part of it? So to me, I think that the big struggle and understanding this is that which one will end up being right? I mean, is it a Betamax versus VHS kind of thing going on or iPhone, Android? I mean, iPhone makes a lot of sense, but if you're Apple, but is there an Apple moment in the machine learning? In proprietary models, there does seem to be a jump ball that there's gonna be a virtuous flywheel that emerges that for example, all this excitement about chat GPT what's really exciting about it is it's really easy to use it's the technology isn't so different from what we've seen before even from open AI. You mentioned a million users and in a short period of time all providing training data for open AI that makes their underlying models their next generation even better. So it's not unreasonable to guess that there's gonna be power laws that emerge on the proprietary side. What I think history has shown is that iPhone, Android, Windows, Linux there seems to be gravity towards this yin and yang and my guess and what other people seem to think is gonna be the case is that we're gonna continue to see these two poles of AI. So let's get into the relationship with data because I've been immersing myself with chat GPT fascinated by the ease of use, yes but also the fidelity of how you query it and I felt like when I was doing writing SQL back in the 80s and 90s where SQL was emerging you had to be really a guru at the SQL to get the answers you wanted. It seems like the querying into chat GPT is a good thing if you know how to talk to it labeling whether your input is and it does a great job if you feed it right. If you ask a generic question like Google it's like a Google search, it gives you great format sounds credible but the facts are kind of wrong. That's right. That's where general consensus is coming on. So okay, what does that mean? That means people are on one hand saying ah, it's bullshit because it's wrong but I look I'm like, well that's compelling because if you feed it the right data so now we're in the data modeling here. So the role of data to be critical is there a data operating system emerging because if this thing continues to go the way it's going you can almost imagine as you and look at companies to invest in who's going to be right on this what's going to scale, what's sustainable what could build a durable company it might not look like what people think it is. I mean, I remember when Google started everyone thought it was the worst search engine because it wasn't a portal but it was the best organic search on the planet that became successful. So I'm trying to figure out like okay how do you read this? How do you read the tea leaves? Yeah, so there are a few different ways that companies can differentiate themselves. Teams with galactic capabilities to take an open source model and then change the architecture and retrain and go down to the silicon. They can do things that might not have been possible for other teams to do. There's a company that we're proud to be investors in called Runway ML that provides video accelerated, sorry AI accelerated video editing capabilities that were used in everything everywhere all at once and some others. In order to build Runway ML they needed a vision of what the future was going to look like and they needed to make deep contributions to the science that was going to enable all of that. But not every team has those capabilities maybe nor should they. So as far as how other teams are going to differentiate there's a couple of things that they can do. One is called prompt engineering where they sort of shape on behalf of their own users exactly how the prompts get fed to the underlying model. It's not clear whether that's going to be a durable problem or whether like Google we consumers are going to start to get more intuitive about this that's one. The second is what's called information retrieval. How can I get information about the world outside information from a database or a data store or whatever service into these models so they can reason about them. And the third is this is going to sound funny but attribution just like you would do in a news report or an academic paper if you can state where your facts are coming from the downstream consumer or the human being who has to use that information actually is going to be able to make better sense of it and rely better on it. So that's prompt engineering that's retrieval and that's attribution. So that's that brings me to my next point I want to dig in on is the foundational model stack that you've published. And I'll start by saying that with chat GPT on the if you take out the naysayers who were like throwing cold water on it about being a gimmick or whatever and then you got the other side I would call the alpha nerds who are like they can see wow this is amazing this is truly next gen this isn't yesterday's chat bot nonsense they're like they're all over it is that everybody's using it right now in every vertical I heard someone using it for security logs, I heard a data center hardware vendor using it for pushing out app sec review updates. I mean I've heard corner cases we're using it for the cube to put our metadata in so there's a horizontal use case of value so to me that tells me there's a market there so when you have horizontal scalability in the use case you're going to have a stack so you publish the stack and it has an application at the top applications like Jasper out there you're seeing chat GPT but you go after the bottom you got Silicon Cloud, foundational model operations the foundational models themselves tooling sources actions where'd you get this from how'd you put this together did you just work backwards from the startups or was there a thesis behind this could you share your thoughts behind this this foundational model stack? Sure so well I'm a recovering product manager and my job that I think about as a product manager is who is my customer and what problem he wants to solve and so to put myself in the mindset of an application developer and a founder who is actually my customer as a partner at Madrona I think about what technology and resources does she need to be really powerful to be able to take a brilliant idea and actually bring that to life and if you spend time with that community which I do and I've met with hundreds of founders now who are trying to do exactly this you can see that this stack is emerging in fact we first drew it not in January 2023 but October 2022 and if you look at the difference between the October 22 and January 23 stacks you're gonna see that holes in the stack that we identified in October around tooling and around foundation model ops and the rest are organically starting to get filled because of how much demand for the developers at the top of the stack if you look at the young generation coming out and even some of the analysts I was just reading an analyst report on who's following the whole data stacks area you know data bricks, snowflakes a variety of analytics, real-time AI data's hot there's a lot of engineers coming out that we either data scientists or I would call data platform engineering folks are becoming very key resources in this area what's the skill set emerging and what's the mindset of that entrepreneur that sees the opportunity how does these startups come together is there a pattern in the formation is there a pattern in the competency or proficiency around the talent behind these ventures yes I would say there's kind of two groups the first is the first is a very distinct pattern John for the past 10 years or a little more we've seen a pattern of democratization of ML where more and more people had access to this powerful science and technology and since about 2017 with the rise of the transformer architecture in these foundation models that pattern has reversed all of a sudden what has become broader access is now shrinking to a pretty small group of scientists who can actually train and manipulate the architectures of these models themselves so that's one and what that means is the teams who can do that have huge ability to make the future happen in ways that other people don't have access to yet that's one the second is there is a broader population of people who by definition has even more collective imagination because there's even more people who sees what should be possible and can use things like the proprietary models like the open AI models that are available off the shelf and try to create something that maybe nobody has seen before and when they do that you know Jasper AI is a great example of that Jasper AI is a company that creates marketing copy automatically with generative models such as GPT-3 they do that and it's really useful and it's almost fun for a marketer to use that but there are gonna be questions of how they can defend that against someone else who has access to the same technology it's a different population of founders who has to find other sources of differentiation without being able to go all the way down to the silicon and the science yeah and it's gonna be also opportunity recognition is one thing building a viable venture product market fit you got competition and so when things get crowded you got to have some differentiation I think that's going to be the key and that's where I was trying to figure out and I think data, scale, I think are big ones where's the vulnerability in the stack in terms of gaps, where's the white space I shouldn't say vulnerability I should say where's the opportunity where's the white space in the stack that you see opportunities for entrepreneurs to tackle I would say there's two at the application level there is almost infinite opportunity John because almost every kind of application is about to be reimagined or disrupted with a new generation that takes advantage of this really powerful new technology and so if there is a kind of application in almost any vertical it's hard to rule something out almost any vertical that a founder wishes she had created the original app in well now it's her time so that's one the second is if you look at the tooling layer that we discussed tooling is a really powerful way that you can provide more flexibility to app developers to get more differentiation for themselves and the tooling layer is still forming that's this is the interface between the models themselves and the applications tools that help bring in data as you mentioned connect to external actions bring context across multiple calls chain together multiple models these kinds of things there's huge opportunity there well John I really appreciate you coming in and I have a couple more questions but I will take a minute to read some of your bios for the audience and we'll get into I want to embarrass you but I want to have context you said you were covering product manager 10 plus years at AWS so now see we're covering from AWS which is a whole nother dimension we're covering all in all seriousness I talked to Andy Jassy around that time and Dr. Matt Wood and it was about that time when AI was just getting on the radar when they started so you guys started seeing the wave coming in early on so I remember at that time as Amazon was starting to grow significantly and even just stock price and overall growth from a tech perspective it was pretty clear what was coming so you were there when this tsunami hit that's right and you had a front row seat building tech you were led the product teams for computer vision AI text track AI intelligence for document processing recognition for image and video analysis you wrote the business product plan for AWS IoT in green grass which we've covered a lot on theCUBE which extends out to the whole edge thing so you know a lot about AI ML edge computing IoT messaging which I call the law of small numbers that scale become big this is a big new thing so as a former AWS leader who's been there and at Madrona what's your investment thesis as you start to peruse the landscape and talk to entrepreneurs you have got the stack what's the big picture what are you looking for what's the thesis and how do you see this next five years emerging? Five years is a really long time given some of this science is only six months out I'll start with some no pun intended some foundational things and we can talk about some implications of the technology the basics are the same as they've always been we want what I like to call customers with their hair on fire that have problems so urgent they'll buy half a product you know the joke is if your hair is on fire you might want a bucket of cold water but you'll take a tennis racket and you'll beat yourself over the head to put the fire out you want those customers because they'll meet you more than halfway and when you find them you can obsess about them and you can get better every day so we want customers with their hair on fire we want founders who are who have empathy for those customers understand what is going to be required to serve them really well and have what I like to call founder market fit to be able to build the products that those customers are going to need and because that's a good strategy for an emerging not yet fully baked out requirements definition enough where directionally they're leaning in more than in they're part of the product development process that's right and you know when you're doing early stage development which is where I personally spend a lot of my time at the seed and A and a little bit beyond that stage often that's going to be what you have to go on because the future is going to be so complex that you can't see the curves beyond it but if you have customers with their hair on fire and talented founders who have the capability to serve those customers that's got me interested So if I'm an entrepreneur I walk in and say I have customers that have their hair on fire what kind of checks do you write what's the kind of the average you're seeing for seed and series probably seed rounds and series A you know it can depend I have seen seed rounds of double digit million dollars I have seen seed rounds much smaller than that it really depends on what is going to be the right thing for these founders to prove out the hypothesis that they're testing that says look we have this customer with her hair on fire we think we can build at least a tennis racket that she can use to start beating herself over the head and put the fire out and then we're going to have something really interesting that we can scale up from there and we can make the future happen So it sounds like your advice to founders is go out and find some customers show them a product don't obsess over full completion get some sort of vibe on fit and go from there Yeah I think by the time founders come to meet they may not have a product they may not have a deck but if they have a customer with her hair on fire then I'm really interested Well I always love the professional services angle on these markets you go in and you get some business and you understand it walk away if you don't like it but you see the hair on fire then you go in product mode That's right Alright John thank you for coming on theCUBE really appreciate you stopping by the studio and good luck on your investment great to see you and thanks for coming on Okay thank you John CUBE coverage here and Palo Alto I'm John Furrier your host more coverage with CUBE Conversations after this break