 Hi everybody, welcome back to SuperCloud Six. We're here live in our studio in Palo Alto. I'm Dave Vellante with John Furrier. George Gilbert is here as well. We're really pleased to welcome Uday Kiran Medesetti, who's a CUBE alum and is a distinguished engineer at Uber developing the app that we all love. Uday, welcome. Thanks for coming out to our studio. Thank you for having me. Yeah, you bet. So we're super interested in learning about the platform that you're building and it's not just you, it's your team. You have a very large team. We had you on a breaking analysis last year and really dug into it. So we want to extend that. Let's start with the platform. You're thinking about the platform and going back to, I think it was around 2015 when you guys started developing this. How were you thinking about platform and how are you thinking about evolving? This is of course before Gen AI, you had to forecast what was going to happen technology-wise and business-wise, but you can't tell for sure. So what went into that platform thinking and the decisions you made back at that time to enable you to thrive as you are today? Yeah, you know, I think we, like Uber, you know, with our CEO, Dara said, like Uber is a giant machine intelligence problem. Now we are trying to navigate the physical world and there's a lot of complexity the physical world brings and we need to model in the differences in various business lines, various geographies, how different cities work, how people work in the physical world into our systems. And obviously, you know, we cannot predict what's going on, but we need to stay agile to adapt best in the industry and then figure out what works for us. Now back in, I think the platform that you talked about was how we redesigned our core ordering system back in 2015, the state of that was, Internet scale applications can only be built using no SQL systems. And fast forward to four years, five years and then we see emergence of new SQL kind of systems where we get the horizontal scalability and we also get the asset properties. So that's when we took a leap of faith back in 2018, 2019. If we were to rethink our core ordering system for Internet scale applications using new SQL, what would we do? And then we did a two year rewrite and now the core system that handles all of the online orders, all the online driver sessions, that's all built using a new SQL spanner kind of system. And we also have a lot of other systems that leverage a most cost efficient internal storage system so that first we can pick the right technology for the right use case and then we can stay nimble if some new system comes like Pino that we need for an OLAP kind of use cases and we plug in that system where we see there's a good fit. So I want to ask you about just philosophy. I remember the old data center days when a disk drive broke, you went and replaced it. You had to call somebody and they came in to fix it and then when the cloud came about, they architected it. So you didn't have to worry about it. The disk drive breaks, throw it away. You just keep running. Do you have a similar philosophy with software? In other words, you had to develop some code that didn't exist. It might have been the semantic layer to make things more coherent, whatever that is. But do you have a mindset of we will throw the code away, like Tesla just supposedly did and rewrite it, or do you have the mindset of we must be able to maintain that and evolve it? What's that philosophy? See, I think that's a, I mean, we always, I mean, we cannot always keep rewriting it every one, two years. We have to have a pragmatic choice on whether it makes the right sense for that right time. And we also have to think longer term on, if we were to think for the next five years, does this architecture handle the kind of scale that we are predicting for the next five years? So when we see there's a right fit, sometimes we will be too early trying to move something. But when we see that, okay, it's ripe for a rewrite because we know that fundamentally it's not able to handle the kind of growth that we envision in that particular area. That's when we take that bet. And when we take that bet, we also need the full organizational commitment to follow through that because you cannot have many dangling rewrites going on. Then it creates a lot more mess than what we can already started with. So Uday, back to where Dara positioned this as Uber is a machine intelligence problem. Tell us about AI starting, give us the context at multiple scales, first like how it helps you plan at the city level, like how to anticipate surgeons of demand or bring on more drivers or how you think about route planning, or then AI at the trip level where you're doing matching and calculating routes and fares, and then maybe AI even at the management of operations level. Yeah, so I think it's not like one single machine learning model that solves every leg of the ordering lifecycle for us. What we try to focus on is democratize creation of machine learning models within the company so that if you take the entire lifecycle of figuring out the right set of features and creating the data pipelines for that, doing the model evaluation, then deploying that to production at scale, any kind of like along with data engineers, machine learning engineers, applied scientists, software engineers, they can work collaboratively inside our machine learning platform to discover the features, to build new models and ship it to production. And what that enables us is now we have hundreds of machine learning models in production, optimizing every phase of the order lifecycle, whether we have like powerful deep neural model that is predicting what is the ETA for the trip. We have personalization models that predicts if Dave opens the Uber app, what product we should suggest him. If you open the Uber Eats app, what specific item we should suggest you so that we can optimize the conversion rate. And it's also optimizing not just end users but also internal users as engineers. We want, this is the kind of costly commodity, right? Like how do you get, how do you optimize the life of engineers with AI? So we kind of look at every life, every phase of the life cycle and how we optimize that with AI. So one came in and came up earlier from the startup panel we had earlier, the hot young guns rising up, getting their series B, a lot of doing a lot of things that's very Uber-like. The question that came up was with vector embeds, that does a great job of identifying context, but behavioral data is where the personalization comes in. What you're getting at is, as you look at the system at scale, I mean, doing it in isolation, okay, I can get that, do it with some embeds, use some retrieval. But to get that, that's all I'll say, it's kind of, it is a search problem, but like to get it right with context and behavioral data, how do you look at that problem? Because a lot of people are looking at GNA AI right now saying, I get the contextual behavioral interaction, that was the old search days, now it's in real world. At scale, what does it look like? How should people be thinking about this? Yeah, so I think we are in the early innings of how, what kind of end user experiences we can build with the new trend in LLMs and RAG models. What we are trying internally is, can we provide access to the best-in-class proprietary models and open source models to all of our engineers and data scientists so that we can do a lot of experiments and see what works. So we have internal deployments where we optimize how we do code reviews or how we do write tests, how we automate UI tests or we have, we optimize the life of our agents where they are in solving user's problems. Ultimately, we need to take the static world knowledge and we need to join it with Uber-specific semantics and knowledge graph and user-specific context. Like if I'm trying to onboard to Uber and if I have a question on, hey, why am I not, why, what is stopping me to take a ride? We need to take Uber's information about all the requirements you need and this user-specific information about what have they done so far and we can give them an answer in a format in which they like that we support them. You're combining this. And real quick clarification, you say proprietary models. Do you mean proprietary data from you guys or proprietary models being open AI and thropic? Yeah, proprietary models like open AI or Gemini and stuff like that. And then integrating that with internal Uber data, that's where Vector Search and Rack comes in where, you know, because we need to be careful about what kind of data exposure we are providing to these companies because we are also in the early phases of like figuring out the data access policies and security. So we want to take the more conservative side. Like in cases where, you know, in some cases we leverage open source models, so we deploy them on-prem within our VPN so then we can be absolutely sure there is no data leakage. In cases where we have proprietary models, then we are more on use, we're not trying to use any user's data, we are more trying to help engineers or like in access, providing, using information that is not- You're controlling the state of that. Exactly. LLAM by controlling what you feed in. What you feed in. You're not relying on their data with hallucinators. Exactly. To prompt it. So let me ask Uday, you made a reference to the Uber Knowledge Graph and Knowledge Graph is kind of, kind of like this hot sort of meme right now, but I want to more ask you about when you organize your data into a model of your business, are you doing that with a Knowledge Graph and then are you using that to feed the context into LLAMs for different use cases? And can you give us examples? Yeah, I think we are also in the early phase of building a Knowledge Graph with Uber Business Semantics that we can leverage to build the right LLAM applications, but in a traditional machine learning models, whether it's like XeBoost or deep learning models there, we have like a strong data lake with really good quality data pipelines that we can use to build features that engineers use to build machine learning models. So that's kind of our data lake and we are now augmenting that with what kind of format or Knowledge Graph that we need to also build the next generation of LLAM applications. So just to be clear, what is today like a bunch of tables which have columns which masquerade as features might tomorrow be organized into a Knowledge Graph so that it's easier for the data analysts, the data scientists to consume, to find, to combine when they're building traditional machine learning statistical models or even feeding data into content into LLAMs. Even today we have a variant of that, like we have a system called Data Book where any data engineer or any machine learning engineer can go there and get access to the tables, like what are the relationships with other tables, the column names, descriptions, what are the data access policies, what are the business classifications of that data that allows anyone to come in, find all of the data sources that are blessed and do accurate data analysis, trusting that the data that they're using is accurate and complete. So those are, that's your data products today are manifested in this catalog. Exactly. Okay, so George and I were talking earlier, let me set it up, George, and you get into it. We kind of, the question was like, how much is AI versus human, right? But so it's kind of a weird question, but when you think about a trip function, we talked about, okay, you got riders and drivers and ETAs and costs, et cetera, and you build a workflow out of that. We're trying to figure out, okay, how much is procedural and how much is AI and how has that changed over time? So see, I think in general, we try to, one of the core important aspects that we need to always think about is reliability. Like at the end of the day, if everything goes down, what can be the least common experience that we can serve the user because users trust us to go from A to B to get paid for a particular trip. So in some sense, we have most of our core trip flow functions are procedural, like we have different microservices that handle different aspects of these services. And there also we think about what is the degraded experience look like. And where we use machine learning is how do we optimize every leg of the journey? Like I can tell you a five minute ETA, but maybe if I use a machine learning model, I can tell you it's exactly three minutes, 20 seconds. But if that part of the system is not working, it's better to tell you five minutes rather than, hey, I can't do a trip without that. So that's kind of the- Where it's maybe a little off, I've noticed, but that's okay because the application is up. And you're optimizing and brew, cap theorem, reverse theorem. Yeah, same thing with each personalization. I can tell you like your nearest restaurants that at least you can order something instead of saying, hey, like, I can't tell you what you might order that day. So the resilience is implemented as rules that are procedural with a, like a, I'm sorry, I've got it backwards. It's first is the AI, which gives you the optimal experience. But if that's not working, you have rules to fall back on. Yeah, I mean, if you think about various checkpoints across the trip lifecycle, there is a checkpoint where I might want to access the ETA. That checkpoint behind the scenes can leverage, can use a machine learning model inferencing to predict, okay, what might be the ETA, but it has a fallback experience to what might be a, maybe fall back to historical data, maybe fall back to more static rules for that city. And like even the feed that I gave, like there's a fallback feed that tells you some great experience, but there is a machine learning powered experience that looks at a lot of context to tell you the right set of restaurants to serve you to optimize the conversion. How about thinking about macro AI across the life cycle? How much is AI sort of spreading across that life cycle? Whether it's data engineering or AI ops or even extending out to the user experience, is it, should we think of it as sort of an overlay in the whole system or is it more injected into the different components? Yeah, I mean it's kind of injected in every step of an engineer's life cycle, every step of a machine learning engineer's life cycle, every step of user experience life cycle. And then we think about at that leg, what kind of machine learning model might give us the best ROI? It could be a maybe simple traditional machine learning model or it could be a new age LLM kind of model that can optimize that experience. But within all, so this is interesting to understand how it's injecting and enhancing at many steps. What, is there some outer loop where you say, I want to optimize this city, you know, this month for maximum profitability or I want to improve, you know, rider experience so I want more liquidity. Is there some way to link an overall objective to the behavior of all these specialized models and all these different workflow steps? Yeah, I mean, I think like, you know, different models are optimizing different phases, they are kind of in some, in most cases mutually exclusive, right? Like me trying to optimize what specific product I want to show you in the product selection may or may not be the same model that has to really decide what specific matching function that I need to evaluate. And in cases where they need to work together, then we make sure like, you know, we have mechanisms in place where they work together. Guys, I want to bring up something real quick while you're here, I know this is a good topic on deep dive, but as you guys are successful, you're like a leader in what you're doing, you're an innovator, thank you for coming on. Other enterprises are now kind of catching up, they're reading about the reports. We had a debate earlier on about what's this new group that's emerging in enterprise? You guys are kind of the first ones to kind of, not first wave of innovators, but in your basic enterprise that are now going cloud native, they've had data analytics for years, data warehouses, we've been covering it deeply, see snowflake, data breaks, data leaks, but now a new department's emerging, a new persona, that's the reign this in, is it cloud native, cloud operations, is it data centers, is it software engineering, platform engineering, or is it data science? Because not one department does this now. Exactly. What do we call this? I think, as you said rightly, it's not necessarily one department, even today, when we have to bring a machine learning model to production, in the initial stages, we might need a data engineer who thinks about building high quality data pipelines, then you need a applied scientist who thinks about what specific features and how do I create that model? Then you might need a machine learning engineer to convert this into a model that I can deploy to production. Then we need a software engineer to run this at scale for millions of QPS that we need to handle. And so... Yeah, it's a multi-discipline, it's a system. It's exactly, and you need some platform engineers to think about what are the common bells and whistles that I can provide to all of these folks so that they can focus on their core business logic without worrying about integrations across Uber and external systems. This is why I bring it up because you guys, I know this is not the Uber conversation, but for our other conversations where these young startups and innovators, they want to be the supplier to the enterprise. If the enterprise has to form their own teams, the vendors don't have the answers because they're one-dimensional, they don't fit the system. So the thesis is, who serves the enterprise? Because you can't just roll in and say, hey, I'm a big-time database company, or I'm a networking company, or I'm an observability company. They might not have the one... What's the pieces together? Well, what we're hearing and we're seeing is that it's the systems team, if you want to call it that, for lack of a better name. But that's a perfect chance to describe now how Uber is becoming a fleet management app. It's not just your in-house app. Maybe because what you were getting at was they've built a system, a coherent system out of many pizzas, many of which they had to develop, but now you're selling. But John's trying to understand. Or if I'm say Cisco or I'm a company and I want to sell to Uber or Enterprise, they don't have the team. I'm selling to IT in the old days. Then I'm selling to data scientists of some analytic solution. Oh, that's for sure. I'm selling databases to that person over there and oh, the DevOps team is over there. Who the hell's in charge? Yeah, and it doesn't stop there, right? Like, even like you also need some observability integrations because if you're running a model in production, you need to get the feedback loop, get the metrics, see how it's performing. Then you need some integrations with all of your metrics, alerts, and monitoring tools. So yes, I think like companies need some way to integrate all of these different things and think about the entire lifecycle. Otherwise it takes a lot of burden on individual cohorts to really get and move faster. We're gonna come back to this topic, but I'll let George, because George's on a good thread, I want to get to that. Can I just interject something for a second? Because Uber had such an impact on the economy. I mean, some companies like Amazon as well, but you think about the gig economy, the ability to self-serve, a surge in demand. I heard just recently that restaurants are gonna now do surge pricing. So, thanks for that. So, my question is, we use Uber for all as a metaphor for the future of being able to build a digital twin of business, people, places, and things, which you sort of referenced earlier. Kind of knowing what you know now, do you feel like AI will enable the masses without thousands of engineers to actually create an Uber-like experience for their own business while you go off and invent some new stuff that we can then copy 10 years down the road? Or do you feel like there's still a giant gap between what you were able to achieve and what the masses will be able to achieve? See, I think if you think about the amount of effort it takes to build an internet-scale application 10 years ago versus now, there's a lot of off-the-shelf tools that can get you to a relatively good state with very small number of engineers. Now, getting from zero to one is one thing and getting from one to 100 and getting to a global-scale understanding all of the local policies and making sure your system adheres to that. That's where you need maybe a bigger workforce to be able to translate that into a working system, but getting to a zero to one, like looking at what's going on in the industry, I think you don't need a large team to do something quick, something that's also specific niche. Great, George. Maybe then take that thought and elaborate on Uber has been maturing its platform and application for well over 10 years. Talk if you can briefly about the fleet management, but you were mentioning earlier about how you can plug in providers and customers with different business models and essentially requests, if I understood it, so that, and you were just saying that has to be adapted to different local conditions. So talk about how Uber is becoming this platform independent of Uber the service. Yeah, now think of it as when we first started, every rider is maybe using Uber Rider app to request a ride and then every driver on the platform has onboarded to Uber as a first part. Now, over time, instead of riders requesting through Uber app, we can expose API, we can to hotels, to health organizations so that they can provide Uber like experience for their guests, for their patients. That's on the consumer side. So then you have one P3P model. And then on the provider side, then we can provide APIs for large fleets so then they can bring in their hundreds of vehicles, thousands of vehicles onto Uber platform so that they can optimize their overall fleet efficiency because we have enough liquidity in the marketplace that we are able to give to them, which they might not be able to get on their own. Yeah, this is exciting, George. We talked about the Saber example, right? The original company that built something internally and then sold it to the industry and became a standard, certainly Amazon similar. And was worth more than the underlying airline. Yeah. The rusty asset. Is that right? I didn't know that. Yeah, the rusty asset versus the software company. Yeah, Uber has all that data. They have the scale, they have all that information. They're leveraging the data and liquidity. And so why build it when I can buy it from Uber and then focus on my business? And then that's your scale model. So. Yeah, exactly. That's the new way. George, we'll give you the floor here. Maybe another question on, you know, we've been talking, we've been touching on gen AI. Have you started to form plans on how it may affect all steps of the software development life cycle? Not just code generation, but you know, you alluded to some of the planning, but also like the data engineering, the analysis, even if it's relevant in operations itself. Yeah, you know, in the last six months we've had at least two internal hackathons where first the machine learning platform built the right set of building blocks and asked engineers, hey, think about all of the ideas. And then like we've had amazing ideas across the board. So if you think like all the way from trying to onboard to a new system, now I can take all of the Uber knowledge that siloed and then help engineers onboard to new space. I can help you with code suggestions. I can help you with maybe if I'm doing code reviews, I can give you the contextual suggestions. And I can go back and fix all of the lint issues in the repository and generate code reviews for you that you can accept and land. So I can clean up our code base so that it's more maintainable. And then on the AI ops, we can build assistance for on-call engineers to give them the right contextual information so that it helps them to mitigate the issue faster. For analysts, we can give them, they can write natural language and we can convert them into queries and give using the right blessed tables and columns so that they don't have to set up the right joint conditions, right where clauses because they can express, I want to do how many trips happen in San Francisco with this kind of constraints and we can convert that into a query. So we are thinking about different cohorts of employees and then figuring out what is the best fit that can help them really kind of a co-pilot for them to optimize their day to day so that we can offload some amount of work for them and so that they can focus on the important parts. Our time with you always flies, Uday. We hope we can have you back and dig deeper. I really appreciate your time. Thank you so much. It's very clear you're getting ROI out of AI, which is a big theme here. We want to help people understand those who are actually applying it in practical terms. So thank you for sharing. Thank you so much for having me. Always a pleasure to have you. You're very welcome. Okay, keep it right there. We have Walmart's up next. Last, a couple of super clouds ago, we heard about the Walmart cloud native platform and they're super cloud. Well, they built an abstraction layer on top of that called Element and Paul Gillan talks to Hari Vasudev, who's an executive vice president at Walmart about their ML and AI platform. So keep it right there. Super cloud six right back.