 Hello, everyone. It's great to be here today with you. I'm super excited to talk about how we can put our customers first and create data-driven products with a human touch. But first, just a little bit about me. As you might be able to tell, I'm American. I grew up in Silicon Valley. I've been in London for about the past five years. And as a young kid, there was nothing that I loved more than being able to go and visit my dad at the office, mainly because there was an awesome cafeteria there. But also, because I got to go and watch people with their white lab coats, working in the IC fab, actually building computer chips back in the day when we actually did that in California. Nowadays, it looks a little bit different. I lived there during the dot-com boom and bust and figured that I'd always end up in tech. But it's not how my journey started. I actually started as a teacher. And it's as a teacher that I combined the curiosity that I gained as a young child in California with a growth mindset, a mindset because, frankly, I wasn't very good at it. I had to work every single day to get better, better for my students and better for our ultimate outcomes. I've applied that same thing as I've come into product, for my teams and for everyone else. And finally, I love food. This is how I ended up working in food delivery in the first place, early as a startup named Maple, which was based in New York. We were an early delivery-only concept. And ultimately, here at Deliveroo in London. All right, enough about me. Let's talk a little bit about Deliveroo. Deliveroo was born from a need, a customer need. Frankly, our founder and CEO, Will Shue, he was a banker working in Canary Wharf for those of you that are familiar with London and felt disappointed with the quality of food options that he had available to him when he was stuck at the office late at night. He talks about this all the time. And so, he got on his bike, started talking to restaurants, and he built Deliveroo. Today, we're on a mission to be the definitive online food platform. Much like all of the other companies that you see behind me. So what exactly does that mean? And when we start thinking about, or you hear often, about companies that work in logistics space, they talk a lot about the complexities of operating a multi-sided marketplace. To give you a bit of an idea, I wanna start with talking about the life of a Deliveroo order. Take, for example, you, a friend. You're together, it's Friday night. You're enjoying, having a nice conversation. You start to get a bit hungry. You decide, do we wanna go out someplace? Do you wanna cook something? Ultimately, you settle, you wanna order. So to you, pull out your phones. Hopefully, you open the Deliveroo app and you start scrolling through, deciding, what do I feel like? You debate, do we want Thai, Chinese, and the mood for burgers? You all get this. Ultimately, you settle on ordering some Thai food. Build a basket, couple clicks, you're done. Put your phone away. And what seems like somewhat magically later, your food, the doorbell goes, your food's there, you enjoy a happy meal. As a customer, it's all quite seamless and easy. We hope. But when you put that into perspective, think about it, we have customers who are hungry or sometimes, hangry. You have food that is having, you have food which is a highly perishable product that needs to take itself across the city in a matter of minutes. Nobody wants a soggy fried chicken sandwich. And you need to do that all often for the price of less than a postage stamp. So to make that happen, we have to do an insane number of calculations every single second. To give you a sense of what that might look like, take a segment of London. Here you can see all of the locations of a handful of our restaurant and grocery partners here in London. If you layer on top of that, the live location of the riders who may be there at any given second, ready to accept orders. Now, let's add into the picture the group of orders that are there in that moment. Okay, lock going on. Now, take into account the vehicle type of those riders. You have some people on bikes, you have some people who are moving on scooters. Okay, those are gonna take different times. How do we decide which vehicle type, which customer, which order, who do they go to? You get the picture. Yet, I wanna take you back. Back to that early customer. And to be able to say, who cares, right? All that complexity that I just described, it doesn't matter to you at all. All you want is your food. You want it hot. You want it to be in good quality. You wanna feel like you paid a good price for it. The complexity is our job. And our job is exactly to make that feel a little bit magical for the user by every day coming back to their needs and what they care about most. So to talk a little bit about the marketplace and the user's needs, I wanna talk to you about our three users. We have customers who are the people like you and me who are ordering food or groceries. Our partners who are the restaurants and grocers who are so proud of the food or goods that they've created. And finally, the riders who deliver the goods. What do these different sides care about? For our consumers, they care about what's available. How much does it cost? How quick and reliable can I get it? For our partners, they care about what is the customer experience going to be like? These people care deeply about their food. What is the incremental demand that we can generate and what are the tools and innovation that we can provide to them? And finally, for our riders, they care about flexible work, attractive earnings and security. Great, so we know at a surface level what our users care about. But frankly, it's not enough. It's not enough to be able to build great products. And that's where we turn to primary research. I'll tell a little story. A few months ago, I was sitting in a research session here in London, one of the ones we get to sit behind a two-way mirror, get to hear people talk about their experiences. It's great, I love it. A group of our frequent customers was going around talking about why they loved, didn't like delivery, things that we could do to improve for them and for their experience. Great, most of this was quite positive, but we came to one customer who kind of was a little bit more tentative about things. They talked about how recently it felt like their experience had changed. They couldn't quite put their finger on why, but they stood out. They were still super excited and positive about Deliveroo, but I noticed something a little bit different about that user. I was curious to know more. It's exactly this sort of insight that comes an ability to build empathy with our users that we believe so deeply in at Deliveroo. So much so, we have a program which is called We Are Deliveroo. We Are Deliveroo allows every single one of our employees to put themselves in the shoes of a customer, a partner, or a rider. Outdelivering orders, making food, dealing with trying to handle orders on a tablet in the middle of a service on a Friday night in a restaurant, or handling customer care calls live as they come in. It's great, I love it. A few years ago, as part of this, I was in Hong Kong visiting our users actually getting to go into their own homes and hear about their own experiences, as well as talking to the team there. But the single most memorable thing I did on that trip was deliver orders during a hot and steamy lunchtime in Hong Kong Island. I was running up and down the hills, trying to get these orders to customers on the escalators, trying to figure out where I was going, where to pick up the food, and I showed back up at the office and I looked at the team and I was like, I understand why you need us to take into account topography and do a better job of showing our users that hard work that the writers are doing. Nothing can replace putting ourselves in the shoes of our users. And so, that's all fine and good, but I want to spend a little bit of time talking about the data side of things. And to do that, I want to introduce three As that will come back to throughout their time together. The first is anecdotes. The second, averages, and finally, awareness. Now, I want you to remember that you must trust the anecdotes, avoid averages, and build a deep awareness for your users and your product's performance. Let's talk about each of these and why. Early on in my time as a PM, I learned not to fight the anecdote. Why? Well, the anecdote nearly always wins. We've all been there, or if you haven't, you certainly will be at some point. The moment where your friend or you're a senior team member or your cousin's best friend's uncle shows up and says, why does your product do this thing? Or why don't you have this feature? The immediate reaction that you have is often, it's like that because we try to explain away or we try to justify the choices that we've made. Instead of taking a moment and pausing and saying, what can I learn from this? What could I do differently? We often also try to use data to justify when we think that the anecdote is wrong. But if you do that, you'll lose. Another example, a few years ago, we started getting a bunch of feedback from our internal employees because we run a robust internal employee beta program. They were talking about how every time they would look at a menu, or not every time, inconsistently, if they look at a menu, they'd go to checkout, the delivery time would change a bit. Not big, not a big deal, but something they were noticing. The team looked into it and they were like, our metrics look good, nothing is happening, there's a million reasons, think back to the map, as to why your delivery time might change from your menu to your checkout. So don't worry. And so they went on with their business, got back to their roadmap, delivering impact to their customers, believing that their data was giving them the right answer. However, fast forward a few weeks, more and more of these started to come in, more frequently, suddenly, this anecdote was showing up in their metrics. If they had trusted the anecdote from the beginning, they would have found, but they ultimately found, which was that they were using two slightly different data sources and things had gotten out of sync. If they, and so, remember, to not fight the anecdote, you will lose. Okay, so, fine. We don't wanna be in a position where every time every anecdote comes in, we are completely pulled from pillar to post about going and building a new feature. So what should you do instead? Well, make sure to go back to the primary research point that you're minimizing the space between your team and your end user. And when I say team, people talk a lot about product managers being the people who are the conduit for users and information to their team. No, you want your engineers, your designers, your data scientists, anyone who is working on building your product to be directly talking to your users. It builds empathy in a way and they will build better products. I guarantee it. Second is what we talked about just now. Remember, between the anecdote and the data, the anecdote will win. Some of you are probably saying, I don't believe you. I can understand that. Data is great. We wanna measure everything that we can, right? It's what we're trained to do. We wanna measure it. We wanna see the metrics. We wanna be able to point to anything that might be going on at any point in time. But because we measure everything, we have huge amounts of data. And when you have a huge amount of data, it will inevitably mean you need to aggregate that data. And when you have to aggregate that data, it's gonna hide the insight. And that's why you have to make sure that you figure out how to use the anecdote to get to the general underlying problem. Relying on those anecdotes and your data together will get you faster to a solution. All right, on to our second way. Avoiding averages, they hide insights. Now, trust me, averages can be great at understanding the average performance of your product, giving you a sense of how things are going, how you're doing. But as we were just talking about, when you have a bunch of data, just like an average is going to naturally be, you're going to hide an insight. I'm gonna pause for a second. I want you to turn. Meet your neighbor. You're gonna have one minute, and I want you to answer three questions. How many times have you used food delivery in the last month? Or in the last year, sorry. What about the last month? And for funsies, what's your favorite order? Go. 10 seconds to wrap up. A trick from my old teaching days seems to still work. Well, first of all, like so much enthusiasm, one about meeting your neighbor, getting to talk about these orders. I went through that exercise. It helps you to build this point about averages, right? So I imagine, if I had to hazard a guess, when you started chatting, you guys probably had fairly different behaviors. If we were to take the behaviors of everyone in this room and try to group them together, divided by everyone who was in the room, you'd end up with something that probably looked very different than yourself. And then if you tried to do that in a mass scale, you'd end up with a totally wrong answer. Let's take, for example, again, the delivery example. We have a business which varies a ton by time of day, day of week, time of the year. Is it raining? Is it snowing all of a sudden? All of these cause peaks and troughs in the way that we see demand. And if we used averages to try to predict what was gonna happen from our business, we'd be completely off. Let's take a user-specific example, though, with averages as well. One that we learned recently in experimentation. We hear often from our customers that sometimes they just have too much choice. How do I decide? You give me 50 Indian restaurants. Which one should I go for? And so we had a team that was thinking about, how can I help a user find a new favorite restaurant? What would it need to look like? They had validated it in user research that it was a problem that our users had. It showed up in the data. And so they launched their feature, came to the end of experiment, and they ended up with what looked like a flat result. We'll get to this later. But flat results are the enemy. So, taking that flat result, they segmented their data. They tried to look more into what might actually be going on. And when they did, they found that looking at their new user behavior versus the existing user behavior, there was actually a huge opportunity. New users loved this feature. They needed the help to figure out where they should order from. They didn't have name recognition of places in the same way. Whereas our existing users, they had what one of my PMs likes to call, their stable of restaurants. They knew which places they were interested in trying and going to. They didn't want us to get in their way. And so the averages would have told us, roll this thing back, don't do anything about it. But because we didn't rely on averages, the team was able to move forward. All right, our last day. Build a deep awareness of your users and your product's performance. Specifically, I talk about this as building data fluency. And data fluency means that you have alignment of your product and your performance and how that, and crucially how that connects to your user and your business outcomes. We are not in the business of building products that our users don't want and that don't generate our business outcome that we need. The second is in building an intuition and muscle around experimentation. What I mean by that is dive into the data. There is probably a wealth that you can learn from prior experiments that have been run both inside your team and out. Don't repeat metrics that you don't actually know what they mean. Understand the drivers and get underneath them and use that to fuel the growth for your business. And finally in this one, befriend a data scientist. A great data scientist can make all the difference in your ability to build great products for your users. And lastly, but certainly not least, don't mistake features you want to build for your customer's needs. Think about that one for a second. We have all had a feature that we have been wedded to and loved. We've probably also all had data that showed that it wasn't working for our users. We have to not get too focused on what we feel like are our babies and instead listen to what the users are saying. Quick story on this one as well. When I was an UPM, I was working on a product that was growing like crazy. We had all sorts of new users coming in. We were trying to expand faster than what we could keep up with. Great experience. But at the time, we had built a dashboard which showed how our product was performing and it looked like everything was great. The problem, the trap that we had fallen into, dashboard was built by a bunch of engineers as quickly as they could to give us some insight into what was going on and showed only how our services were performing. They hadn't considered, to go back to the point, of making sure your engineers are close to your users, what the actual product experience was. I opened up my app and I said to them, hey, you're saying this is loading in 300 milliseconds and it's taking me 10 seconds. They're like, oh yeah, that's not what the dashboard measures. Not super helpful. We need to make sure that we always keep in mind both the services and that they're healthy but also that we have a good representation of what our user experience is. Okay, that's all fine and good. We've got our three As, we did a bunch of primary research, we're getting close to our users but how do you actually translate that into what you should build? I want to pause for a moment to let you think about this. What you decide to build is less important than how you decide to build it. What I mean by that is, it's much more important that whatever feature it is, you've aligned success criteria, you obsess over the outcomes of that thing as opposed to the feature itself and that you're bold and biased for action. One of the common pitfalls that I see is people build a new product, they release it, they come to the end of the experiment, they look at the results and say, I'm not sure if we can roll out or not. How are you not sure? Do we not agree up front what success looked like? Do we not know how to evolve forward from here? What is it that causes that? On outcomes over features, the best PMs that I know are the ones who know when to pivot, when to continue, and frankly, when to pull the plug on something. Sometimes it's just not working. It's not worth continuing to invest in. But if we get too sucked in to a feature and too obsessed with it, we end up not focusing on the outcomes for our users and for our business. And finally, delivering value for our users is the most important thing. And that comes the faster that you can learn and evolve forward. So embrace that evolution, get things out quickly, and move. Finally, a quick case study. I told you about that customer that I met in primary research, well, I didn't meet, but that I heard in primary research a few months ago. I want to return to them and talk about how we were able to use that insight and leverage the data that we found from that user to discover really an opportunity in our product. This user, as I said, was thinking, like, maybe my experience has changed a bit about delivery. As participating in research, we were able to dig into their data a bit more. We had permission to look into the ordering habits in history. And so when we did that, we noticed a trend that the user had actually moved house recently. They had moved from what was a much more central location out into further outside of the city. In doing so, when they were using our product every single time that they would look at it, they would search for something, and more frequently than they had in the past, they wouldn't get a result. We called these zero search results. Great, we had this anecdote that came from this user, a bit of a gut feeling. We dug into the data. We found something that seems like it's not going right for them. Okay, so now what? Well, we want to make that problem generalizable, right? If we go back to the point of anecdotes at the beginning. We don't want to just be fixing something for one user. We want to figure out who are all the users that might be having this problem. So looking at the location where that user was, the team noticed, oh, it seems like there's this group of new buildings that have sprouted up out here. And if you look at the zone overall, the zone is a whole it's experiencing more of these zero search results. Users aren't finding what they want. And how could we solve that? Well, they looked at the maps and they said, okay, well, there's a city center here that if we extend the zone radius by about a mile, we'll be able to actually deliver all of those restaurants in addition to what these users can get right now. It would cover about half of the items that we didn't, about half of the items that they were searching for and not finding. Seemed really great. But let's go back to the complexities of the three-sided marketplace. Could they simply just be bold and launch that feature in its form? Well, no, because they know from their intuition, having built products for a long time, that if they release in that form, there's a chance that there's gonna be a downside to either the riders or the partners. And if that happens, then there's a chance that the experiment ends up being a failure. And so they think about it and they say, okay, well, what might go wrong with riders? Well, they may not wanna deliver this far. Okay, riders don't wanna deliver that far. How can we find out? How can we find out if the riders in that area are okay delivering this kind of additional mile? Talk to them. Go back to your primary research. So they jump, they talk to a few of the riders. The riders are like, ah, that's fine. I can go from the city center on the highway to that place in almost no extra time whatsoever. Great, seems like we'd de-risk the main problem here. And so the team launched it. They monitored their key metrics. They saw what was, and with a keen eye on the rider rejections and the customer care contacts throughout, just to make sure. And what they found come the end of the experiment that indeed they had been able to take that one tiny anecdote turned into a generalized finding and multiply that across a group of users for a positive outcome. The job wasn't over there though because they did see that because it was taking us slightly longer to deliver the food, there was a small increase in cold food contacts. And so they needed to continue to evolve forward, figure out what they could do next. Could they work with partners? Could they work with riders? We know, check on whether or not they're using their thermal bags, figure out the ways in which we could ensure a top-notch experience for our users. And so to go back to the beginning, I wanted to just wrap up with a few key takeaways. The first, being ensuring that you start with that end user and remember your three As. And then the three things we talked about afterwards in building. Align your success criteria, obsess over outcomes, and be bold and biased for action. But before I leave, I wanna take a moment. None of this is intended to be a blueprint. It is not a linear path for you to follow in order to build great products. Instead, you're gonna have to figure out how does this work for you, your own style? What does it look like within your organization and your organizational context? And finally, what does it look like to deliver the right outcomes for your users and for your business, for your product? And so yeah, that's all I've got for you today. If you do this, you'll be well on your way to building data-driven products with HumanTouch. And thank you for having me.