 Hello everyone good afternoon. This is my first slush and I have to say pretty impressed this is pretty impressive venue So really excited to be here and talk about all things metrics So I just got a really warm introduction But I'm Jessica lacks and I lead analytics for DoorDash and for volt So if you don't know DoorDash and volt are both technology companies that connect consumers with the best of their local neighborhoods And I've been with DoorDash for almost ten years So I started as a GM and that evolved into my role leading analytics And over the last ten years We've evolved from a small startup to the large public company that we are today And so I've learned a lot and I've seen a lot as the company has changed over the years But the one thing that hasn't changed is our data-driven approach to running our business and so Since the early days we've measured our performance. We've set goals around our key metrics And that's what I'm here to talk to you all about today Tracking the key startup metrics some lessons from my experience at DoorDash There we go. All right, so over the next 20 minutes We're gonna cover a few topics so first off We're gonna start with measuring what matters why it's important and and how to do it then we're gonna talk about Five principles that I have for setting good goals things that I've learned in my experience at DoorDash And hopefully you can avoid some of the mistakes that I made and not make them in your own careers And then we're gonna briefly gonna cover who should own your metrics and then we'll close out the session So let's get started So measuring what matters. This is my favorite slide by the way that pasta looks amazing. Glad we're right after lunch All right, so in this first section We're gonna talk about why it's important for startups really any company to measure the right things So we're gonna talk about Why metrics and goals matter and then we're gonna talk a little bit about how to select the right metrics? Why you should track your goals and make them visible to everyone and then we're going to talk about how to evaluate your metrics and iterate So let's start with why metrics matter No There we go so Tracking the right metrics will Objectively tell you if you're on the right track and how your company is doing So whether you're small and searching for product market fit or you're scaling Or you're a large company just trying to hit your goals every every quarter the numbers are a feedback mechanism So these insights into performance will enable you to move fast and speed is a competitive advantage No matter what size your company is So once you've identified the key metrics you can set goals against these metrics and then this helps your team to focus It also helps your team to prioritize so that they can really move the needle for your business So, how do you pick the right metrics? well First think about your business model and how your product delivers value to its customers So business models fall into a few categories. You've got marketplaces like door dash and volt You've got e-commerce Enterprise models SAS subscription You've also got transactional models and advertising models And so while the suggested top-line metrics for different business models might might vary and you can find lots of information On what those should be online the key metrics generally cover user acquisition Customer engagement revenue generation and operational efficiency So and the suggested approach that I'm going to talk to you about should work with most business models and most industries All right, so What is this approach the first thing to do to figure out what the right metrics are for your business is To think about how you deliver value to your customers So we're going to use door dash as an example So how does door dash deliver value to our customers? No pun didn't know pun intended But I do still get joy from it. So as a three-sided marketplace Door dash actually has three sets of customers. We have consumers who are placing orders We've got our delivery drivers that we call dashers and we've got merchants and so the value that we provide to consumers is Being able to deliver their favorite restaurants grocery stores and and local shops to their doorstep For dashers door dash provides flexible earning opportunities and for merchants incremental revenue for for the merchant But all of this hinges on consumers ordering there are fewer earning opportunities for dashers or couriers if people aren't ordering and Merchants are going to get less revenue if if consumers aren't ordering So it all hinges on the consumer ordering and so that's where you're going to start at least that's where we started we tracked order counts and For volt same thing order volume was the north star metric and you'll see that other market places have similar metrics So Airbnb has bookings eBay has items sold and uber has rides So from that north star we are going to embrace an annoying child the one that always asks why Incessantly over and over again, except we're going to ask what more specifically what drives? And what we're going to do is we're just going to keep asking what drives Starting with orders. So what drives orders? Well, it's the number of customers and The number of orders per customer. All right, so what drives? Customers well, it's the number of new customers and the number of existing customers that are ordering in a period Well, what drives new customers? It's the marketing spend paid-cac that you're spending to acquire new customers It is the number of organic customers that Naturally come to your site or your to your app and it's the funnel conversion So how many of those customers are actually converting to place an order? And so you just keep doing this over and over and over again and what you're doing is creating a value driver tree and So these become the inputs that you want to start to track and Generally what you'll find is that the root of this value driver tree and The the different leaves or branches that are repeated most often are the things you really want to pay attention to those are the metrics that you really want to track and so for DoorDash we found that it was The number of orders it was the number of new customers acquired Cohort retention the number of merchants our delivery times the percent of late orders The customer star ratings dash or efficiency and we had a constraint metric of unit economics And so you may not be surprised, but those metrics cover user acquisition customer engagement revenue generation and operational efficiency We did add a constraint metric. You heard me reference unit economics Constraints typically don't show up in your value driver tree So this is a great opportunity to sort of put on the evil genius hat and think about how you might game the metrics It'd be really easy to acquire a lot of consumers and have them order all the time if you gave away Your product for free, but you wouldn't have a sustainable business So it's important to have a constraint and for DoorDash We had goals around unit economics both the revenue side as well as the cost side and we also included a Payback constraint for our marketing spend to make sure that we were spending that money efficiently And so I encourage you all to take to take the time and do the work to identify Your own value driver tree so that you can select the right metrics But one of the things that you also have to remember Winston Churchill famously said that perfection is the enemy of progress Something I remind myself of often so don't get worried about having the perfect and most complete value driver tree It's most important to just start with what you think is best and you can make changes along the way All right, so you've ID'd all the input metrics You've built your value driver tree and you've selected the most important ones to set goals against well now what? Now you want to make them visible to? Everyone in our first office our first DoorDash office We had a TV screen that displayed a company door dashboard It's what we called it and we emailed a daily recap of this DoorDash board to everyone in the in the whole company all You know 50 of us And we regularly reviewed our progress So that we could identify what was working and find those things to double down on or Accelerate and also to identify if we were missing our goals. We can figure out Where the gap was and put together a plan so that we could get back on track Everyone in the company knew what our top metrics were and how we were performing against them And it was one of the sort of origins of our company value one team one fight, which remains a company value today All right, so after all that I want to give a little bit of a public service announcement or warning Which is that you're gonna get it wrong? Hopefully not all wrong, but you're gonna get it wrong And the important thing is to figure out why you were wrong Yeah, what matters is Identifying the metrics that maybe aren't giving the right signal and are giving too much noise or the metrics that you just don't pay attention to that much because they just Really ultimately aren't important to what's driving your business as your business evolves You're gonna learn new things about your customers. You're gonna you're gonna add metrics You're gonna deprecate others and that's completely normal At door dash while many of our core metrics remain the same as that original list that I that I gave We got smarter about how to measure things. So for example, I mentioned that number of merchants was something that we tracked What we found though was that some merchants are more valuable on the platform than others Think about your neighborhood the first Thai restaurant that we add in your neighborhood is gonna have a much bigger impact on Customer engagement than say the 16th Thai restaurant we add And so what we needed to do was to change our metric to evolve what we were measuring to get a little bit more nuanced as we learned more about our customers and so As you get smarter as time goes on as you continue to iterate you'll find the early Indicators the the proxy metrics that that you can track in the short term and that are good Good forecasters for some of the longer term more lagging indicators like customer retention and LTV so if you start tracking kind of what you've got to go on you learn you iterate and you sort of Evolve the set of metrics that you're tracking you'll find that you're more More able to to manage the business and to find the right set of metrics for the stage that you're in at that time All right, so we've talked a lot about tracking the right metrics, but what about setting goals? Goals are incredibly powerful. They can focus the team's attention on achieving a desirable outcome Which of course is what we want So just remember that goals are not a to-do list They're the final outcomes that you're gonna lead that are gonna lead your business to success So once you've set these goals you can then break them down into action items and key milestone with associated dates But that's a topic for a whole other talk So over the years I've learned a lot about setting hitting and yes even missing goals And so I tried to distill it down into five principles things that maybe aren't talked about a lot So here we go. We're gonna get started Principle number one good goals stretch us to perform at our best and put us on a course to win principle number two is good goals are simple principle number three good goals drive incremental value Principle four good goals address the causal input metrics that drive the output metrics and Principle five is good goals incorporate the fail states. So we're gonna go through these one by one All right number one good goals stretch us to perform at our best and put us on course to win So we're gonna start with the second part of that statement because in this room we are all winners The question to ask yourself is if you hit your goal Are you on track to succeed? If you or your team are hitting your goals month after month Quarter after quarter, but you're not capturing market share or you're not really moving the needle to your north star metric You're probably measuring the wrong thing and so a practical example of this is early on in our Days the initial search team goal was search conversion Which sounds like it makes sense the search team is trying to build a better algorithm to improve search conversion the problem is that some of the first projects were moving search conversion, but not overall conversion and so Was the team really adding value or were we just simply shifting traffic from sort of one surface area in the app to another? In fact to go to an extreme example the search team could improve search conversion by making search really hard to find So that only the you know existing customers who were familiar with the with the with the app Or really high-intent consumers would ever find it and would have naturally higher conversion But that's not the spirit of the goal And so we had to figure out What would be the right goal to set for the search team so that we could see the Ultimate goal we were looking for which is an increase in orders an increase in customer engagement and so the the idea is that You pick a goal that's going to drive business impact that's ultimately going to put you on course to win and so That's what we have to start bowling our teams on So moving to the first part of the sentence which is good goals stretch the team to achieve More than they thought they could without being demotivated of course So if you're 1% behind goal the things that you might do to get back on track are quite different if you were to say 10% behind goal and so you want to Goals are typically seen as ceilings by teams and so you don't want to unintentionally limit your team's potential By setting a goal that's too easy to achieve. So you're like I'm done move on to the next thing so If you set challenging goals ones that make people feel a little uncomfortable the team is forced to identify new and innovative solutions to problems so that you can Achieve more than you even thought was possible So in every planning cycle, I've been a part of it door dash the the goals that were given seem completely insane And there's always a big gap between sort of the current course and speed of the business where where our business would Naturally go and then where the goal is But that's where the magic happens That's when the teams need to come together to figure out how we're gonna get from where we are today to where we need to be and so Even if the initial goal seems really hard You'd be surprised how often the team can come together and figure out a way to solve for that gap and ultimately when looking back realize they've done so much more than they ever thought that they could and So setting setting difficult goals for your team that really stretch them is one thing But it's also good to pick sort of one area particularly for a team that has a history of achieving goals and Set a B. Hag. So B. Hag stands for a big Harry audacious goal It comes from it comes from a book I believe called built to last and the idea here is that if you set sort of this big Harry audacious goal then you can inspire the teams to get creative and inspire radical change and That it's when things are going well So as I mentioned for some of your top performing teams when things are going well That is when teams are most receptive to these sort of aggressive goals And so pick your best team it is true what they say if you do good work You just get more of it but pick some of your best teams and Pick one area to really set an aggressive goal and just see what your team can accomplish All right, so principle number two is to keep your goals simple this is something that I know my team often has to remind itself which is Sometimes in an effort to capture everything you find the team setting a metric that takes you know One thing from over here and 50% from over here and then the square root of this other thing Into a big amalgamation of absolutely nothing. And so don't do this Whenever possible you want to keep things simple if a goal is hard to explain It leaves other teams and leaders unclear about what success looks like if it can be hard to understand What moves a metric if it's complicated and of course for for a team like mine It can be really hard to explain why a metric moved if it's complicated So complexity should be avoided whenever possible Simplicity guarantees the greatest level of user acceptance and interaction Never forget about incrementality So principle number three is to make sure that your goal will drive incremental value When we initially started running our new new consumer promotions When I was GM in Boston our team measured the performance of campaigns based on the number of new consumers that we acquired The problem with this is that it didn't account for the fact that Many of those consumers would have joined or dash anyway Even if we weren't running that promotion, but because we were we just unnecessarily subsidize those orders so Incrementality is an important thing to incorporate into your goals in a perfect world We would only ever discount an order for someone who had no intention of ordering on the platform But in reality that's nearly impossible to do and so what we need to look at if you if you Look at this sort of test and control group on the side. You can see that the red people we're gonna order anyway It's only the blue people that ordered just because we were running a promotion But we have to pay for both the blue people and the red people And so what you need to see is that the number of blue people Make up for the cost of the promotion to both the blue and the red people and so there are things you can do to improve incrementality with better targeting But ultimately we want to set our goals on driving the blue people the the incremental value that you get from say this promotion and so The idea here is that you can set up an experiment you can measure the incremental value and set your goals Based on on that and so in this case you have a test in a control group running a true a B test Sometimes that's not possible in this example for the early days in Boston at DoorDash We didn't have the ability to run an a B test so we would do a match market test You can also use difference and difference synthetic control or a number of other quasi-experimental methods to try and measure the incremental value and goal your teams on that All right principle number four is to understand the causal input metrics that drive the output metrics So it's common to dig in when you miss your goal teams want to know what happened Did we did we have a bad assumption? Was there a bad execution? Does the causal mechanism exist, but we just haven't hit it yet But it's equally as important to confirm the causal mechanism when things do work So if you have a hypothesis for example that consumers Want fast delivery and so you're testing whether a new carousel in the app that's sort by speed is going to improve Conversion and you run a test and you see that yes conversion went up It's not time to pat yourself on the back and move on to the next thing What you really want to know is was it speed that led to that increase in conversion So you should check to see if delivery times actually fell and if that what caused that increase in conversion So going back to the example I gave on the search team with the bad goal of search conversion let's say that the search team rolls out a new search algorithm and Actually does realize an increase in overall conversion now is the time to go and look to make sure that search conversion actually increased and is What's causing the increase in overall conversion? So it's always important to know the why and to validate the mechanism of change There we go. All right. So last but definitely not least is principle number five Which is don't ignore the fail states So only looking at the averages will cause you to miss key insights at DoorDash We deliberately focus on the edges will track the outliers will look at the 90th percentile We want to understand these fail states fail states can have a much bigger impact on your business then just their Frequency might suggest so looking at the edges can help you to identify bugs Fraud of course and also can can show you use cases that your your consumers might be using your product for that You weren't even aware of and so we monitor the edges to Deliberately identify these fails fail states and then we set goals for ourselves to eliminate them So a few common product examples of fail states could be login failures card declines or credit card expiration Also account deletions and of course app restarts So think about where your customers might encounter friction Friction that is preventing them from realizing the value of your product And then you want to measure these fail states and set goals so that you can eliminate them All right. The last section is about metrics ownership So do you need a VP of analytics or a data science leader to own your to own your metrics and your goals? So My peers might not like this answer, but probably not at least not in the early days so a feature of the startup team is that jobs are fluid and ambiguous and You know people are doing things that maybe aren't in their specific career definition But that's that's what's fun about a startup And so whether it's a finance person or an operator a product manager or even the CEO Really anyone can start to taking charge in defining what the right metrics are Defining those input metrics creating your value driver tree And ultimately everyone in the company should be tracking these metrics on a daily or weekly cadence and so you know for door-to-ash things changed around our series B when all of a sudden there were too many questions and the Complexity of those questions were beyond what any sort of part-time finance person could handle And so at this time we had GM's like myself As well as our head of finance who just needed some help to answer these questions to identify What the drivers of our business were and how we could move them to help us set up goals for different city launches to know What a good launch looked like and develop that playbook And so that was the the impetus to to the move from for me from the a general manager to creating our bizops team Which would ultimately become the analytics and data science team? And so just some closing thoughts before we wrap things up While tracking metrics and setting goals are incredibly important that is just the foundation The real value is using that information to actually drive business impact And so I always say my job isn't to measure retention I mean it is to some extent, but the real value the real job I have is to figure out how you take that retention curve you've just measured and shift it up and so in conclusion Metrics and goals are incredibly powerful tools for driving your business Towards success so be intentional about what you measure and make your metrics visible to everyone and Then when it comes to setting goals, remember these five principles and In and lastly and maybe most important don't overthink ownership Fill your team with smart scrappy Analytical doers and just get started tracking those key metrics Thank you so much