 Hi everyone. My name is Nick. I'm a PM at SANA. I've been there for about two years before that. I worked as a PM at Yammer. I worked with Neil and I had started a SaaS software company. That's how I got into the PM space, realized how much I didn't know and how much I needed to learn before I ever tried to build something again and that led me down the path of product management. This is Neil. Thank you, Nick. I'm on the product team at Patreon. I've been there about a year and before that I worked on the product team at Yammer for about five years. Again, how we met. I'm working on stuff at Patreon like the reward system that creators that use that product can use to incentivize their patrons to join and also stay pledged to them so that they can make money as an artist. Welcome to our product teardown workshop. This is a process that Neil and I have used when we were PMs at Yammer. We often held product teardown workshops among our team and it's a process basically just to go through apps and learn from other products that are out in the world. We're really excited to bring this to the product school. We actually brought it to the Mountain View campus last year and thought it went well so we're here in SF. So why do we do these product teardowns? The way to get great as a product manager is to constantly ship stuff, learn from your wins and losses, but that takes a long time. So how do you get better quickly? One thing we can do is learn from other products. Obviously we won't know exactly why things were built, whether an AB test won or lost, what the metrics looked like, but we can make some assumptions about goals and hypotheses that were made and start learning from other products. We can also kind of learn to appreciate new trends and pick up design patterns. So by looking at all these other apps and products you start building a library in your head of what are potential solutions to different problems. And then finally, we think it's fun. Hopefully you want to become a PM or you are a PM because you really like products. So it's really cool to open up an app like House Party when you're in your mid-30s and be like, what do the kids do in these days? So that's a really fun part of doing product teardowns, trying to understand why are apps popular, why are they catching on? So today we're going to teardown two apps. We're going to compare Caviar, which is owned by Square, and Uber Eats. We chose these apps because the food delivery space is really interesting. There's been lots of consolidation, and then we're also seeing a lot of the large tech players like Amazon with Amazon restaurants get into the space. And it's also really helpful to compare two apps so we can look at similar apps and see how they have different product strategies and choices they're making and make some assumptions about why. I think also Caviar and Uber Eats specifically have their own drivers, so we're not going to have to make different discussions based on business model like if you looked at things like E24 that tap into different driver pools. And finally, it's interesting that they're both part of larger tech companies, and Uber has a very, very different business model and strategy than Square, and that drives kind of why these apps exist in the product choices they're making. So when we're tearing down products, we need to think about these broader business goals and the strategy of the products and how that affects all the product decisions that are being made. As we're going through this, we should assume that the product teams at these companies are good. Often you run into something that seems like a bad product experience, but there's probably a logical reason that it was built that way, so let's assume that. And we have made a bunch of assumptions because we don't work for Caviar Uber Eats, we don't have any insider information, so we're probably wrong about some of these assumptions, but it's still useful to make these assumptions and learn from them. Cool, so let's dive into the two apps. So pulled some quotes about Caviar and their value prop, split this into how they think about their different user types, so customers, restaurants, and for the primary business. So for customers, Caviar says that it's an easy way to order meals from popular local restaurants across the US. This doesn't sound very different than probably the value prop that any one of these products would tell you. To restaurants, this is a little different. Caviar is part of Square's full suite of tools for businesses, enabling restaurants across the country to reach more customers, grow their sales, and expand the reach. So you can kind of see how this would be different than what Uber would offer. And then the value to Square, so I pulled this quote from something that Jack Dorsey was saying, that he believes Caviar solves the biggest problem that every restaurant owner faces, which is physical space constraints. You just can't get more people into your restaurant, which they think can ultimately lead people to becoming Square customers. Uber Eats, pretty different. To their consumers, Uber Eats is an easy way to get the food you love delivered, doesn't sound very different. But to restaurants, again, the fast way to get your food to your customers, they're not really focused on this broader service that they can provide to restaurants. And then to Uber, I said that Uber has a network of more than 2 million drivers who can also deliver food. So their focus and one of their differentiators is they actually don't have to worry about the supplied side of drivers the way Caviar does. And this leads to some, you'll see as we go through this, some pretty different product choices and probably some pretty different strategy choices. Cool. So let's jump into the exercise. Great. Thank you, Nick. Okay, so we are going to do an exercise. That means the rest of this, after this part of the presentation, is going to be interactive. We're not going to be up here talking. You're going to be getting together into groups and doing stuff, like actually having product conversations and trying to come up with something in the end. And that's what I'm going to describe right now. So imagine you're on the Caviar Exec team and you see a chart like this, which is actually a chart that we found online. It must be true. Yeah, so, right, it must be true. Daily active users is not revenue or anything like that, but still, this seems troubling. And I would expect as a PM of Caviar, when the leadership team realizes that this is happening or something like this is happening, for them to come to me with some problems that they think are causing this, that if I come up with some solutions for, that we might be able to fix the situation or make it better at least. So we have like three versions of what the Caviar Exec team might bring to me as a problem that they want me to offer or think of solutions for. So the first one is about the sign up and the checkout flow. Uber Eats is just way more frictionless and I'll talk about that later. And this executive maybe thinks that this is what's causing the problems for us when we compete against Caviar in the marketplace. The second one is the executive thinks that like Uber Eats is stealing our low end consumers, but our high end consumers less so. So they've done like a segmentation analysis or something like that. So what should we do? And then the third prompt is Uber Eats is now getting just as good at helping consumers discover new restaurants to try as we are. And that was like something we used to be the best in the business set. So oftentimes in the scenario in reality, I think PMs can play a role in calming down leadership and preventing the company from overreacting to competitive pressure. But for the sake of the exercise, let's say that our instinct as PMs at Caviar is we should do something. So let's think of things that we could do. Question? It could be a different thing. Well, in this exercise, like, okay, one of the questions that you could or one of the things that you could present is here is some here's some stuff I would research either by looking at these specific metrics of our existing users and customers or by doing some kind of qualitative method or something like that to try to learn this. You could also state assumptions like that that we learned this and this allowed us to this like made us lean toward this solution. So yeah, I think that's fine. So I included in this deck kind of a problems first type of framework to approach these problems with with some like leading questions to ask yourselves that might help you develop opinions about what options are good options and what options might not be as good. You can use whatever framework you want if you prefer or see an opportunity to use like the neary all hooked framework or like the startup metrics for pirates framework or whatever your preferred thing that you want to try is you can go ahead and try to use that. I think this could be like a useful way to start though to get organized. Okay, yep. Right. Right. So then the outcome of this exercise is to look at the products themselves and then come up with a few options to present to the rest of us like the entire room as if we were caviar execs. And I'll go into a little bit more detail there before that. So you've all installed the apps I hope install caviar and install Uber eats if you haven't already. Best if you don't go through the signup flow until you're like in critical thinking mode. But if you already have it's okay. And but we're still going to like prime you a little bit and go through some of the things that we thought were interesting when we went through similar places in these two apps. We actually both ordered last weekend, Nick from caviar me from Uber eat or me from caviar Nick from Uber eats and we both randomly chose Burma superstar. So it's like it was not planned. So we thought that it was kind of interesting in onboarding that caviar doesn't have any kind of like login with Facebook or any other third party SSO option. And we also thought it was interesting when onboarding into Uber eats that we already had the Uber app so the Uber eats app already knew who we were so we didn't have to do any kind of like email verification at all. We just had to say yep, that's who I am and then went into the app. caviar payment is kind of a similar situation where caviar does support Apple Pay in addition to manually ending entering credit card which is not pictured but Uber eats has the advantage of having your credit card already. So again, like less friction. So that's like very relevant to the first prompt and possibly others. Okay, and then when you go to Okay, so this is the Uber eats screenshot where it already has your credit card. When you look at the fees that caviar and Uber eats charge, caviar charged me $24 on a $98 bill and caviar charged Nick or sorry Uber eats charge Nick $5 on an $89 bill. So there's a significant fee difference. Caviar also has the fees split into two, two types of fees. So we thought that was interesting. When you look at the categories, they have some categories in common and they clearly have some exclusive deals with restaurants. The caviar on the left has actually has an exclusive by caviar category in the app where they're calling out some like pretty nice like $2 sign and above fancy sit down restaurants that you can order from. Whereas Uber's exclusive deals for Uber eats seem to be like low end like mainstream franchises like McDonald's. Interesting. And then caviar's home feed. The home feeds look visually pretty similar, but we thought it was interesting that caviar mixed a bunch of different like pitches in their home feed like, you know, delivery in under 30 minutes with Chinese in the same feed. Whereas Uber eats, what we thought was interesting here is that when Nick opened this app after ordering from Burma superstar, his like second or third module in the home feed was maybe you want some more Burmese food. And it also didn't include as much like type by type food type by food type modules, but it did include like restaurants specific modules in the feed. Okay, so at this point, we're about to jump into the exercise. So we're going to leave this slide up so that the groups that are focusing on a single prompt like can see the prompt. Cool. So this is again, this is prompt one. The executive says Uber eats is beating us on sign up and check out friction. What should we do? Okay, so we came up with a couple ideas. We looked at both apps. And some of the things that we noticed, some of the things that we noticed is that with caviar, it's probably a little bit more frustrating to sign up with than Uber. You have to create an account, input your address after choosing a share location. And then you have to manually input your credit card and information. You know, no no kind of quick options like taking a picture of your credit card. Whereas Uber eats somewhat easier to sign up, especially if you have an Uber account already, you set up your Uber, if you put input your Uber account, you basically have all your credit card info already stored and it's ready to go. You can literally swipe something and have it ordered in two minutes or less. Okay, so I heard one option in there potentially and that's the credit card photo thing. Yeah, so the things that we thought about in terms of like, you know, the question is how do we reduce the sign up checkout friction? Things that we looked at our number one, I think you'd already kind of gave us like the softball and said Facebook sign up. But we also like the idea of guest checkout. We also like, you know, maybe options of other payments for like, say like Venmo, Venmo, you know, taking a picture of your credit card, even allow you to do a lot of progress browsing to make, you know, ensure that people even just want to find the food that interests them. You know, really, we should drive account sign up times under two minutes. Okay, so I'm going to kick it to the, I'm going to ask the first question and then kick it out here. So why do you think they don't have those other payment options? It seems obvious, right, like PayPal, Venmo, etc. Like a lot of other sites have it. Why do you think they have chosen not to assuming that's a conscious choice? I really couldn't answer that. And why they why you think it's easy? Yeah, I mean, when I think about it conceptually, why do I use food apps? It's usually a situation, maybe I'm working late, or I'm coming home from work and I'm just really, really tired and I just want food quick. And I want it to happen now. You want to be able to load. Yeah, it's almost like the Steve Jobs rule where he said he wanted everything in the iPhone to be accessible and like, say three clicks or less. And I think that's really the way it should be with eating apps. All right, first question. Okay, yes. And wait, in real life, or here you pretend? Oh, she's gonna get really I would agree completely. I mean, I already have a square account that I've used for other things. And, you know, that'd be a perfect way. I don't know. I've looked at this a total of 10 minutes. Next question was over here, I thought I saw one. Okay, no, I saw one just here, man. I mean, I'm sure this is probably on the agenda. There's one integration, state time, Facebook signed account. You know, many other apps out on the market, I'll allow you the option to sign up via your Facebook account where you can have your details already loaded into the new application. Yeah, so there's like friction, like lower friction, but then maybe is there anything we could do with that data, like who the friends of the person are, or whatever else Facebook absolutely can study. I mean, Dan likes, you know, caviar, you know, and all of a sudden that starts appearing in trend speeds as well. Okay, last question. So like the big issue is the credit card, I think it's a good idea to get the account created. But I don't think Facebook stores your credit card data. So I don't think you think I've ever put a credit card associated my Facebook. Yeah, but with your ID for Apple, which does have credit card data. Yeah, Apple pay. I mean, you need to offer different payment options. And like I said, you the goal, you can you do many different options. But really, you want to drive your sign up time, right, like under two minutes. And you want someone that already starting to look for their food, which is under the credit card is a big issue. Somebody already has a credit card. But the other part we found out you have found frustrating is not only can you say share my location with the app, but then you still have to put in an address of where you want to deliver it. Well, and if you're working at like a business location, you might not know the exact address off the top of your head, it just adds like small little frustrations that make you say, I'm just going to go with Uber Eats. Thanks for volunteering. What was your name? Dan. Dan, thanks, Dan. Alright. Alright, we're going to go 123123 until we're out of time. Right. Okay, so we need to volunteer from the two group. Alright, first hand. Come on up. Alright, so the prompt again is Uber Eats is stealing our low end consumers. What should we do? So our team back there, we're trying to solve the problem to incentivize low end customers. So we're thinking about the total cost of the caviar purchase. So that includes from delivery to the items for each entree. So we try to solve that by when you open up the app, we provide you with a list of restaurants offering free delivery. And how we do that without burning all our VC cash or like squares money is by kind of reverse surge pricing. So when a lot of people are ordering from the same restaurants, we can optimize those driver routes to deliver it to cheaper. And so that's why the curated the list at the top that offer free delivery are from popular restaurants. So that's the idea of how we can lower the cost. Alright, so Dorsey here. So, you know, squares business is retaining restaurants. We don't actually want low end customers. Why would we focus on meeting their needs? That's a really hard question. Another, another exec told us you broke the prompts. CFO said they wanted more low end customers. So I think like just prompted, they said they are stealing our low end customers, which we do. And I think part of the question is, should we go after those low end customers? Well, what do you think? What's your gut? Should they? Yes, I do think so. All right, because it's like an on ramp to future. It's a very large market that we can satisfy with the solution. So I think it's something that we should great. Okay, question there. Oh, yeah, yeah, we have an answer from the crowd. Do you want them to be your customers when they graduate and they actually go give jobs? So definitely that's something that you should look at. I'm sure there's a marketing firm that has told you that what it is is stealing your customers. But the first question would be do you want to go after them? And second would be if they are students, we do want to go after them. All right, great. Okay, another question. Sorry. Go for it. Another answer. Kunal. Hello. So let's see if it's worth it. Yeah. Oh, there we go. Are you guys a caveat considered adding ratings? That seems to be something very easy to find. What do you think? We're targeting low end customers. Okay. Yeah, well, that's a great idea for high end customers. Okay, maybe we have time for one more question for this one. The feature was like higher volume consolidating orders into the like same rides basically, right? And then offering free delivery for those and free delivery for those thoughts. Okay, I okay. Wow. What do you think of that? That's a good idea. Yeah. Okay. So then final challenge, like if you had to choose between your feature and that feature, how would you choose between delaying orders and yeah, slow orders that are cheaper or consolidating orders? So we thought about the idea where if you schedule your delivery four hours ahead, then we can consolidate orders. But when you think of when you want to order something, it's like that sudden urge of like hunger, like you see a tasty video on your Facebook feed and it's like 7pm and you're hungry, you want it now. And so you don't want to wait. It's not something that you're willing to like wait an hour or two. So you don't think users would use it? Less likely. Yeah. Because when you're hungry, you're hungry. And not to build something usually. Yeah. Great. And what's your name? Eric. Eric. Thanks so much for volunteering. Thanks. All right. Group three, discovery. There we go. Now just as good at getting customers to discover new restaurants. What should we do? Here, I'll hold it for you. All right. So we kind of tried to break the question a little bit. So maybe discovery isn't the problem. So forcing discovery impossible says the executive forcing to forcing discovery potentially inhibits the consumer experience by increasing search time duration. Lower discovery as a priority focus on speed and ease of navigation and discovery could be a byproduct. In other words, make it easier for people to find what they want rather than trying to force discovery. Okay. Nick, do you have a first question for this one? Okay. Let me try to figure out how to push back. So basically you're defining discovery as like, is it like the algorithm that you use and the groupings that you surface or like, what do you mean by like navigation instead of discovery? So discovery is kind of a coincidental byproducts of someone just going into an app and finding out that they want something. Okay, like I want Thai food today. I go and find Thai food. I want Chinese food tomorrow. The app didn't well, maybe it helped me discover it. But nothing about the app is trying to tell me to discover it. I just knew what I wanted and I got it. And in a way I discovered it. So then you must your hypothesis must be user, most consumers come into the app with intent, like at least enough intent to be like food type specific, right? Okay, I'm just thinking in terms of myself, that's okay. Question from the crowd. Yeah. And then this is where we took out the two different apps and then we looked at the feed. And actually most of them disagreed with me. But what I really liked about Uber is that it listed pizza, Vietnamese, Thai, Chinese, and it was just so the two boxes lined up next to each other. It was a lot of options really quick and easy to see. While in caviar, it was one option. And we'll say what might say Indian food. And then it said, maybe price point, cost of delivery and duration. And it was just too much to say and duration. And it was just too much information. Well, I was just thinking I want this category of food and I'll worry about price, I'll worry about cost, don't worry about all those. So you're saying there's a trade off between prioritizing discovery and helping people who know what they want, find it fast. Other questions? For discovery, you guys might look at it like the so the urgency of the consumer might be also have a budgetary restriction. Like some of my budget is 10 bucks. And like, if I find something under 10 bucks that's right there, I might be just purchase whatever is next. So I mean, you guys evaluated that as part of the discovery, like the price has not so much like the type of food or location. Did we talk about that one? Oh, apps, but they're both catered towards very different audiences. Low end customers versus high end customers is one thing, but it's it almost seemed to me like Kavya was more elitist, very specific, exotic ish restaurants, Uber was more pizza burger, you know what I mean? So if it's that hunger pang customer, I just want to, you know, I just want to eat. Yeah, they'll probably go to Uber eats, but Kavya seemed more like, I want to eat something nice and have a nice meal at home. I just don't want to step out. Yeah. So it seems like discovery of restaurants around you was something that Kavya did very well, as opposed to Uber is which is more towards this like junk. Yeah, how long, how long before Soylent is on the Uber eats feed at the top, right? Yeah. Search easier, like, you know, like you go to the first search and you see, it seemed like it was very much about that. But Kavya was more like these are like restaurant recommendations, dish recommendations, this is more like things. So what should we do? We actually felt that this was not entirely the problem, but it is how they are positioning themselves. Here's a specific question. So Kavya shows prices of like the restaurants and the dishes in the discovery feed. Yeah, you're right. Should we remove prices from the feed? I was thinking overall simplifying the get someone to the type of food that they want and they can worry about price later. Also, the type of food that they want is probably going to correlate the price. I think that is important. Okay, last question on this one. So yeah, the so discovering restaurants is probably should have included that word in the prompt in hindsight, right within the app. So it kind of like the you must believe in order to believe this problem is important, you must believe that there are enough consumers who come in with not with low intent, other than just getting food to make it worth investing more in discovery. And in my opinion, like, I talked to some groups about this, like, because caviar is owned by Square, there may be another strategic component to consider where caviar is a selling point to for sales people that want to get caviar or get square terminals into new restaurants, saying like, we'll bring you this volume, but that selling point won't work if they if that restaurant is not very well known, and they're like optimizing their discovery feed for like just general order volume. Like if they if that's their goal, then they need a different type of discovery food than Uber eats probably needs, they need to be like suggesting restaurants that aren't getting enough love so that they can retain those customers as Square customers. So that's one thought I had, but I tried to leave it open ended just, you know, or we did. That's very on strategy for the Jack quote, Jack Dorsey quote that Nick presented for sure. Okay, we got to move on, though, we got to move on. We're going to go back to a one. I want to give another. Yeah, thank you. Oh, sorry, what was your name? Keith. Thank you. All right, let's get a one volunteer and we've had three men in a row. So I'd love to get some diversity, if possible. Somebody who wants to, though. Okay, here we go. Flow was so frictionless, we had an accidental McDonald's order. Did anyone have an accidental caviar order? No. Interesting. Oh, sorry, I'll hold it for you. Hi, I'm Diana. Hi, Diana. I think the first thing that we wanted to do, which we don't have the information on is really look into the analytics and see where people really are dropping off so that we can kind of focus our attention. So is it really in the signup flow or is it in the checkout flow? So first thing, if it is in sign up, I think we've covered it a lot, but really trying to cut down on the number of required steps. So one either is doing some sort of social sign in, or maybe you don't even require some of those steps until you're paying. Like, do we need your credit card information right now? Maybe you need to explore and see that you really like what's on caviar, and then you put it in later, and then you can save it at that point. So at least you're in, because we, if you look that and see when people are in, then they're actually ordering, then it's like we need to get over that hurdle. If it's the actual checkout process, then we thought when we're looking at the fees, there were just, there was like some like health fee, there's a service fee, there's a delivery fee. And even though some of them are free, it's just a lot. And Uber each just had one flat fee. So even if we combine those fees into one, then maybe it wasn't like, feel like we're paying more. I mean, even if they're the same amount, it seemed like maybe caviar is charging more for fees. And maybe that's a reason to drop off. So let's say that we can't do much about the fees we need it for our business to be profitable. Although it's probably not. And so that's not an option. But did you happen to notice like, whether you see, so if you didn't put your credit card in the beginning, you can like get, you can add stuff to your card, you can get there without putting your credit card in if you want. But then you have to put your credit card in. Did you notice whether you could see the fees when you were prompted to put your credit card in, or whether they left it until afterward. And if you if you didn't notice that, like, what do you think they should do there for people who are entering their credit card at the end of their first order? For me, I had Apple Pay. So it already connected it to Apple Pay. And I'm actually not used to seeing Apple Pay as a connected payment type. So when I clicked it, I wasn't even sure that it was connected or just telling me that Apple Pay was an option. But I think you had joined with without a credit card, right? And it prompted you with credit card information. So I don't know if you remember seeing the fees. But I think the payment information is on the bottom. So I think you would see all the fees first. But we were just saying to combine the three fees into one line item. So a small change that maybe there could be a question mark and you like, there's a hop up and then you can see the fees, but just making it seem like there's a fee versus three fees. Seems like an interesting way to start the conversation with the lawyer at the caviar. Yeah. Okay. True. They're not known for following laws though. Too soon. Okay. Okay, questions from the crowd. Questions about, yeah, when when to put in payment method? Yeah, I guess an open ended question. Why do they have Apple Pay and no other payment types other than manual credit card for you or for anyone? Okay, questions for this one? Yeah, I've even more fee too much fees or something. Okay. Oh, there was a question. Oh, yes, question. Like using a square piece of hardware. Yeah. Hmm. Yeah, they didn't have that. That's an option. But I don't even think that adding credit cards now out of a mobile phone is so normal. Like for me, that's not a barrier. It's more like when we got into caviar, there was just so much information. It's like, do I even want to go through? I can't find what I want to order when I get to the page. There's too much information. And yeah, when we were on Uber, we like checked out by accident. It was so easy. There's something to that where maybe there's just too many steps because my credit card information was in caviar, but I still didn't check out on accident. So it's funny that you accidentally checked out, but like, how did Uber do that? Was it like tricky? And do you feel like they got you or something? Or yeah, I mean, they're known to be, right? So do they have a pattern in there? Like, okay. So you're just used to it being too hard. It's no confirmation, either. Yeah. To the cancellation rate. Okay. Yeah. Yeah. Could you cancel? I had a hard time canceling. Oh, difficult cancellation flow. I got a big mic. So I'm going to ask a question. I'm like, head of user operations. We have all these people that are upset that they're ordering. Should we add a confirmation step to the caviar? No, Uber, Uber, right? Yeah. Well, this is for caviar. Okay. Sure. Sure. Yeah. Confirmation, either. Sure. Let's say we have the same feedback. Should we address that problem or not? And ignore the users and keep the orders? Sorry, I'll speak up. Sorry. The question is, should we add so users are writing in and our customer support is telling us they're unhappy about accidentally placing orders? Should we add a confirmation step is the question? I mean, it seems like at that point, you're actually getting people through the process, which is great, because we're not even at that step yet. So maybe that means we've done something good. Are there any maybe? Oh, yeah. So adding cancel with grace period instead of the yes, no. Why is that? Okay. So you think that's just also just like a nicer user experience? Okay. Yeah. What do you think Uber's board would say? Or I mean, okay. Cool. Yep. Yeah, I have a question. So assuming you can only optimize the credit card capture so much and it has to exist, how do you tell when is the best time to force user to enter it? Whether it's on registration or when they're about to buy something. I'll take this one. A B testing. Okay. So your question is, let's say you have the choice between prompting for credit card information during the sign up flow or prompting during for credit card information in the checkout flow the first time when you don't have it for caviar. So and the the answer here is like a B testing. Do you want to like, I can I can elaborate on that if you don't want to. But if you have it like an AB test design in mind, maybe you could say it. I mean, I guess you could see what the drop off is. If you have checkout, if you have the credit card information in sign up versus when you don't and see if there's if there's a huge drop off, then maybe that is problem. I still don't think that adding your credit card information is a problem. It's like, I don't want to have to enter my email address my address that might be in like Facebook or something else. I feel like those fields cause me a lot more pain. And then even just having the auto fill that remembers from your phone or browser would be great, which it didn't have. So yeah, I would definitely AB test this also. I would have the, you know, add credit card step if we don't have it in the checkout flow regardless, because we need it there. But I would AB test whether to put it in the sign up flow. And rather than like I would definitely look at the drop off of that step that that step is causing, but it will exist, right? Because it's one more step is one more place for people to drop off. But companies that have written about this and an AB test or two, in my experience have noticed that when you put a frictiony step in sign up flows, sometimes you filter out only low quality users that would have been worthless anyway. And you actually like improve the experience of your app even depending on the app, because only high quality users that are likely to convert or post if it's a social product or whatever are getting through. So it's important with this, especially sign up flow, AB tests, I think to like look at the core metrics of that app, like order volume or lifetime value of customer or something like that rather than just like the funnel metric. But you need to look at that too.