 Hey guys, I'm Rihanna, Rishi. Today we're going to talk about how not to lose your customers in 14 days, and so we're really going to focus on growth, what it means, and sort of walk you guys through that. Feel free to interrupt us with any questions you guys might have. It's probably going to be a lot more exciting than listening to us talk the whole time. I also might throw in questions randomly. Yes. Cool. So I'm Rihanna Faez. I'm a Senior Product Manager at Box. We're right now managing the monetization and growth of our Self-service experience. So what that means in English is basically you can buy Box online, And there's a lot of different flows for you to do that. So my team owns that. And then we also own the free trial experience. So when people are testing us out, seeing if they want to buy, We kind of have that as a playground to test in. Hey guys, my name is Rishi. I'm a Senior Product Manager at Xenofits. I also work on our growth and activation team covering a lot of The same areas that Rihanna talked about. So we have a 14-day demo sort of sandbox that potential customers can use. And also spend a good bit of time with the sales and marketing team Sort of understanding what their needs are and figuring out how does The growth function properly serve both customers that buy on their Own and how they also buy through sort of the sales process. Do you have the manager? No. Well, I don't manage any other product managers. I have an apm beneath me. But I don't directly manage her. But I kind of own the strategy and then she helps me build. Yeah. This is so weird. Okay. All right, this one's for you. It's going to be some strange hand-offs the entire time. But I think you guys will get used to it. So before we get started, for a quick agenda, First we'll cover some basic definitions. Because I think there's a lot of confusion in general about What is growth? What function does it serve? How does it fit within an organization? And then we'll just cover some high-level lessons around Sales and marketing, analytics, and product-led growth as well. I think you've got the clear, though. Cool. So I think my favorite definition of growth, And I think OpenView Venture Partners, Which is a fund out in Boston, Has a really good definition for it. It's really a go-to-market strategy that relies on features And usage as the primary drivers of acquisition, Retention, and expansion. So I think about Slack as a company that's a really good Example of this, where their growth came very organically. They're individuals in an organization that were using The product over and over again. They would invite other people in the organization to use It again. And over time, the organization sort of had to adopt this tool Because everybody was using it. So I think about this as sort of the new strategy for a lot Of SaaS companies in terms of really acquiring new customers. It's becoming very organic. It's no longer as top-down as it really used to be Ten or 15 years ago. Sure. So let's kind of talk a little bit about what growth is And how it works with other functions. I'm just going to give this to you. So usually when we talk about growth, We look at the intersection of marketing analytics and sales. And really it's how you support those other teams And how they can also support you. So kind of deep diving into sales. Usually the way you'll do this is you'll talk to your sales reps And kind of really understand how the company is selling Currently to your customers. And so one tool to use is customer segmentation. I know this is kind of tiny, but that's the first step. This will tell you how your company is currently selling to Customers. So usually for enterprise, this is done by like company size. So at least a box, we look at like our zero to 25 bucket. And we usually serve them online and through our self-service Channels. But then obviously larger companies, the larger they get, Will have more human touch, more account executives and Sales reps assigned to them. And they're really great resources for figuring out, You know, why people purchase from you and what sort Of takeaways they have from talking with their customers. We also can do win-loss analysis after we talk to our sales reps. Now we want to talk to our customers. And here the biggest takeaways are figuring out the customer Purchase process and really understanding, you know, what was The problem they're trying to solve for when they came to Your product and how you're solving it for them. And really the goal for this is really aligning your Selling process to their buying process. And lastly, you can also help out your sales team by Building sales enablement tools. And again, at box, what this means is that we actually have A tool that allows sales reps to use to generate transactions. And for example, if you're buying 10 seats and you want to Make sure that you can manage your content on the cloud And you want to buy maybe like regional data storage, you Can sort of create that own transaction as a sales rep and You can give it to a customer. And so these are all ways that sales can support you and You can support sales. So once you kind of have an understanding of how customers Are purchasing from you, you can align your process and Then do flow conversion optimization. So that kind of gets into the intersection between sales And analytics, which we will talk about. Cool. So let me trade again. So i think one of the more interesting things that i've Learned anyways in spending time with the sales team is that It really sheds light on what a lot of your data actually Means. So one of the things we Found out a few months ago is we rolled out a new demo Environment for potential customers to play with. So you plot the form on the marketing website, you create A password and we give you a sandbox with fake data. So we make hr software. So you can hire employees, On-board employees, manage pto, stuff like that. And one of the things we noticed was we were giving folks A 14-day window to play with the environment. But 92% were buying on day zero or day one. And we're like, man, that's awesome. We're crushing it. We don't even need 14 days. Maybe we should make it two days. And then when we went and talked to sales, what we discovered Was people weren't buying immediately. They had actually done some due diligence in the process Leading up to even creating an account. So what our data was telling us and what the market was Telling us were actually two fairly different things. But that's something we wouldn't have known. And we might have actually made some bad decisions around Flow optimization had we actually done that. The second thing that we've discovered is it helps a Lot around ab testing. So i think there's a lot of Good content out there around the mechanics of ab testing And the science of ab testing. But there's a lot less Information about what type of test should you run. What type of question should you ask. And i think that's where sales can also be particularly Helpful and we'll provide some more information there. And the last part of analytics that we'll touch on a bit later Is how to think about your funnel. Either sort of the Purchasing process itself or the end-to-end customer Experience from search engine marketing all the way through Purchase all the way through customer retention. Yeah, absolutely. And i think one of the biggest Ways from this whole thing if you don't listen to anything else We say is that ab testing is one of the most important tools You can use in growth. Running experiments will have Another slide or five on this. But yeah, pretty much just Remember that. Write it down. So once you meet with your analytics team and really use That data to understand your customer, it kind of moving Along into marketing, you really want to develop your User personas and understand who's coming to you with What problems and who they are. So when you're developing user personas, you want to humanize Them as well so you can give them names like lonely larry. Maybe he only wants to buy one seat and he has no friends to Help store his data with. Maybe stingy steve. These aren't ones we actually use, so don't call me on that. Stingy steve who only will buy the cheapest plan. So really helping humanize that. You can even use a photo to Really drive home that this is the product that you're Building and this is who you're building it for. This is a tool you can use outside of growth but it's Really important for when you're thinking about how people are Buying from you. So let's just drive into marketing. So again, kind of with the user personas, you can do customer Targeting as well. So thinking about how you Communicate to your customers and who they are. You don't want to just blanket e-mail everyone the same Thing. Maybe like, i don't know, Someone came for payroll, you won't be trying to sell Them healthcare. I don't know what you guys Are doing. Exactly. So really thinking about The problem and who they are and having the same conversational Tone or what they expect and meeting them there. And then there's also great ways to do this through e-mail Nurture. So e-mailing people as they Go through the product, really helping them find the value That they need. And then also in-app messaging To really get them to engage while they're in the product. And then lastly, another great tool you can use is Reactivation campaigns. So the cost of acquiring an Existing user or one that has already left you is a lot cheaper Than a new user. Do you have a question? Yeah. Okay. Let me just finish like one second. So it's always best to, like, if someone leaves you and you Kind of can figure out why from a cancellation reason to Reach out to them again. Go for it. So on personas, how many different personas are you Optimizing for? So i think it's different for every team. You don't want to have too many personas because then You're trying to make too many people happy and then You're really losing the benefit of, like, what your product does And who it's, like, advantage to. I think usually i usually Aim for, like, four, maybe five. It's still a lot. But i know you have some thoughts on the ideal user. Yeah. I'll cover this in a little bit. Yeah. Like our analytics slides, but we probably Have three to four personas as well that we, like, really Think about as, like, big buckets of users. Cool. I think this one's you. Sweet. All right. Cool. So i think the first area where We're going to dive a little bit deeper is around sales And marketing and what should the relationship be and Functionally and practically that a product manager has with Those other functions. So starting with sales, i think There's one of the most important things i think you can do And it's actually very simple to spend time with them. I think on any given day you should be chatting with Sales at least three to four times probably. And by that not, like, formal meetings or, like, data Gathering exercises, but you should always be in the loop With them. They should be coming and Bothering you with questions about the product. You should be bothering them back and being, like, hey, What about this? What about this? What was the last deal that you lost? Why did you lose it? So i think having those, like, tight interpersonal Relationships with folks on other teams is probably Something that's, like, underreported on all the Medium blogs about growth, like, product management. And the second thing i think is just digging into the Data, like, digging into sales for your crm of choice And doing this on, like, a weekly or sort of bi-weekly Basis and really understanding why are you winning and Why are you losing and just read all of it every single Week or every single two weeks and you'll get, like, a much Deeper understanding of how our potential customers Thinking about and evaluating your product. And i think more important to that is who are you actually Losing deals to. And that actually, that becomes Very good knowledge for you to then go and do competitive Research and analysis and take that data and share it With the rest of your product team and start to place Bets around where you think the business should grow. Because sometimes, like, your support calls will be And you can actually really understand, like, the real Problems people are having. And it's really hard, i think, As a product manager, like, you're constantly looking at Data or you're looking at the product and you're, like, Okay, this aggregate, 100 people having this problem. And you just put a number to it. But by actually going Through and reading the support tickets, you really Understand that these are people having real problems. Cool. And i think as important as spending Time with sales is really spending time with Marketing, both sort of the folks and the work That they're doing. And i think a lot of people get Caught in this trap of, like, i need to optimize Everything, right? i need to have four personas, three Tracks per persona. If they click the e-mail Three times, you need to send them track a. If they click the e-mail five times, i got to send Them track b. And i think that's, like, Definitely the wrong way to do it. To start, just keep things very simple. When a customer signs up for your product, whether It's a free trial or a demo or they just paid For it, they expect you to communicate with Them. Like, this is totally acceptable. Like, don't send them 15 on the first day, but Like, this is very common practice. And from that, you'll actually learn a lot about what kind Of content is working and what kind of content is not Working. For example, you might want To have a mix or really just pick one. You might want to start with informative content. And then, depending on where they are sort of in their Flow, you might want to give them things to go and do, As opposed to learning about a particular feature. So i think just starting somewhere is really better Than not starting at all. And, like, you don't need Something as fancy as this, like, stock photo That i found. But really just, like, Start simple and then you'll really learn quite a Bit from there. And once you have a good Baseline, you can start to run interesting tests. And to go to that point, for, like, the four user personas, That's, like, definitely help that i've gotten from my Marketing team. Day one, you want to be Running tests, right? Like, that's really, like, What our value add is. And, like, being able to do Personas is if you have a time thing. But if you can get that support, it's a lot easier. So the last piece about sales that we'll talk about is Your acquisition strategy. When you're in finance, We always say, like, to divest, to do something Different. When you're in finance, we always Say, like, to divest your portfolio and make sure that You're, like, diversifying your risk, sorry. And the same goes with your acquisition strategy. So there's a couple of different tools you can use. Search engine marketing, which is your paid. Like, think about when you search for something in Google and you see a sponsored ad. Search engine optimization, which is just your regular Search results. But there's a lot of hacks that You can do. Sorry, i told my team i wouldn't Use the word hack. A lot of tricks that you Can do for this, which are, like, optimizing your Keywords, making sure that you have the right titles and Your headers, making sure that you have good content, And making sure that people are linking to you as well. And so we have paid social as well, which is your LinkedIn, your facebook ads, and your paid display, Or your display ads, which are just, like, click-through Ads, like, on your screen. And here, the real thing I'm trying to drive home is you want to make sure that You're diversifying across these different sources, because If one day one of these stopped working for you, and that's The only thing that you're using, you're going to stop Getting enough users down your funnel, and it's really Going to screw you over. And the test that we're Running is, like, not going to do anything. So, again, hopefully most of your marketing team will be Supporting you, but i know in a lot of smaller Startups, it's sometimes you're doing all the things. So this is something to keep in mind as well. One of the other things to keep in mind when it comes To acquisition strategy is you really need to be mindful Of what kind of customers are you getting. So, for example, if I can get a two-person company by Spending $100 on a pay-per-click ad, or a PPC ad, Or I can spend $500 to get a 50-person company on a Pay-per-click ad, I should probably spend the $500. So you want to be really mindful of sort of where You're, like, drawing that line and what numbers You're really trying to aim towards, because you might End up optimizing too far up in the process and kind Of screwing your customer base later down the line. So, or what might end up happening is you'll Acquire a bunch of customers that fall off really fast. And this is something that I think we saw last January and we were running some ads around, like, A new concept that we were looking at. We were hitting all of our numbers. Benchmarks were really good. Then we looked at day two Or day seven retention, but we realized they weren't Coming back at all. They're spending a lot of time on The first day, but they really weren't the kinds of Customers that we wanted to be getting so we turned it off. Right. And to speak to that a little bit too, Also thinking about strategically where are you Going to drop people into your, like, landing pages too. So, you know, you might be hitting that great customer, But if they're not seeing the right content that They want, maybe they even drop off before they get into Your funnel. So, again, the example I use is If you're at zenfits and you're looking for payroll, You would drop them to the payroll landing page. You wouldn't drop them somewhere else. So, really being mindful of how you're getting people where They're going and the quality, I think, is more what you're Talking about as well. So, I think this one is you. Sure. Maybe. I don't know. I'll take it clear. Cool. So, let's spend some time talking about... I lied. Sorry, guys. It would be way less awkward without this. So, okay. So, we're going to talk about analytics. They're also a great team to work with and also learn from. But that being said, don't bury yourself with data. I am, like, the worst person because I'm so guilty of always Asking for every possible dashboard ever. And so, like, I think my analytics team is hiding For me right now. But it's true, right? Like, if you sign into your computer every morning and this is What you're looking at, sure, that's great. You're being, like, very on top of what's happening. But at the same time, like, how much is, like, your sales by Segment going to be changing from day to day in certain regions, right? Obviously, it's really good to look at for, like, product health. But, you know, overall metrics don't shift overnight. This isn't something that you need to really be concerned About unless you're worried about an outage. Instead, what is really helpful for analytics is really diving Deep into specific questions. So, for example, in our sales Enablement tool where our reps can actually sell or generate Links for our customers, one of the questions I had was, like, How many sales reps have actually used this tool, right? My dashboard was, like, sure, that's fine. But, like, no one had really been using it in the past Couple of weeks, right? And so that told me that I need To have trainings for my sales reps so that they know What to sell. Otherwise, we're working on this tool And we're spending all this end time and no one's using it. So being mindful of the right questions are to ask. And then doing those before you ask for, like, a three-month Dashboard that will probably be useless by the time it's done. Sorry, team. One of the tricks I actually like to use is build the Dashboard once. And then, depending on what tool You're using, so we use mode analytics as, like, one of I think we use, like, all of them, actually. But one I prefer is mode. Just sort of set, like, a recurring Schedule reminder and email yourself the dashboard every morning At 9 or 10 a.m. And that way you're not, like, constantly Hitting refresh on this page or, like, checking it all the time Because it's sort of wired into your mind that, like, I know I'm going to get my high-level metrics every morning at 10. I'll, like, check it on my phone, and then I'm good to go. So that's one way to sort of prevent yourself from, like, Getting just drowning in, like, dashboards. What did you think of, like, so I used to work at Zenafits. That's how we know each other. What did you think of when we put the dashboards on the TVs and that would cycle through? Did that actually help? So I don't think that actually helped. I know a lot of Companies do this. I was, like, thinking about doing it And I was, like, did I ever actually look at it? Because, again, like, the same problem happens is that the Metrics aren't really changing. My total sales are going to follow, like, roughly the same Trajectory. Sort of, active users are going to Follow the same Trajectory. And, like, these numbers Don't fundamentally change until you make a fundamental Change. And before you make the fundamental Change, you're going to be looking at something much, Much deeper and that doesn't really warrant, like, A company-wide dashboard because they probably won't Fundamentally change. All right. I'm probably going to cancel that request. Cool. I think I have another one. No, I don't. Cool. So one of the tricks I think a lot of folks fall Into or one of the traps, anyways, and myself included Is don't sort of optimize prematurely. I think we're sort of stuck on this concept of, like, I need to have a funnel. I need to know where my User is starting, where they're going, where they're Falling off, what path they're taking. But I'd actually argue that this is the wrong way to Start thinking about it. And you should first know Exactly who your customer or customers are before You start thinking about this, like, fairly Complicated funnel. And there's actually a really Good tip that I read a few months ago. It applied now just like software companies. It applied to all kinds of companies. And it basically said to start with your sort of ideal user Or start with your ideal customer and really ask yourself What makes this customer successful? And once you have an idea of what that is, work Backwards from that point in time and figure out what are The n number of things that that user actually did in Order to be successful and then start building your Product flows. Because now you've sort of defined Success on the tail end, right? I know user a is Successful when they do this. And now I also know how to Get them there. And then you can orient a lot Of your analytic strategy around how do we get people to Take progressive actions that get them to that moment of Happiness or that moment of truth instead of starting With the funnel and making the button orange or Something. Yeah, this definitely happens a lot. And I think it's kind of interesting because I think the More analytic strategy around how do we get people to take Progressive actions that get them to that moment of Happiness or that moment of truth instead of starting With the funnel and making the button orange or Something. And honestly I feel like a lot of us are guilty of just Doing the optimization because it's really easy to be Like got rid of this one field like 20% changed to 15%. I'm definitely guilty of that too. But as you know. Yeah, this definitely happens a lot. And I think it's kind of interesting because I think The more analytical you are or want to be the more you're Guilty of this and so you end up trying to optimize Everything or you ask almost like I'd argue too many Questions and this actually becomes counterintuitive to The goal that you're trying to accomplish which is how Do I acquire more users how do I make more users Successful and I think it you get lost in the weeds And I'm very very guilty of this so it's it's You have to pull yourself back every once in a while And make sure they are asking the right questions to Begin with. Cool, so I think the next part we're going to Talk about is like product led growth. Before I jump in, do you guys have any questions? I know we're kind of going really fast. I'm covering everything but we can answer. No? No? Okay, fine. Make me continue. Okay, product led growth. So we've talked about sales. We've talked about marketing and analytics and now Is like probably the stuff that you guys are most Excited about which is like the actual product Demeat of what you can do in app to lead to growth. So first of all, I think probably the hottest Topic in growth right now is onboarding although I might be wrong. At least that's what I read about every day. Anyway, so onboarding is a really great tool to once Users have gotten through your sign up through the Purchase or if they're in a free trial experience They're now in the process of getting to the Product. So here's the real test of how good is your product? Is it solving the problems that they came here for? But that doesn't mean that they need to just be Dropped in and have no idea what they're doing. In fact, it's really great to have onboarding to help Them figure out, you know, this is why you came In here. This is the part of the product that we're Going to be talking about. So here's the real test of how good is your Product? Is it solving the problems that they came here For? This is the part of the product that will solve your use Case. So at box, one of the tests I'm running right now is Actually an onboarding flow in our free trial. And what we knew from looking at our data is that a lot of People come to us to figure out how to manage their content In the cloud. And so one of the things that we did in our onboarding and One of our first steps is have users download box drive Which is our tool to do that. And we saw a 30% increase in that action that, like, we Wanted people to perform for, to solve their problem, right? So it's not just about what we want you to do, but it's Actually surfacing what they needed to get the goal that They wanted. And so after we saw that, we knew, okay, that's not the Last test we're going to run, obviously, but we know that We're on the right track. And so all our next tests are going to be, like, how Can we surface the right action to the right people And target them correctly. So I know that box at least uses a free trial for Our onboarding, but I know that you guys do Something a little different. Yeah. So zenith says something a little bit Different in that we actually thought about doing a free Trial, like a few months ago, actually, when we were making This decision and the reason we didn't do it is because Honestly, like a lot of our product is hard to set up. And we noticed that users were, it took users a lot of time To go through, like, very long setup flows to, like, properly Set up all the things. And so what we did was we decided to really flip the model. So we said, instead of trying to shorten onboarding, What if we showed you what the product could be if it Was fully set up. And this is the sort of interactive Demo or sandbox that I've been referencing for a while. And because the original problem was really hard and it would Have been a very expensive problem to try and solve. So we basically said, okay, let's flip it and let's show, Like, let's be very clear with users that you are playing With a thing that has fake data, has fake settings, Fake everything, but you can click all the buttons, You can see what it could be like if you chose to buy it. And that was actually, like, a really interesting concept. And one of the things that we discovered sort of within The first two weeks of running that was actually speaking To onboarding, we noticed that users were clicking on the Buttons left to right. So when I was looking at sort of What routes or what pages are users clicking on, we noticed They were clicking on a settings page a lot. I was like, that's weird. You're in, like, a demo environment. Like, there's nothing to set up. We already did that for you. And then we sort of looked at the dashboard again. And we noticed the settings page was the first thing on, Like, the top left, or top left for you guys. And then they were going to the next tab and the next Tab and the next tab and the next tab. And so what that told us was they actually don't know where to go. And so we sort of combined that analytics approach with We run full story. It does, like, auto screen capture. So we watched a lot of full story as well. And noticed, like, the exact same pattern of users landing on The demo, not knowing where to go, not being oriented, Sort of clicking left to right, like oddly sequential order. That sort of was the impetus for our onboarding guide as well. So we spun one up and we saw within the first two weeks Like a 45% lift in the amount of time that users Were spending in the demo environment. Because they actually, we told them where to go. And so I think to Rihanna's point, we took that as a sign To invest in that a little bit more. His idea to make it easier for you to set up a time off Policy. And I was like, no, we have to do it right. We have to set it up, but it's so true. For every different product, there's a different way To treat onboarding. And so really, yeah, To get people in there, you don't want them to spend An hour setting up a PTO policy. You want to make sure that they can play around with And so this will be different for every company and every Product, which I thought was really interesting. So for us at Vox, what works best is like pre-populated Files that are actually guides to how to use a product. And so I think for any company that you work for That's something to be mindful of, make sure that You really understand what your users are trying to get From that trial or demo experience. And that will point you into the right direction Of what you should give them. Makes sense. Right. And so we talked a lot about growth as requiring new users And helping them come into your product. But what's also really important is to think about the Inevitable them leaving you. So we usually talk about this in terms of churn, In churn management. And while you want to make sure That your product is engaging and that you're engaging Users while they're in there with in-app messaging And email marketing, this will happen. And so there are some tricks that you can use to try And manage that. So from the analytics side, That's usually predictive churn analytics. Looking at user behavior right before they leave you. And seeing if there's any clues there that you can maybe Stop and message them at that point to get them to stay. So usually this means like a long period of inactivity Before they drop off, at least for enterprise software I found. But at least at that point Before the inactivity, you can hopefully market to Your end user or your admins and really understand How to drive them. And in the case that you can't do that, There's also the adding friction into your cancellation steps, Which i know a lot of people have mixed feelings about. How many of you guys have used blue apron? Okay. How many of you guys canceled blue apron? Okay. Cool. So, yeah. So i don't know if this is an example Of good or bad, but it is an example of how you can reduce Your friction or actually increase your friction and Cancellation by making your users call you to cancel. There's also class pass. Anyone use that? Yeah. I cancel that, too. And so when you do that, they'll actually offer you a Discount. I don't know if i should be telling All their secrets, but they offer you a discount. And if you don't accept that discount, they'll Offer you another discount. So always wait for the second Discount, guys. But yeah, these are great ways to Think about, you know, why are people leaving you? Is it too expensive? Give them a discount. If there's a reason that they can't find the thing that They need, give them a link to your community page Or your education page and really help drive at home. This is something that we're currently testing right now. So i can't give you any insights on that yet, But it is something to really think about when You're thinking about churn. And then lastly, we kind of touched on reactivation Emails before. But again, the cost of acquiring An existing user or one that has already left you is much, Much lower. So we talked about all those Ads and all the ways that you can get people to come to you. But that's expensive. So you already have that person's Email address. You already know that they came to you And they paid for you once. So you can easily reach out To them and be like, hey, i saw that you canceled Because you couldn't store your data in mars. But guess what? We can do that now, whatever it is. So yeah, this is something to think about when you're doing churn. So actually, is it also a thing where you do premium products When they don't upgrade, but they never cancel? Are they still paying? Or you're saying they're just a free user? It's a free user. Okay. So we kind of do something a Little different for them. We don't consider them churned If they're still using the product as a free user, But we do consider them prime for upsell. So what we'll kind of do is, at least in box, if you're a free user And you try and upload a file that's bigger than the requirement, We'll be like, hey, you try to do this thing. By the way, if you upgraded, you can do that thing that you wanted to do. Free users are actually a great pool. Because again, they're already in your product. They're seeing the value. And so what's really good about that Is understanding what they want. And if they stop using your product, Is it equivalent to canceling the subscription? Yeah. So you can look at inactivity and define What churn means for you. So I think in this case, Like, yeah, we're talking about very specific, like buying Seats, dropping off. That's like, I guess, like churn-churn. But in the case of free, you can, like, define it as you want, Like a lapsed user or, like, completely inactive. That might be your churned user. Yeah. One of the definitions we use, because we have a Corporate of, like, free customers as well. So we look at who hasn't logged in the last 30 days. And then to Rihanna's point as well about how to use, Sort of, try and help sell them. We still show them all of the premium features as well. But when you try and use it, we sort of redirect you Like a buying experience. So you always know what's Available. You just can't actually use it. So we can, that gives you an opportunity to layer A little bit of, like, in-product marketing. Just sort of show them the value of what it could be. And then they can go and buy round trip all the way back. Right. And the best part about that Is making it contextual. So like you're saying, Like, if they're trying to use that thing in that place, God, I need better technical terms. But yeah, they're clicking through your product and it's like, Hey, this is awesome. I have, I can, like, store my data in Europe and it's like, nope, you got to buy it. At least you're, like, primed for that interaction. I thought I saw a question. Right. Well, so you always care about churn. These steps won't help. Well, predictive churn analytics Will help you. But for the most part, like, Trying to tell a gigantic university that, oh, no, You got to call me. I'm not available. Like, that's not how you're going to, like, get them Daught to quit. For larger enterprises and bigger Companies, they usually have specific sales reps that are Managing those relationships. So I think in that case, It's a little bit less of these tricks that we're Giving. I don't have any thoughts on that. But mostly when we're focused on growth, we're focused on The first, kind of, I don't know, 30-ish days of their Experience. And then from there, sales reps are Managing those relationships and trying to address their Needs. I'd say I had one more thing to That. One of the things I think a growth team can do to Actually support sales and marketing, especially larger Customers around the concept of churn is by helping your Internal teams understand how the larger customers are Using the product. So if I know, if I can build a Dashboard, or not, no, dashboard, go figure. Yeah. But if I can go, if I know the account manager for This, like, 10,000 person company, they might want to Know how their employees or, like, students, for example, Are using the product. And that's information I can Go and give to, like, my internal stakeholder and Arm them for sort of, like, a renewals conversation or An upsell conversation. And again, I think it's, I don't know, it's a bit blurry on whether or not This is, like, a purely growth function. I think it's something that we've ended up doing anyways Because the business found a lot of value out of it. And then I think over time you can start to systematize These kinds of things. But I think not only helping Yourself, but I think helping other teams understand how Your users are using the product is super helpful. Awesome. Actually, I think it might be. Oh, no. Any more questions on that? I know churn is kind of hot, too. Cool. So I'm going to switch gears a little bit and Talk about the importance of user research. This is, sadly, sometimes the first thing that gets Dropped off if you have a really busy day with, like, eight Meetings in a row, which is my yesterday. But it's really, really, really important to make sure that You have the user's voice in your ear at all times. I think, you know, honestly, product manager's role and, Like, real value ad is that we're advocating for the user. I'm sure you've heard that before. But it's really true. If there's a meeting where You're prioritizing a feature and, like, in this Example where you're, like, yeah, yeah, sure. Like, I think, like, a 14-year-old girl would want this, But you never actually talked to a 14-year-old girl. You're probably wrong, right? And so you really want to Make sure that you're utilizing all the tools you have Available to you to help not only inform your roadmap, But really actually understand what you're doing and who You're doing it for. And so some of the tools you Have are listening labs where you can actually observe Users using your product. When I actually worked in The apps, 14-year-old girls were our target demographic. So we actually invited, like, 12 of them into the office, Which, like, we lost all our snacks. But we did get to see Them using the app. And so it was super slow. And it was, they were just clicking right through it. And they just went right to instagram. I'm like, holy crap. This is really important. So it tells you about things like performance Optimization and stuff that you might just be, like, Looking at, oh, okay, this how many seconds it takes To load or something like that. Seems fine. Yeah, exactly. There you go. There you go. So yeah, really observing people in the wild and not Guiding them will really help you see how people go through The flow of your app and really understand how they Think about that. And then there's also the Guided user interviews you can do as well. So actually sitting down with customers and talking to them And figuring out, you know, okay, so what do you think About this feature? I know you have a sweet technique That we'll talk about in a second that i'm going to Do. And we also can do win-loss analysis, Which i talked a little bit about before, right after Someone buys your product. They're a recent evaluator. And so when you talk to them, you can figure out why Did you buy us, like, what other alternatives were You looking at. And then you can really find out Who your real competitors are, not just the people That you, like, angrily google at night and, like, Look at what they're doing. No one does that, maybe. And then also you can use your other teammates to Help filter through their stories, like talking to support Understanding what kinds of problems come through them, But also realize that is actually being filtered. So as long as you can talk to the customer directly, usually That's your best bet. And there's, like, a bunch of Other techniques i had on this note slide. But if you want to, just come talk to me after. So one of the things, same thing, like, it's hard to Make time for these things, because there's always, like, 40 other things that you have to do. So unfortunately user research is one of the things that, Like, very often falls to the wayside. So there's actually a really cool gmail plugin called mail Merge. I think it's, like, 30 bucks or something Like that, and it just hooks up to google sheets. And you can use it to mass e-mail, like, a thousand people At a time from your e-mail address. So as long as you can get people's e-mail, so, like, Either with marketing or wherever, like, your database Of choices, it's, like, a really, really fast way to Just e-mail shit tons of users and get lots of feedback, Either in terms of e-mail or setting up phone calls. So, like, one trick i've actually seen someone use is They threw a doodle link into their mail merge. They had potential customers or current customers just book up Their entire calendar for the next couple of weeks. And so it didn't interrupt anything else that they were doing. So it's just little, like, hacks, like, tricks like that, I think. I don't like using the word hacks, either. But it's tricks like that that really help you, like, Accelerate the amount of user research you can do without A lot of, like, the scheduling and, like, Administrative work that, like, usually is required to Go on, like, talk to five customers. And we got one feedback that was, like, Stop sending me e-mails. I'm like, okay, that's kind of a bummer. But, like, holy shit, we sent a lot of e-mails. So it looks good to know. Get that feedback, even if it Is negative. Yeah, and, like, the user feedback isn't always going to Be about your product. So a few months ago, I was on this thing where I wanted To meet a bunch of our prospects in San Francisco. So, again, I used the mail merge plugin, filtered my Data set by, like, local zip codes, and met with, Like, half a dozen folks, including a bike shop in the Mission. I think they sell, like, Bicycles or something. It was really expensive. Made a cool office. So when I was talking to the owner there, one of the Things he was mentioning was our sales process wasn't Actually as fast as he wanted. So he ended up going with a competitor. So he actually had no feedback on the product. He had no feedback on the demo that I spent so much time Working on. He had no feedback on the marketing campaigns. His only feedback was, hey, I needed to get Something soon, because he, like, had to run Payroll the next week. And one of our competitors simply emailed him back Faster. So we basically took that to go and sort of look at our Sales process in a little bit more detail and sort of use that Anecdo to, like, light the fire under people's feet. And then we made changes. But we wouldn't have made those Changes if one, we had stared at the dashboards all day, And two, if we hadn't gone and met that customer. And so the feedback isn't always going to be about Product. It could be about your business or your Operations. But that feedback is Super valuable, one for you to know. And second, I'd argue it's even more useful for you To distribute to other folks in the company. And you can literally just have one to somebody and tell Them something like that. And the more you can tell people, the more you can Share that information, again, it's going to come Filtered from you, probably in the form of, like, a Feature request. Yeah. Probably. But at least, like, are sharing that information, And people can understand your customer just as well. Cool. So I think when you're talking to a customer, One of the skills that's really important to pick up Is learn how to read them between the lines. Because users say they want one thing, but they Might actually want something totally different. And it's actually really hard to figure out what is this Like, nice, narrow, like, sweet spot of what should we Actually build as, like, a product team or an Engineering team. So one of the tricks, I read this on, like, a blog post. I wish I could take credit for it, was you basically ask User, when you're interviewing someone, you ask You sort of get your feedback, what they like, What they don't like. Then at the end of the interview, you sort of pose A hypothetical. And you ask, if I gave you $100, how do you Relocate that $100 across, like, problems or, like, new Feature requests and things like that. And you'll very quickly find out how your user is prioritizing. They might put, like, $90 to some, like, stupidly small Bug that you thought. Like, you chose not to fix because it was inconsequential. But in the user's mind, this is the biggest thing that's Going on. So I think just sort of asking that question forces a Potentially forces the user to go through the same Prioritization exercise. And it's really hard to get by asking folks about what They like and what they don't like and sort of doing user Studies. But it's much more tangible when you say, I give you $100 and How do you spend it? Because it's very finite, right? And this is how product managers think about resource Allocation, right? I have a team of five. I have a team of 10. How do I allocate these resources accordingly? And this is where you can use your users to really help You make that decision. Yeah. And one thing I'll add about this slide, too, is it's Really easy to get the user feedback that you want. You shouldn't do that. So if you really want to build this super cool feature, like Machine learning, I'll go up to a customer and be like, hey, Machine learning is pretty cool, right? What do you think about that? Should we build that? And they'll be like, yeah, of course. Sure. Sounds great. I don't even know what that is. So make sure you're asking the right questions, right? Like, oh, is, you know, having the right tools in Front of you more important or, like, you know, exactly, Using the $100 question, which, like I said, I'm Just totally steel because it's true. Customers will tell you all these things, but they're Not really thinking about, like, what that really Means for a team to build. Yeah. And I'll add one more thing as well. So it's not necessarily just the function of, like, the Product manager to go and talk to users, right? So edXenifits, for example, were rolling out some new, Like, advisory services. So let's say you use us for, like, payroll, and you need Help understanding how the tax filings work or how the Laws are changing in some random locale that you're in Because it's, like, a big patchwork of regulations. And so one of the things our advisory team was actually They went out and did their own user interviews. But before they did that, I was like, hey, guys, can I Can I read the script before you go out and do it? And they were actually asking questions, like, hey, wouldn't It be cool if you could have someone that Advise you on payroll? And of course, like, the answer is yes. Like, no one's going to say no to that. So we sort of, we just changed the verbiage a little Bit and actually elicited much more interesting responses. And so it's not just your job to go and talk to users, But it's also your job to, one, encourage other parts Of your business to go and talk to users and sort of Help them ask the right questions. Because everyone knows it, like, that you shouldn't do That, but you'll make the mistake the first time. Right. And you'll also probably make a mistake of taking it personally When they totally crap on something that you've done. It's really hard. I'm always trying to, like, over face. So what you try to do is just always say, like, And tell me more. Or just, like, keep them talking and really trying to React or be like, well, we have to do that because we Are under, like, the gun and we have to shift this thing. They don't care, right? They're actually going to help you by telling you that Thing. And if you act that way the first time, they're not Going to ever do it again. And that bug, that was their $90 thing. There's actually a really good story about it. I think we were talking about this one yesterday. There was, like, a team from Microsoft or it was Google. They were working on, like, spreadsheets. And there was, like, a busted formula or something like that Or, like, a busted macro. And this team of, like, ten product managers and Engineers was, like, going all around the country, Interviewing tons of users, showing them, like, The ten new features that they were building. And then at the end they sort of asked this $100 question. And what they discovered was nobody wanted the ten new features. What they wanted to fix was this, like, stupid macro that Didn't work. And, like, every user they talked to was, like, That's the one thing they pointed out. And so, again, it's really important to, like, find that, Like, really, really narrow sweet spot and ask customers The right questions. You can, you want to, okay. Sure. So, like we said, if you take anything away from this Presentation, I know we kind of just firehose you guys. Prototyping, experimenting, and failing fast is, like, The most important thing for a growth PM. It's still pretty new. A lot of people haven't really figured out. There's no specific formula that this will make you, like, A million, a billion dollars. It's more like everything is different for your company, Different for your product, and different for your target audience. And so the only way to find out what works is to test it. And so I know we kind of disagree on this point. But when you're doing tests, it's really important to make Sure that they're clean. And so I think we came to, like, a solution. Mostly agree. If it's about revenue or about conversion, like, Shit is going to get real from upstairs. So make sure that what you're doing is actually, like, actionable. So an example of how we fucked this up was on our pricing Page, we made changes to our copy of, like, how to describe One of our plans. And I think we also changed, like, where one of the Plans was on the page. And our conversion dropped. And we're, like, okay, let's not do this thing. But we actually don't know what that thing was that made Conversion drop. So usually it takes tests, like, a long time to be statistically Significant. So I think we wasted, like, one or two months of tests. Because we didn't know what was actually causing the drop. So we had to start all over again. So at least for revenue and conversion, make sure they're Clean because if you say you ran a test for two months And you have nothing to show, that's a really hard Conversation to have. But also I know you kind of have some thoughts on that. Yeah. So one of the approaches I like to take when running tests, And I think it depends on, like, how mature is the growth Function at your company? And at what scale are you operating? But one of the principles I like to use when running Experiments is, like, run a really big experiment. Right? Test something, like, fundamental and foundational, Although you might get yelled at. But sometimes it's kind of interesting. But really test, like, a core belief that you have About your product, right? So one of the things I was talking about earlier is we Tested, set it up on your own for the first time, Or maybe it's too hard to set up. And that was, like, a really difficult thing to go. I think we told Rhianna this, actually. Like, maybe, like, the products are too hard to set up. Right? And it's going to take way too long to go and make all those Changes. Like, people will get mad at you for doing that. But it's really important to, like, have that debate And that negotiation. And so, like, look, like, one, we're not changing Everything. We're running it as a test. It's also a really good way to make changes. Because if you can prove your test successful, It'll happen anyways. If you're not the growth PM, that's, like, the worst thing ever. So, yeah, it's important to have good relationships With the other PMs. We're so proud of you. But so, I guess, my point is that test big assumptions. And I think when you're at, like, very large scale And, like, minor things make a big difference, Then it's, like, more luxurious, I'd say. Or, like, more, like, you've already optimized everything, Right? You're going from, like, an extra half a percent. But if you think you can make a big fundamental change, In fact, something like 10, 20, 30 percent, You should go ahead and try that. As long as the test isn't too crazy expensive, for example. I know what AB testing is. Most of that made sense to you. For anyone who doesn't know that, it's when you have a control Or a variation. So, something with the way it is. And then a change that you've made. And then you're testing to see what performs better. So, hopefully, what we just said made sense. I just realized that. Cool. But anyway, experiment and fail fast. Always have at least one experiment running. Always one. Because that way, that's just, like, a free warning for you. And it's not really going to affect you much. If you just have that, maybe, well, It's 10 percent of customers or something like that. It's always a great, like, free knowledge thing That your product that no one else can tell you. Yeah, you always want to be gathering data. Even if you don't necessarily have the time to analyze it Until much later. You should always have something on and in the background Gathering data. And then you know you have it. So, whenever a little bit of time opens up, You can go and, like, do the analysis. It's like, we actually do this a lot. Because we, I think PMs always get pulled in different directions. But just have one thing sitting and running. And then carve out the time to do the analysis. And then make changes from there. But always have that one thing going. Cool. So I think the biggest thing, or one of the many things To keep in mind is that growth is really a joint effort. And I think product-led growth means that this person Or this team is sitting at the intersection of sales, Marketing, and analytics. And sort of having a really deep understanding of users, Of analytics, of sales strategy. And as well as being sort of deeply analytical About what our customer is doing. And I think sort of using qualitative and quantitative approaches Is a really good way to do that. But I think, I mean, it's still a really new discipline. I started working on growth maybe like six or eight months ago. Maybe a little bit longer. So I think I'm still learning a lot and like picking up a lot. It's a new discipline, right? So I think every company, every team might run this In like a slightly different capacity. Or it might be bucketed in like a different department. So like, I'm in product and I think you're in marketing. Yeah, I'm like in marketing, but I'm also in sales. But I'm also a product manager. So I don't know. It's really, it's a very like hazy line where you sit. But you're kind of doing all of it. Yeah. Oh, and one thing I wanted to outline too is, I'm actually, this might be a question for you. I don't know if you can answer it. For anyone who wanted to be a growth PM, Where do you think is the best place for them to start? I'd probably say marketing. Marketing. Yeah. Really understanding who your customers are and what their Problems are. I think actually that's not just growth. That's probably all products. Yeah. So making sure like if you want to be a product manager, One of the biggest assets that you can give to your Organization is being not just the voice of the product, I mean the voice of the user, but really understanding Who they are. Thanks guys. Anyway, this is all we really have. But feel free to come talk to us after if you have Specific questions. Thanks for joining us.