 Good afternoon. Oh my gosh, it's amazing to see so many people here So I'm here today to talk about the most underrated virtue and Because I kind of do a side gig as a teacher. I'm actually not afraid to call you out So I'm gonna get audience participation here. So I see somebody smiling. I'm gonna ask what do you think is the most underrated virtue? Sure. Yes specifically Okay making assumptions. Yeah You right here humility Yes Bingo on the nose You probably were thinking about, you know, all these other things that you could be doing product vision Business chops superior communications. You're looking at yourself and you're like check check check But I'm here today to talk about Humility Some of you Those who are probably not very Have humility right now are wondering what is the actual definition of this? And what it is, it's a modest view of one's own importance Think about that for a second Evaluate yourself on a scale of one to ten. Where are you on the humility scale? So I'm gonna spend the next 20 minutes today to talk about why I believe that humility is a number one Underrated virtue of a great successful product manager My name is AJ Arora and if you look me up on LinkedIn, this is what you'd see I'm senior vice president of product management at Disney streaming my team and I work across multiple product areas driving growth Driving commerce experimentation for our streaming services Disney plus Hulu ESPN, etc. I Also advise as a product advisor here at product school a company that I'm deeply passionate about because of the mission And I also teach a course at Stanford University Prior to my time at Disney I spent four years at Netflix on the growth team and I led the product team at audible and was on digital audiobook subscription service But Those are things you find on LinkedIn. Let me tell you about things you won't find on LinkedIn Turns out I love data if that means spitting into a test tube to find out my genetic makeup. I'll do it So I'm 99.8% South Asian Though you wouldn't know it because I actually spent most of my life in Canada and the US go Canada You see I graduated from engineering at the peak of the dot-com boom Moves to the Bay Area and have been here and around but back here at home right now So another thing about me is that I love music I get my money's worth from Spotify In turns off like you just talked to me about music tips and I'm there I like to brag about my rap summary You can see you know hip-hop and dance and the South Indian this comes through as Bungra number four Also, I Love entertainment. I love movies. It's not surprising for someone who worked at Netflix now at Disney who doesn't love movies, right? Turns out I can point to certain movies has changed the trajectory of my life Who is a matrix fan matrix fan? Oh nice It's hard to believe that this movie came out in March 1999 we're all dating ourselves here But there was one scene in the matrix the first one that literally changed my growth trajectory It's this scene right here You see in the scene Let's kick it off Operator tank. I need a pilot program for a B212 helicopter Hurry Let's go Do you guys remember that scene? Yes Okay, so until they invent a neural link that can basically just say hey download a helicopter manual into my brain I Will sign up when that's available, but until then I find myself to be a perpetual student I love learning new things and I want to find a way that like how can you learn things faster even better? Let's stop talking about me Let's talk about all of you These incredible product managers that in some respects are actually more like superheroes You see I know what your day is like You're sitting in the room being cool as Clark Kent But when things go crazy everybody goes let's look at the product manager. What do we do you change into your superhero outfit? It doesn't help That there's so much pressure on you from companies like Amazon where they have leadership principles Principle number four our leaders are right a lot It doesn't help doesn't help So I'm here to tell you Look, I know I've been there. I'm wrong a lot and I'm proud of it I'm gonna walk you through some examples It turns out that some of my biggest career successes Actually were because I was wrong So I'm gonna give you two examples the pinnacle of my you know achievement at my time at audible and what story from my time at Netflix But here's something I'm extremely proud of when I was at audible We launched a product that the Wall Street Journal called the killer app for books It's quite a compliment when you think about it books have been around for hundreds of years to say you've got the killer app You know, it's it's kind of nice. So let me tell you what we did You see we had this crazy cool idea That we take a Kindle book we suck at all the words We take an audiobook Professionally narrated and we do speech to text and we get the words there and we do some magic and we sync it together We invented something cool It was called Immersion reading any immersion reading fans out there No crickets crickets Okay, but maybe maybe if I show you an example You'll be convinced you see immersion reading is super cool in the sense that you can simultaneously read and listen So you can immerse yourself deeper into the story. Are you sold? Okay, maybe okay, let me give you a demo. Let's see if I can convince you with this book Friday December 20th the trial was irretrievably over Everything that could be said had been said But he had never doubted that he would lose The written verdict was handed down at 10 on Friday morning And all that remained was a summing up from the reporters waiting in the corridor outside the district court Carl Mikhail Blumquist saw them through the doorway and slowed his step Okay, come on Right, that's kind of cool. That's kind of cool, right? So let me tell you how this story goes We launched the product and basically you send the fire hose of Amazon traffic looks good Then this happens the product bombed Here we are Pumping through tons of AW instances Cranking through all these books trying to make this product work and nobody's buying it So our team is like heartbroken and we go back and you know what we do We actually talk to users imagine that right we try to like see can we cake this product? That's not working so well and make it successful And we had some insights and let me tell you what happened next Wish you had more time to read When you put down your Kindle pick a bright where you left off in your audiobook Amazon has a new app called whisper sync at that somehow giving people more time to read welcome to WSJ live I'm Simon console. We've got market watches Jeremy Olson here who has a story and is an avid reader and is going to explain How all of this works? Thank you for being a my pleasure Simon Well now you know how it works if you want to know more. I'll happily tell you but did you see what happened there? Did you see what happened? Customers didn't want to read and listen at the same time Avid readers never want to put down a good book So the insight was how do you take this cool technology that we built? And make it so that you know, you're reading at home You want to hop in the car you want to continue where that book leaves off? We pivoted Wasn't a blockbuster, but it's in pretty great product Amazon in their in their earnings shareholder letter They talk about it Bezos himself encourages you to try it I encourage you as well and you can see it live today We try by Kindle book today. You'll see a checkbox add audible and proud it still exists You see That was the highlight of my time at Amazon and the highlight of my time at Netflix was right here. I Was in New Delhi, India for a new product launch. I was proud to announce this new tier called a mobile tier You see what was happening is during this time We're trying to crack growth in India and Indonesia and other markets, but the growth just wasn't picking up. I had a thought I Was like our pricing just seems way off and so what we did is we basically you know work with the team to understand like look There's a basically a demand curve and we follow the price high price Look at this quantity you drop the price. We increase the quantity. It actually seemed very simple. I was like guys I think I've cracked it. This is what we're gonna do. So my hypothesis was simple We're gonna drop the price of Netflix in these regions by 50% and in turn We're gonna increase our signups by 50% it's gonna be brilliant. It's gonna work Can anybody guess what happened? Yes, I'm predictable. It did not work did not work. Let me actually tell you what happened You see we dropped the price by 50% The signups increase. Yes marginally But can you imagine what happens to revenue when you drop the price by 50% and not enough people sign up? It is scary In fact, I commend the team to have that fortitude to let the experiment run Something a magical happened The first month renewal comes through and the people who sign up on this lower plan They actually liked it and the retention was awesome and month after month These people were loving the product and what happens in the subscription business. It's not about the signup It's about the retention as we just heard and it made the day it made us that this product actually worked Here's a headline from shortly thereafter nearly half of Netflix's growth comes from Asia Pacific and that mobile tier had a big role to play Guess what the truth is I was wrong So let's talk about humility What is it? Basically, it's multi-dimensional. I want you to be thinking about, you know Openness and awareness and taking different perspectives You see any fans of we crashed anybody watching we crashed on oh, yeah, I see a couple great story and You know, I watched it. I was like I want to learn more about the we work story And so I picked up this book called billion-dollar loser and I got a great story for you Here's how not to act So you see when Adam Newman the founder and CEO was trying to take we work public the first time He went to Wall Street analysts and he was so proud. He basically professed we have never closed a building To which point an analyst from the back shouts out saying hey well that sounds like an achievement. It doesn't make any sense You see of the more than 500 different buildings you've opened The fact that you've never made a single mistake Not very Humble is it? So I want to give you some tools and techniques because one thing to say it's very important to be humble Drive for humility and then just walk off the stage But I'm actually gonna give you four tips and tricks that I think would be helpful as you think about your product journey So I want you to these are all like cute little stories around in late embracing humility Preparing to fail Calibrating your decisions and promoting dissent So let's start with embracing humility Somewhat ironic. I'm gonna use venture capital as an example here, but I'm gonna do it So Forbes has this great list. It's called the Midas list This is like ranking the VCs that they touch a startup and it turns into a unicorn turns to the gold They're just like Midas, but I was looking through that list and I found an interesting example here Bessemer venture partners one of the oldest venture capitalist firms here in the US You go to their website and we've got this cool little tab. It's called the anti portfolio And here's that it goes ordering the companies that we missed and It's a really interesting humble example where they basically present this list to you We recognize many of these names. I'm sure Apple Airbnb. You see Google You see Facebook Tesla zoom and you click your mouse over it And what it shows you is the story of how the founder or the entrepreneur went to Bessemer Wanted to raise money for let's say Intel or PayPal and they said no Only to watch that start up do amazingly well And why is that encouraging because now if you're a venture capitalist working at Bessemer or an Entrepreneur that's pitched there You understand that look they make mistakes. Everyone makes mistakes. It's cool drives humility I think humility and experimentation go hand-in-hand and if you're looking for something great to read I encourage you to check out this book this article on Harvard Business Review building a culture of experimentation In fact, they profile these great companies and there's a lot of humility there In fact the one that stands out the most is booking.com Booking.com is a role model. They're doing thousands of experiments any given time and What they tell their team and their employees is that nine out of ten experiments fail Now if you're a product manager at booking.com and you see this, how do you feel think about that for a second? I Know how I would feel I Would feel comfortable taking some bigger swings knowing that's gonna be okay And it's okay to make mistakes because a lot of things just don't work out The next lesson I want to share with you is preparing to fail Before I do that, let me first do a round of hands. How many of you do post-mortems? Post-mortems in the house nice nice nice. That's about 80% of the hands go up How many do pre-mortems? See about five percent Good, I've got a lesson for you all so I want you to look up a pre-mortem a Premorting is an amazing technique and it's been proven. Here's what happens There's this concept called perspective hindsight If you imagine what can go wrong You're more likely to predict the failures in advance And it's so shockingly simple. You should all be doing this Five simple steps. I'm gonna basically walk you through it You're working on a project. You're about halfway through and You get your team together and said team. We're gonna do a pre-mortem Half the team is gonna be shocked and confused, but it's okay What you're gonna do you put them in a room and you're gonna say Imagine the future. We just launched this product as near and dear to our hearts, but it failed miserably and Then you're gonna ask everyone to fill out some post-it notes on why do they think it failed? You're gonna discuss why they think the project failed And you're gonna review it together and I promise you you're gonna be thinking about your project differently You're gonna be thinking about making sure your customers care agents are better prepared You're gonna think twice about you know working with your comms in PR team You're gonna be fixing those edge cases. You thought maybe we can skip this so I encourage you all to be thinking like this Another trick that I picked up during my time at Amazon and Jeff Bezos talks about this a ton Is this concept around one-way door and two-way door decisions? How many of you use this framework? Okay, good, but 10% cool. So you'll pick something up here, too What's a one-way door a one-way door is a decision that is difficult to reverse It's like you go down a certain path. You can't go back. You can't put that genie back in the bottle Is how I think about it The two-way doors very different You can make a decision and if doesn't work out no problem. You can simply go back. It's gonna be okay So now when I'm sitting with my team the first the question we ask ourselves wait Is this a one-way door or two-way door decision? In fact, I've got three tips for you the first one is to identify When we're making decision is this one way or two way? If we can make this a two-way this two-way door decision, we will do that for example Can we do like an MVP? Can we iterate? How can we leverage our experimentation platform such that if this does not work out? We can just roll it back and Inevitably there are gonna be decisions that are gonna be a decision. That's gonna be a one-way door We're gonna do something. We can't turn back and For those ones you want to spend most of your time you're spend your brain cycles You want to be doing a lot of moderation a lot of discussion a lot of engagement. You really want to make the right decision critical tool Finally my last story for you is promoting descent Might be uncomfortable, but let me tell you what that means So I start off to talk talk about all these things I'm proud of all these great products I've launched, but you see this logo for that audible icon right there on this device Yep, that was me and my team. We spent two years of our life working on this really cool app for the fire phone Any fire phone users out there? No, no didn't work out So, you know, of course Amazon had this miss they talk about a lot There's another company that a pretty big miss right? This is Netflix right in 2011 the stock dropped 71% kind of deja vu. I know it's kind of crazy but the stock drops and Read Hastings doesn't interview and he talks about like what went wrong and reads very you know honest about it He's like look, you know, I Was been running Netflix for such a long time I made some decisions things are going well and my team was just uncomfortable Disagreeing with me. They thought you know Reed's been right so often. He's probably right with this idea around quickster Not so much So the culture at Netflix is pretty amazing because they promote descent and they have this concept called farming for descent And the way it works is the following. Let me give you an example. This is not an actual Netflix memo I've picked up this concept and basically all the strategy docs at Netflix are written in Google Docs What do you do? You write the memo you share it you share it broadly You turn, you know common access on and you want people to comment You want them to poke holes at your arguments? You want them to you know, just tell you where you could be sharpening up this idea how you could be testing it better This is such a valuable tool that I've been now using it while I'm teaching my course at Stanford In fact, all my courseware is in Google Docs because when a student has an idea or is poking They're actually making it better for the future students of the course And so I encourage you to have this mindset of like writing your ideas down opening up for examination and getting people to help You make it stronger Another cool trick at Netflix that actually promotes descent further is this is an actual conference room at Netflix, right? This is where when I was proposing this mobiles here. This is where I was You know the seats are filled with your colleagues people in like customer care and product and experimentation And you're presenting your idea. It's very democratic where people try to poke ideas Are you thinking about this or you thinking about that and of course, you know how to set up the experiment successfully? So I encourage you as your fourth takeaway promote the sense as a product manager and With that Thank you so much everybody. Thank you for joining us here at product con Celebrate the learnings be humble. Thank you