 OK, this is good, right with the lights on. Oh, cool. Awesome. Hey, great. Nice to be here with you guys tonight at Product School. Today, I'm going to be sharing some advice from my book, The Lean Product Playbook, on how to achieve product market fit. I'm just curious, has anyone heard of this book before? Has anyone read it already? All right, cool. So I'd love to hear any feedback you have on it. Hopefully, it's helpful. But this would be like the movie version. You don't have to read the whole book. You can just watch the movie for 45 minutes, right? Great. So just real quick, so you know where I'm coming from. I started out with a technical background. I've been coding since I was 10. My parents gave me a computer then, which I'm grateful for. And then I was an electrical engineering major. And actually, my first job out of undergrad was doing submarine design for the Navy. After doing that for five years, I knew I wanted to go to business school and leverage my technical background and get more on the business side of things. So I moved out here to go to business school at Stanford. That's where I learned about product management as this awesome career. And I was like, that's exactly what I want to do. And since I had never done it, I asked people, where's the best place to learn product management? And people told me into it. And so I was fortunate to get a job at Intuit. I worked on the Quicken team and worked my way up to become a product leader there. After Intuit, I was a product leader at startups. And then I kind of stumbled into being a product consultant to startups, like an interim VP of product. And I did that for Box back in 2007. Then I had an idea for my own startup. Like a lot of people in the Valley, I wanted to start my own company and try that. So I did that too. I've been a CEO and co-founder. And then after that startup, I just went back to being a consultant again. So I've been consulting and helping companies and teams for nine years now. And I also am very passionate about events like this, bringing together speakers and sharing best practices. So almost four years ago, I started a monthly speaker series group down in, I live down in Manila Park. So it's down there in Mountain View, actually, where we bring in top speakers each month. This is my Twitter handle. And I post all my content on dan-olson.com, where I have my videos inside. So don't try to break your hand, taking notes. I go pretty fast. We're going to cover a lot of stuff. I won't go too fast, but the content's all there. And I know they're recording it, too. And my book is the Lean Product Playbook. And I've been this Lego guy, Ninja guy's here, because I've been giving this talk for a long time. I used to be called How to Be a Lean Product Ninja. And I used to ask people in the audience, who's heard of Lean Startup? And over time, more and more hands would go up. Now it's like, who hasn't heard of Lean Startup? And people are embarrassed to put their hand up. If you had to summarize what is Lean Startup all about, I think on a single slide of bullet points, these are the key points that I would say. It's about getting explicit about your hypotheses. When you're trying to build a product, you're making all kinds of assumptions and hypotheses. It's about getting those out of your head, getting them down on paper or Google Docs, and then figuring out how you can articulate those. Figuring out the fastest way to test those, right? The cheapest, fastest way. That's also where we want to keep our scope small. We don't want to overinvest before we realize that we're heading in the wrong direction and we need to change direction. That's where the concept of MVP Minimum Viable Product comes in that we'll be talking about a bit. We want to get out of the building. It's great to have your own ideas, but you need to get out of the building and actually talk with customers and test your hypotheses with customers. And then basically the goal is to learn and iterate as you learn, basically, hopefully quickly with the ultimate objective of achieving product market fit. So usually when I put those bullet points up, everyone kind of nods along and says, yeah, if I get it, I get it. Sounds pretty easy, right? But part of why I have a good consulting practice and part of why I wrote the book is even people that get these high-level concepts, when they go back to their desk and they try to apply it, they run into challenges. They just run into like, okay, I don't know. I get at a high level what's important, but I don't know exactly the details of what to do. And for me, there's a lot of overlap between product management and lean startup concepts. That's why I have the word lean in there. I think lean startup has helped create a vocabulary that we can use, like MVP and product market fit, pivot, things like that, that has a high degree of overlap with product management. And just like people struggle with applying lean startup principles, people struggle with applying best practice in product management. And so that's why I wrote the book, The Lean Product Playbook. We're gonna be, like we said, give a one away today. So I have a tool that kind of randomly draws people. If you just tweet to Dan Olson, we'll pick one at the end. And if you want a slide you like, you can take a photo. That's great. If you wanna throw a hashtag on it, and people see it, that would be great too. But basically my book is a playbook, and it's a playbook because it's trying to tell you exactly what to do. It has a process in there that I'm gonna be sharing with you. But it basically is a guide for how to achieve product market fit. And anytime you have a big movement, like lean startup, there's all these buzzwords, right? MVP, pivot, product market fit. MVP is probably the most hotly debated and contested term out there. People getting on online fights about what it is and our arguments in their company. Product market fits a little different. People actually talk about it overly simplistically. Like it's some true or false state of existence. They'll be like, oh, box succeeded because they had product market fit. StartupX, sadly they failed because they did not have product market fit. And it's just like a true false condition. It doesn't really help you if you're trying to achieve it, right? And so if you go and Google and look for stuff, there's not really a lot of guidance on what it means and how to achieve it. And so that's why I wrote the book. And as I said, my background's in engineering, so if I want to try to achieve something, I want to have like a rigorous framework that I can use and rely on. And so I created a framework called the product market fit pyramid, which is the core of the book. And basically it breaks product market fit down into five components. And basically all five of these things, you basically need to get right enough to achieve product market fit. Let me walk you through it. The bottom two layers are the market and it's a pyramid and each layer builds on top of the next, right? Each one depends on the layers beneath it, just like a real pyramid. It all starts with the target customer, right? Whose life are we trying to make better? Whose pain point are we trying to solve? Who are we trying to create value for? And if you change a target customer, then you may have to change everything above the pyramid, right? But it's important to know whose life we're trying to make better. The next step up is for that customer that we just identified, what are their underserved needs, right? What needs do they have? And then the next layer is like, which ones are the ones that they're not happy with how they're being met today, right? If they're happy with how it's being met today, we probably don't want to target those needs. So I'll share some ideas and frameworks on how we identify underserved needs. But taken together, these two layers are the market. And if you look in a marketing textbook or an economics textbook, it'll tell you the definition of the market is a group of people that share a set of common needs. That's basically what a market is. Now your product, I like to represent with three layers. Those layers are your value proposition and that's meant to ideally build on top of the underserved needs. It answers the question of which needs are we actually gonna bite off and save that our product delivers on and most importantly, how are we gonna do so in a way that's better, different than the other products that are out there? So that's where product strategy and differentiation comes in. The next level up is the feature set. Okay, great. For those benefits that we said we're gonna deliver, what's the functionality that's gonna actually convey those benefits to the user? And that's where the concept of MVP comes in. We don't wanna overbuild here before we realize we're going in the wrong direction, right? And then finally is user experience, which is actually what brings the functionality to life and it's what your user and customer interacts with to bring the product to life. Basically, you can't really control the market. You can pick which one you wanna go after. You can say I wanna go after this market and I wanna go after these needs, but this is where you really get control to make decisions about which value proper you're gonna go out with and what feature set you're gonna go out with and what user experience are you gonna go out with. So when you have this view of all the hypotheses you need to get right, product market fit is just, hey, what are all the hypotheses and execution that we're doing up here? How do well does it resonate with the actual market that we're going after? That's what product market fit is basically. So it's a simple framework that helps you kind of walk through the key hypotheses. Once I had this framework, I realized, you know what? I can create a process around this to guide people through working through these key hypotheses and I call it that the lean product process and that's gonna walk through tonight and close out with an end to end case study at the end. But basically it just starts out by starting at the bottom of the pyramid or working your way up, getting clear on what are our assumptions about our target customer? What are our hypotheses about which other needs are underserved? What do we want our value prop to be as far as which needs are we gonna deliver on and how are we gonna do so in a way that's better or different? What should our feature set be again so that we don't over build? What should our user experience be? And then there's just one sixth step which is once you have, once you've worked your way through the pyramid, I'm a big fan actually of testing and validating your ideas without doing any coding or building if you can avoid it, right? So I'm a big fan of using clickable or tapable prototypes here and when you get to that point, whether it's a live product or a prototype, that prototype represents all the assumptions and decisions you made along the way. And then there's a sixth step where we take that prototype or live product and we close the loop and we test it with the customers and that's how we see where we're at with product market fit. So that's the six steps of the lean product process. And I'm gonna walk you through those. And it's meant to be an iterative process obviously, we're gonna have to, there's some sequentiality in how they need to work together, but you're not gonna get it all right the first time you need to iterate. And I'm sure a lot of you've heard of the build measure learn loop from lean startup. I have a modified version that I prefer. I don't think the thing about build measure learn is it starts with build. So some startups get really excited, they're developers like, hey it says build, let's build first. I actually think you can get a lot faster, learn a lot faster without building. So for me is a modified version that starts with hypothesize. You get clear on what your hypotheses are. You design a way to test your hypotheses. You test them and you learn and then you revise your hypotheses. So that's how we're iterating as we go through the lean product process basically. Cool, so let's jump in and we'll go through the process and highlight it with some examples. The first step is determine your target customer. And I should say by the way, you can use this process for a new product obviously. You can also use it for a new feature of an existing product. You can also use it for like a V2 of a product or a V3 of a product, a new version. And you can also use it if there's a product where you haven't been applying these techniques, you're gonna find low hanging fruit and easy ways to improve product market fit. But let's say you're working on a new V1 product for the first time. I just wanna acknowledge that you may not be clear on who your target customer is. You may not know yet, that's fine. All you wanna do is have a tentative hypothesis because these two steps are so intertwined that as you learn about customer needs, you're gonna go back and revise your definition of target customer. So I just wanna acknowledge that if you're staring at a blank piece of paper and it's like, okay, Dan's telling me I gotta figure out who my target customer is. It's okay to have a tentative hypothesis and iterate it later. But let me show why it's so important to be clear on your target customer. And what happens a lot of times is I find product teams, they kinda stay a superficial layer. And at first it sounds plausible and it sounds good. I remember I was judging a competition at the Stanford Design School and I just went through, I was a judge and I would just ask my questions. The first question I ask each team is who's your target customer? What are you doing for them? What's your value prop? And one team told me, I said, who's your target customer? And they said, millennials. And at first I was like, okay, cool. That sounds good. That sounds specific. But then I thought, gosh, how many millennials are there out there? There's like a bazillion millennials and their product had to do with a certain vertical like food preparation at home. I'm like, there's gotta be some other way they could have clarified or made it more distinct, right? So that kind of, that's why I call it the server. Think of an onion and you peel the onions, got multiple layers. That's like the outer layer of the onion and it sounds okay at first, but that's not how you really create product market fit or value. You gotta peel the onion. So let me show you, this happens all the time whether you're talking about customers or needs. Let's talk about a need that a lot of people have. You have transportation within 100 miles of your home. Whether it's to get to work, to get to school, to get to the store, whatever it is. This is a need that a lot of people have, right? And we could talk about at that level, but as soon as you look through that need through the eyes of a particular target customer, you'll see that the detailed needs are quite different even though they share that high level needs. Let's take two target customer personas. One a soccer mom and the other a speed demon. They both have that need to get around. But if I went and did customer discovery interviews with 20 soccer moms and I said, hey, can you tell me what's important to you when it comes to transportation? They might bring up things like, well, I'm carrying my kids and all their friends and all their gear, so the car's gotta be big enough to just hold all that stuff. And I'm driving my kids around, they're the most important thing in my life, so safety is really important to me. And by the way, I'm doing a lot of driving on the weekend so I'm thinking about how much am I spending on gas so fuel economy's important, right? So those might be some of the things that we hear that are more detailed needs and preferences related to the macro superficial need. If we interview 20 speed demons, they probably wouldn't bring up any of those things. They might bring up things like, well, what's important to me is that the car goes really fast and that it looks cool and most importantly, I look cool when I'm driving down the street in it, right? And you end up with very different products as a result. Now both, again, meet the high level need, but they address the detailed needs quite a bit, right? And just think about all the different cars and shapes and sizes of cars that are on the road. So the car industry has done a really good job of kind of defining those micro segments and building products that are optimized for each of them and their needs. So it's important to do that. Personas are a great way to capture your assumptions. They're often used later in the process in UX design, but I think you can use it early on. The other thing is personas often get a bad rap. It's usually because somebody worked in a company with somebody that misused the tools. So I would just say, don't blame the tool. Give personas another shot if you had a bad experience. The second step is great. Once we got clear on who we think our car customers, what do we think their underserved needs are? And whenever we talk about customer needs, the framework that I like to use is actually problem space versus solution space. I'm curious, who's heard of problem space here before? Few people. So I've been talking about problem space since like 2007. You see more and more people talking about it, which I'm really glad about. Let me explain what it is basically. So problem space is a customer problem, customer need or benefit that the product should address. So it's like a well-written product requirement, a well-written user story. Let me give you an example. If I said, hey, you know what? I want to make it easier for people to share photos with their friends. That statement, make it easier for people to share photos with their friends, would be in the problem space. Something that the product should do and that I'm hoping it will do for people. In contrast, the solution space is a specific implementation designed to address that need or requirement. If right after I said, hey, I want to create a way for people to share photos with their friends, I said, and by the way, last weekend I coded an app for it, that app would be in the solution space. Or if I was like, hey, my friend Manny actually, he's a rock star designer, he designed a great wire room mock-ups for it. Those mock-ups would be in the solution space, right? That's the difference between the need and the actual implementation. And what happens is far too many part teams just go barreling right in the solution space. I mean, we live in the solution space. That's where we live, all right? So it's really easy. It's easy just to start designing or build something and not have to answer all these hard questions about who's it for and what's it gonna do for them and that kind of stuff. It's easier, right? But when people build great products, it's because they've actually stopped before they got in solution space and thought hard over here and come up with hypotheses and tested those hypotheses. The example I like to use to illustrate this is when NASA was sending astronauts into space, they knew that the pens that we use here on Earth wouldn't work because they rely on gravity, right? And they knew that, hey, in space there's no gravity, so how are these guys gonna, what are they gonna do up in space? And it wasn't NASA. So if you Google this, you'll end up on some snopes.com urban legend. And the main point is it wasn't NASA, but one, and NASA didn't ask the contractor to do it, but one of the contractors, the head of it, Fisher said, you know what? I think I can invent a space pen. And he went off on his own, spent his own money, spent a million dollars on R&D, and he invented a space pen. I have a space pen here. They're pretty cool. They tell me it writes in space. I haven't been there to verify it myself. I'm hoping I can connect with NASA someday and verify it, but anyway, so it needs to need, it writes in zero gravity. Faced with the same challenge the Russians gave their astronauts pencils. And you can actually get a Russian space pen. It's a joke. It's a pencil in a red box, kind of poking fun at the whole space pen thing, right? So why do I tell the story? I tell the story for a few reasons. One is obviously, if this is just as good a solution as this to the need to write in zero gravity, then the one that didn't take a million dollars and all that time and effort is better because it has a higher ROI, right? That's like the most obvious lesson from that. But the other lesson is just it's really easy even when you're trying to focus on problem space to have what I call solution pollution in your problem space, right? You're polluting your requirements by embedding some solution artifacts in it. So when that head of that company, Fisher said, hey, I think I can invent a pen that writes in space. And that was his problem statement. He had some solution pollution in there. What was that? A pen, yeah, a pen is a solution, right? So that happens all the time. The way it happens on a feature team is like in the geroticket, it'll say, build a dropdown, build a menu, build a configurator. It tells you the solution, right? And the way to get out of that trap and to back into the problem space is to use the five-wise technique. One way you can do it is the five-wise technique from the Toyota production system from the startup where you go, well, why is that important? Why do they need to write in space, right? And even if you just said, hey, a way to write in space, that would be better than saying a pen that writes in space. Just be vague. But if you applied five-wise and said, well, why do astronauts need to write in space? Like, why do they need to write in space? Anybody? Yeah, you don't want to write. Yeah, basically, they want to write down information and refer to it later, maybe make calculations. Like, you know, there's a couple of things you need to do in writing. That would be an even better way to articulate the requirement, right? Get away from writing things down and just say, hey, here's what they need to do. They need to like document information or refer to it later. When you get that far abstract, then now maybe we could come up with some crazy Siri talking situation, right? The solution where you talk to it and it comes back to you, right? So that's the other thing where, you know, by getting really clear on the problem, it can open up the solution set to a more creative solutions, basically. So those are some of the reasons that I like to share that example. Now, it turns out that pens don't need to just write in space. There's also things about not being flammable and things like that. So space pens are the way to go, but I just use this example to illustrate some of those concepts. Now, let's talk about more of a software example. And the idea is you can map, you want to start in the problem space and then map to the solution space. I used to work it into it on Quicken, but another one of our big popular products is TurboTax. Who's used TurboTax here? All right, cool, great. This is gonna be good. So it's a software product, so it's in the solution space, right? It's not a problem, it's a solution. It competes with another product in the solution space, Taxcut, so they're in the solution space, right? And the idea is like, you know, let's try to map back, well, let me just do this. Like for the people of these TurboTax, tell me what's the problem? Like what's the problem it solves? What value does it give to you? Cheaper than a lawyer. Cheaper than an accountant. The instructions for taxes are difficult. What else? Easy and fast, all right. Do it from home in your pajamas or not, yeah. But it has embedded in it, it's enormous knowledge of the tax system that I don't want to know. Has a lot of knowledge, yep. You don't want to waste space in your brain with that. Yeah, what else? Online filing, great. It validates or checks them, yeah? You don't need stamps. You don't need stamps, perfect. One more? I think you said cheaper than an accountant, is that what you said? I think you said cheaper, yeah? Convenient, right. So that's the thing about the problem space. Did anybody give me the exact same answer? None of you guys gave me the same answer. How the heck am I supposed to make sense out of that stuff, right? Screw it, I'm just gonna design and code a product, I'm not worried about that, right? So there's a lot of different, that's the thing about the problem space, it's messy. But what you want to do, if I had it, and again, we can peel the onion. If I had to define the overall, if I did Five Wives with you guys, and said, well, what does it do overall, or try to get you kind of to abstract out, we'd probably get something like, well, it helps to prepare my taxes, help me do my taxes. That's like the superficial layer of the onion, like millennials, right? And you guys are bringing up much more detailed things that live below it. The other person that helped me appreciate this concept was Scott Cook, the founder of Intuit, really smart guy, and he would give trainings and talks to groups of product managers like this that Intuit, and he'd say, one of the things he'd love to ask at the end, he'd be like, hey, who's TurboTax's biggest competitor? And we'd all be like, pick me, pick me, pick me, and he'd pick you, and he'd be like, it's tax cut. And he's like, no, you're wrong, it's pen and paper, because more Americans are doing their tax returns with pen and paper. So that's the other thing about the problem space is it helps you really understand where the true substitutes in competition for your product, right? And back to the product market fit pyramid, the needs, the market needs don't change anywhere nearly as quickly as the solution waves that come and go. In the book, I talk about the need to listen to music on the go, right? It started a long, long time ago with small FM transistor radios, and then we had Walkman, right? And then we briefly had the portable CD thing, and then we had the MP3 players, then we had the iPod, and now we all have our phones, right? So five or six different solution waves of technology, but that fundamental need to listen to music on the go was the same throughout the whole time, right? So that's why it's important to get clear on the problem space. And like I said, you wanna define the problem space before you jump into the solution space, so this is the fun part. With your team, you get to do diversion thinking and brainstorming. You get a suspend judgment and say, great, when it comes to preparing, to helping people with their taxes, what are all the things we could do? Not saying we're gonna do them, but let's really explore the problem space and come up with all the different things we could do, right? And you guys brought up a lot of them. Computers are great at math. Like you were saying, they can check your taxes, right? Instead of your form, if you think about filling out forms and making math mistakes, they can help you file your taxes. A couple of people said that, you don't need stamps. Instead of having to go to the post office and stand in line, you just push a button. They can help you maximize your deductions, right? They can ask you questions and say, hey, did you do this? Did you do that? Did you know you could write this off? They can analyze your returns and tell you what your level of audit risk might be, right? These are just four examples, but that's what you wanna do as a team is really spend time exploring all the possibilities you could do in the solution space, in the problem space, sorry, before I'm not saying we're gonna do them, right? There'll be plenty of time later to say that's dumb or why we shouldn't do that one and things like that. That's the convergent part of the divergent and convergent thinking. And you'll notice that they follow a well-written problem, say same as follow a certain pattern. They all start with like an action verb, check, file, maximize, reduce. That's cause it's actually doing something for the user. It's creating value, like it's an action verb, right? The second thing is it's written from the users. It's not written from the company's perspective or the product's perspective. It's written from the user's perspective, like my taxes, my deductions, my audit risk, right? So it's kind of like a well-written, natural user story. It's kind of like written along those lines. And again, you wanna peel the onion, right? I'm gonna peel the onion and come up with all the different things that you could do. And when you do that, it's messy as we saw here, right? We saw how messy it is. You guys gave me like 10 different answers and some people gave me two answers, right? Huh? They're like an onion, it makes you cry. I need the little goggles, yeah. But, and basically why is that? Well, basically one thing is you might be fundamentally talking about two different benefits. Someone's talking about it's cheaper. Someone else is saying it's faster. There's no way there's ever gonna be the same thing, right? But even people talking about faster, they might use different words like quicker, faster, easier, convenient. They're like synonyms or kind of closely related words. And that's what happens when you run this. You'll see that there'll be clusters of related benefits and there's a little bit of structure that you can find. So let's take these benefits. Help me prepare taxes, reduce my audit risk, check my return. If I use that five wise technique on people that said this is why they like and what they hope to get out of Turbo Tax or what they get out of Turbo Tax, I said why is that valuable to you? Why is that important to you? What we're trying to do is get them to kind of up level and move up to a higher level story. And we might find that all these things have to do with empowerment or confidence. A couple people said here, hey, I don't know the tax code, it's complicated, right? The story might go, well, you know, before Turbo Tax, I'd have to fill these things out by hand. I'm really bad at math. I don't know the tax code. It's really intimidating. I'm sure I'm filling this thing out right. It's very, it gives me a lot of anxiety because I'm probably not doing it right. But instead, using Turbo Tax, it just asks me a few questions and holds my hands. The next thing I know I'm done with my return right. That would be an empowerment and confidence, right? We call that a benefit ladder because you basically are getting people to climb up and all three of those are on the benefit, on the empowerment confidence ladder. There could be a completely different benefit ladder that has to do with saving time. It has nothing to do with confidence, right? Saving time, preparing, saving time, filing. There could be a third completely different ladder about saving money, right? It's maximized by tax deductions. So this is what I mean by a problem space definition is you want at least, you don't even have like three levels or four levels, just one level is fine. Where it's like, yeah, we brainstormed the detailed problem space statements and then we just kind of came up with what the clusters are. You'll notice this whole time I'm talking about Turbo Tax, I've been talking about Turbo Tax, I haven't mentioned a single feature. I'm talking about the problem space the whole time. It just so happens that they do have a feature that's one to one mapped with these problems, which is what you want. So what you want to avoid is you don't want to have a problem that doesn't have a solution that maps to it. You don't want to have a solution that doesn't have any problem map to it. That's just somebody decided to build that for no good reason really. And a side benefit is basically that if you have this clear one to one mapping and your feature is well named, the side benefit is a user, just by seeing the name of the feature, you can pretty much figure out how it's going to create value for you. So that's basically kind of what I encourage people to do in the problem space. It's actually kind of fun because you get the brainstorming and come up with a lot of ideas. The next thing, if you take my advice, is great Dan, we brainstorm dozens of potential needs that we could address, we obviously can't build them all. So the next thing that comes up is how do we prioritize them, right? And that's where we're getting at the underserved part of their needs. And that's where the framework that I like to use that I developed is called importance versus satisfaction. Let me explain that to you. So basically importance is for each of those needs, for each need, how important is it to our target customer from low to high? And you can think about this two ways. One is like, imagine we did like a formal survey with thousands of people and we said, hey, please rate it on a scale of one to 10 and the average is 8.6. Like you can get really precise that way, but you can also just use it as a thinking tool, like low, medium, high. Within your team, hey, do we think this is low, medium, high importance, this particular need? So you don't get wrapped around the axle because how would I measure the data? It can be used as a thinking tool. The other axis is how satisfied are people with how they're getting that need met today, right? So this isn't the problem space, this isn't the solution space. Again, you're low, medium, high or you can think about it like a formal quantitative survey, right? And what we wanna do is basically think about where each of these needs are. To try to talk about the space, let's just divide it into four quadrants. So in the bottom left quadrant, we have a low importance of user need. We have low satisfaction with the kernel alternatives. The bottom right quadrant, we have low importance of user need, we have high satisfaction kernel alternatives. At the end of the day, neither of these are worth going after, right? Why would you spend your precious time and resources going after a low importance need? People don't do this on purpose, they do this because they think it's important, they think it's gonna solve a problem for the customers or they're not even thinking about customers and they're not using these techniques and then they launch it and then they realize, oh geez, what we thought was important, customers actually don't think is important, right? So by using these techniques, you can avoid going after these two quadrants. The upper right quadrant, we have high importance of user needs, that's better than low importance, but we also have high satisfaction with kernel alternatives. It's an important need but people are pretty happy with how they're getting in bed today. That's the definition of a competitive market, right? And you hear people say, hey, you need to be like 10x better than the competition. This is why you need to be 10x better than the competition. You can't just roll out some Me Too product because people are pretty happy. The first thing I think of here is like Google search. If I said to people, hey, how important is the, you know, a scale one to 10 to find the information you're looking for online, people would probably say it's pretty important. But it's not like people are going on, gosh, I can't find anything on this Google, it's just not working for me, right? It's pretty much find what you want. Maybe you revise your query once or twice but it works, right? And in the book I talk about a startup that's actually tried to take Google on internet search and failed. And then the last quadrant, we have high importance of user needs, so that's good. And we also have low satisfaction with what's out there today. That's where opportunities lie. And they may not be there forever because we live in a, you know, competitive environment, everyone's trying to innovate and find problems to solve for customers. But if you do the analysis, you can find these opportunities. One of the market opportunities I think that lives up here is ride sharing basically, right? Companies like Lyft and Uber have done a lot to get to the level of success that they've gotten to. But I think one of the main reasons is they fundamentally address an upper left quadrant need. Especially here in the city, I remember when I used to try to get a cab, right? You'd call a cab and you try to call. They'd say, yeah, yeah, it'll be there in 30 minutes. It may be there in 30 minutes, maybe 40, maybe it won't show up, right? So you can imagine if you did customer discovery interviews with people who said, hey, I think you're the last five cab rides. How satisfied were you with that, right? Well, let me start out with importance. How important is it to get to the airport, to your flight on time at the airport? Pretty important. How important is it to make it to that job interview? Pretty important, that date, whatever it is. It's pretty important to get where you need to be on time. And if you just said, hey, let me, let's talk to you about your last five cab rides. How satisfied were you? And we could break it down and say, you know, hey, how satisfied were you with how punctually the person showed up or if they showed up at all? How safe did you feel? How clean was the car? How polite was the driver? How convenient was it to pay the person, right? You can imagine low score, not all cab rides, obviously, but some of them you can imagine low scores. That's an example of an upper left quadrant need, basically. And what I like about this framework is it's meant to be a visual framework. So what I'm trying to say is, if there's a certain product represented by this red dot that addresses a customer need that has that much importance and it does so, provides this level of satisfaction, the one I'm trying to say is, when you plot it with its corresponding importance and satisfaction, the rectangle that's formed is a proxy for how much customer value that product creates. And if there's another product that addresses a higher importance user need with a higher level of satisfaction, then it just creates that much more customer value. And when you view the importance versus satisfaction this way, you realize that the opportunity to create customer value is just how much room is there to the right of where the market is today? All right, because that's, we can just increase satisfaction. And that's why the upper left quadrant offers the biggest opportunity because it's got the most area to the right to add more customer value. And if you have an existing product here, you can apply these techniques to move it to the right, increase the customer satisfaction, increase the customer value that you're creating. Now, if these importance versus satisfaction concept seems a little abstract and fuzzy or theoretical, let me show you some real data. So this is more in the quantitative survey use case less the thinking low, medium, high use case. I was a product manager for product. We had lots of users who were able to survey them. We had 13 key features that we really cared about. And so we asked, we surveyed our users and asked them to rate the importance and satisfaction quantitatively. So now we actually have, instead of just low, medium, high, we have numbers going up to 100% on both importance and satisfaction. And each of these points is one of those 13 features plotted with its corresponding importance and satisfaction numbers. And the number to the right is just the satisfaction number so you can more easily see what it is. So as a product manager for this product, the first thing I noticed was this point. I'm like, hey, customers are saying it's 100% importance and they're telling us they have 98% satisfaction. Like great job team. My second thought was, I'm not gonna spend any of my development resources on that because it's already fine. Like let's go find something else that's gonna create more value. And we had a cluster of different things in here. We had this one down here, but this one is the one that was most up into the left. It was pretty high importance and relatively low satisfaction. So we focused on that one and moving that one up to create the most customer value. And a couple of years after I left into it, I was really excited to come across the book what customers want. Has anyone seen that book, what customers want? So it's a thin book by Tony Olwick and he also has his own kind of quantitative framework for importance and satisfaction. So it gave me a lot of confidence. Like wow, okay, I came up with that. He came up with that. There's something here about importance versus satisfaction. If that idea about getting rigorous and quantitative appeals to you, I recommend checking out his book. All right, so that's how we identify underserved needs. The next step is great. Let's get clear on our value proposition, which again is okay, out of this potential underserved needs, which ones are we actually gonna tell people our product delivers on and how are we gonna be better than the competition? And the framework I like to use here to get clear on our value prop is the K-No model. Has anyone heard of the K-No model? A few people, all right, great. So I learned about the K-No model before I moved to Silicon Valley and went to business school. I actually got a master's in industrial engineering where I studied lean manufacturing and that's where I learned about the K-No model. And it's a great tool to get clear on your value prop and think about your product. So let me explain it real quick. On the X-axis, it talks about how fully does the product meet the need that we're talking about? From not at all, it doesn't meet the need at all, to it fully meets the need, right? That's the X-axis. The Y-axis is based on how much the product meets that particular need, how much customer satisfaction or dissatisfaction is created, right? And if this seems a little complicated, don't worry. There's only three different categories within the K-No model. So it's really actually easy to get your head around. The first is a performance benefit or feature, right? The more the product meets this need, the more satisfaction it creates. The less the product meets the need, the less satisfied the customer is, right? So more is better, the less is worse. If we were building like microprocessors for computers and ours was like 15% faster than the other guys, chip speed is a performance benefit for microprocessors. Let's stick with cars. Let's say I was shopping for a car and there were two cars that were pretty similar. They looked similar, their specs were similar, but all of a sudden I realized that one of the cars had like twice the miles per gallon as the other car, I would probably pick that car because all our things being equal, the fuel economy is a performance benefit for cars. I'd rather pick that one, right? So that's performance benefit. The second one is must-tabs. Now having a must-tab, you can see it doesn't, when the product fully meets the must-tabs, it doesn't make anybody happy. But not having the must-tabs makes them unhappy. That's what this curve is saying basically, right? So sticking with cars, say I was buying a new car and I walked into the showroom and I just see this car and I just love the way it looks. And then I read the spec sheet on the window and I'm like, wow, these specs sound great. This is like the perfect car for me. And then I take a peek inside the window and I realize, hey, there's no seat belts in this car. I wouldn't buy it because I'd be afraid of getting hurt or dying, right? So seat belts are a must-tab for a car. But if car A has 100 seat belts and car B has five seat belts, I don't say it's 20 times better. I want you to have one seat belt per person, you're flattened out basically, right? So that's the must-tabs. And the third category is delighters. So not having a delighter doesn't cause a problem because people don't expect it to be there. But having a delighter can create a lot of customer satisfaction. We call that like wow features basically, right? Not today, but sticking with cars. When the first cars came out with GPS navigation, it was a delighter, right? Before that, people were printing out Google Maps or asking for directions or getting lost. And then the first couple of cars with GPS, you just put in the address of where you want to go and it just fundamentally changed how you got from point A to point B. But as we know, over time, more and more cars got GPS navigation. Garmin and Tom-Tom came out with their add-on units and now we all just use our phones, right? So this is not a static picture. So there's needs and features migrate over time, right? So that yesterday's delighters become today's performance features and become tomorrow's must-tabs. And the pace with which that happened just depends on the level of competition and innovation within your space. But tying back to our value prop, what we want to do is think about these three categories of benefits, right? Must-tabs, performance and delighters and use those to categorize and get clear on our value prop. And again, our value prop is, hey, out of all the benefits that we talked about, which ones are we actually going to tackle with our product and how are we going to be better or different than our competitors? And the way to do this is basically to create a grid and then for your product category, I left the generic on purpose, list one per row, what are the must-have benefits that apply to our product category? What are the performance benefits and what are the delighter benefits, right? That's step one. Step two is you want to create a column for each of your competitors and create a column for your product. That's step two. Step three, you want to basically rate or score your competitors, right? And it can be, again, it can be low, medium, high. If there is some quantitative measure, you can use numbers if it's amenable to it, but you can also just use low, medium, high. So this is our competitive landscape. Let's say we have two competitors, they both do the must-have, of course. These guys, competitor A are the best at performance benefit one. Competitor B is the best at performance benefit two. They're both so-so at performance benefit three. These guys have their own delighters, right? That's the kind of competitive analysis backdrop that we want to figure out what our value prop is. This is basically the essence of product strategy. And I honestly think like less than 10% of our teams go through an exercise like this to make sure they're clear on how their product is going to be better or different, right? And one of my favorite definitions, it's super easy to go, hi, hi, hi, we're going to be high, we're going to be to everybody at everything. It's super easy, right? One of my favorite definitions of strategy is it means saying no. And there's a great Steve Jobs quote in the book about strategy means saying no. Like it's just saying yes, yes, yes, to everything is not strategic. So you got to pick your battles. You don't have the resources to beat everybody on every dimension. And even if you could, it would not be a well-positioned, a clear, really positioned product, right? So given this backdrop, we might go with a value proposition like this. Of course, we're going to have the must-have. We're going to do somewhat on performance benefit one, but we're not going to try to beat these guys. We're going to say no to performance benefit two. We're not going to worry about that for now. And we're going to be the best at performance benefit three. Maybe we've identified some market segment that really values that performance benefit. Maybe we have some ideas on how our solution and technology can provide better levels of satisfaction. And then we have our own Delighter. What matters the most from this exercise is what's called your unique differentiators. It's what's the performance benefit where we're going to be the best and what's our unique Delighters, right? You don't compete on must-haves, right? And what we want to do is we want to get clear on what are these before we go into the next step, which is our MVP, because if we build an MVP and it doesn't have these things in it, what the heck are we testing with people, right? That's what we want to test these things in our MVP to see if it resonates with customers. So I thought to bring this more to life. Has anyone used Instagram here? Yeah, right? A few people. So if we think about this must-have Delighter benefit, like let's just talk about what, you know, when Instagram came out, there are already a bunch of other photo sharing apps out there, right? So what was it for those of you that remember back then, what was it that made them a part, like made them better than the other ones? Yeah. Filters is one, and I think there's one other thing. I think the other guys would share two, but maybe they did a better job on that. Social? Yeah, I mean, the other ones were social too, I think, but maybe they were more social. Yeah. Did anybody use the other ones? Do you remember whether these guys took the same amount of time or were faster? They were actually faster because they did this hack where they would like start uploading the picture right away before you even push, like upload it or whatever. They kind of did some technical hacks, compressed the image. So part of what they did is like, they literally, the amount of time it would take to share your photo was less because they were doing these technical hacks. So most people say that Instagram, one, it was better for two reasons. One is they uploaded your pictures more quickly. And secondly, they did filters, which is what you said. If we go back to our perform must-have performance benefit Delighters, uploading pictures more quickly, what would that be? Performance, that's a performance benefit, right? How about filters? What would filters be? A Delighter, right? So it's very common, if you look through this lens and you analyze products that are winning in the market, very common pattern you see is there's a performance benefit where they're beating everybody and they couple it with the unique Delighter. And that's a really powerful comment. You can't always have it, even if you know, if you're just outperforming somebody, that's good. But if you can pull the one, two punch like Instagram, then it's really good. So anyway, it's just a way to kind of bring that concept to life, hopefully for you guys. All right, cool, next step is great. We're clearing our value proposition. Let's get clear on what our MVP feature set is, you know? What's an MVP? This again, this is probably the most highly contested topic out there. What do you guys think an MVP is? Go for it. It's viable. It's viable, okay? That's the V, yep, viable. Foundational level one of the feature of products. Foundational level one of a feature of product, cool. Yeah? Usually it's whatever I could, not that I get a reaction from my users. Whatever you can test. Whatever you can test? Right, right, all right, cool, yeah. So I think those are capturing a lot of the points there. I see two main mistakes that teams make with MVPs that I like to just talk about. One is, the whole point of an MVP is to not do it the old way where you had like a team of engineers and they would go off into a cave for 12 to 18 months and code and then come out and then launch their product on the world and then realize nobody wanted what they just build, right? So the whole idea is how can we like, before we invest too many resources, how can we like not over scope things and make sure we're going in the right direction and make whatever course corrections we have before we invest a lot, right? So even though MVP is intended to keep you from overscoping your V1, I see people over scope their V1 all the time. I even had people read my book and say, hey Dan, Dan, I read your book, totally get MVP. But here's why my MVP needs to have A, B, C, D, E and F in it. So it's almost like this thing where you need objective third party eyes looking at it and asking you tough questions. There's almost always 80, 20 at play. And it's easy. It's human nature to be like, oh geez, someone may complain because we don't have X, we better put that in the MVP. Oh, someone may complain because we don't have Y, let's put in the MVP. So again, it takes that willingness and strength to say no to do that. The second mistake I see people make, I like to explain with this framework. So is anyone familiar with MailChimp? So I don't know, I use MailChimp when it first came out and I remember it was just head and shoulders UX-wise above all the other products that were out there at the time. And Aaron Walters was the head of UX design there and he wrote a great book called Designing for Emotion and a designer named Jesse Bassan and modified his framework, then I modified it. So I just want to give them props. But basically he has another pyramid where he talks about how functional is a product, how reliable is a product, how usable and how delightful is it, right? These four attributes so we can kind of score how well is our product doing on each of these four levels. And I was at a workshop once and someone used the acronym DERF. So it calls the DERF pyramid basically. So I was like, okay, it's DERF because they didn't want to rewrite it. So they just put D-U-R-F. But basically the idea is, we can color in how much at any point in time, how much is a, how reliable, functional, usable, delightful is our product to color in, right? And obviously with an MVP, you're not going to color the whole pyramid in in one fell swoop. It's going to MVP, it can't be, right? But what I see the mistake people do with MVP is they use MVP as an excuse to do this. So they don't build all the functionality, which is great because you're not, you have to build a subset. But they use MVP to ignore reliability and say, you know, it's okay if it's buggy, it's just an MVP. It's okay, people, you know, if we'll do the UX design later, we'll make it easy to use later. It's okay, it's just an MVP, right? So that's frankly people misusing and not understanding MVP. And a lot of times this happens when organizations trying to transition from waterfall to agile, they just kind of get, don't get it and then they do that. How do you think these, those MVPs test with customers when you go out and test with customers? They don't test well. It doesn't matter if you have the functionality. If it doesn't work when people want to use it or they can't figure out how to use it, you're not going to get credit for the functionality, right? And the MVPs are going to fail. And then the people go, oh, this whole lean, agile thing, let's go back to waterfall. It's not working out with this MVP stuff, right? So it's true that you want to build a subset of the pyramid, but you want to build it more like this, right? So you're going to build a subset of functionality, but the subset that you build, you want it to be reliable enough. I'm not saying it's going to be perfect, but reliable enough and usable enough and delightful enough so you can truly test your value proposition. You really trust your unique differentiators, right? So that's the other mistake I see people make and this diagram helps people explain it to others. So I don't have time, I have a lot more advice in the book about how to break features down and decide on your MVP. Don't have time for that today. Creating that next thing is once we get clear on the feature set, now we need to create, we need to do the UX design. We need to do the UX design once because we have to do it for the product anyway, but secondly, I think it's a great way to create a prototype that you can test with customers. For the first time, so we basically, one, two, and three are all in the problem space with four we went into the solution space, but usually when we talk about features, it's like words. It's words in a Google doc or words in Confluence or a PowerPoint, it's words or a Gira ticket, right? So this is where we go from words to a user experience. So we're making that transition. And when I think of user experience design, I think of an iceberg because just like an iceberg, it's very misleading because there's only a part that sticks above the water that's obvious, but there's a lot more under the water you can't see, but that's really going on, right? That part that's above the water is visual design, right? We all take in information visually. You guys are looking at my slides, you're looking at me, you're looking at the colors, the fonts, the shapes, right? That's how we take in most of our information, maybe a little bit more through our ears when we're listening, but mainly through our eyes, right? But when you use a product that is intuitive and easy to use, it's because the product team has done a really good job not only at that layer, but at these more foundational layers and these layers build on each other. And I don't have time to get into it again, but in the book I have a whole chapter on UX design because I think it's so important. I don't expect product managers to be designers, but I think it's important to know a lot about design. I learned that early in my PM career to more effectively partner with designers and developers. But it all starts with conceptual design, which is like what's the fundamental concept for how we're gonna approach the design of this product. Information architecture is how you structure and organize your product. If it's difficult for people to find things, the labels don't make sense, then someone hasn't done a good job here. Conversely, things are where you expect them to be and it's kind of intuitive to find things. They've done a good job. Interaction design is literally how the user and interacts with your product. Anything they can tap or click on, controls, forms, buttons, navigation, that's interaction design. When you're doing design, again, we're going from words to pictures, if you will, or drawings. There's a range of different design deliverables or artifacts that you can create. It's probably not best to go straight from words to coding. That's a big hop, right? It's probably not the best also to go from words to high fidelity mockups and then code. So this is also what I kind of view as a best practice of how do you break up the process of going from words to live product in a way that you can iterate and test. So I like to categorize the design deliverables by how interactive are they? Compared to the final live product, how much can I tap and click on them, right? And then fidelity, which is compared to the final product, how closely does this resemble the look and feel and how pixel perfect it is? So in the bottom left, lowest interaction, low fidelity, we have the hand sketch, whether it's on paper or on a whiteboard, right? I wouldn't show this to customers, but it's a great way for you and your team to kind of make that first step from words to pictures and then iterate until you get to the point where you're happy with it. The next design deliverable that I recommend is clickable wireframes. So now they're higher fidelity because we're putting it into a digital wireframing tool. They're higher interactivity because in the old days, they would actually be static wireframes, but nowadays the tools make it so easy to make clickable or tapable wireframes, you have no excuse, you just have no excuse, just suck it up and learn how to do that. You just have to as a PM. When I interview a PM and they're like, I don't wireframe, it's just not a good way for the interview to go. How's, who's used Balsamic before? All right, cool. So Balsamic makes it super easy, right? It makes it really easy to do. And you know, end of the controls are baked in. Okay, when I click this, it's gonna go here and it's gonna go here. So you can like string together a multi-page or multi-screen experience. And that's a great tool for you as a team to work through. It's actually good enough to test with customers. The cool thing about Balsamic is you can only get so how you can't get high-fi, right? For the longest time, the lines were jagged and there were so many OCD people said, can you just make it straight that they finally put a straight mode? Does anyone know about the straight mode in Balsamic? Okay, so defaults to the jagged line mode, which is visually conveying to people, hey, this is a prototype, this is rough. But there's a hidden setting where you can make all those lines straight if you want. But anyway, you can test with customers and you're not gonna be testing the visual design or things like that, but you can test, one of the best things you can test is in that MVP discussion, you might have had a tough decision in your team about should we include feature A or not? And you might have had the courage to say, let's try it without feature A. Well, this is when you're gonna find out if people have a problem with feature A and not being here or not, they're gonna interact with your wireframe and they may go through it and not say anything about feature A and you go, cool, we got it right. Or they may say, are you crazy? I can't use this product without feature A in there. So better to learn that then than waiting until you finish all these high-fidelity mockups and have to redo your whole design, right? You can also learn about information architecture and interaction design here. Once you, I recommend talking to customers in waves of five to eight. Once you've rinsed and repeated and iterated to the point where people are relatively happy with the wireframe, then you can proceed to the next step, which is clickable mockups, right? So clickable mockups, they're interactive or tapable, they're higher fidelity, right? Mockups, now we are worried about, now we are paying attention to that tip of the iceberg in the visual design, right? So I forgot to mention wireframes are typically grayscale on purpose, right? The number of times I've seen a product team bring in their initial designs to some key stakeholder and the first thing the stakeholder says is I don't like this red. It's like, that's not what we're trying to talk about right now. It's like, do we have the right feature set? Do we have the right layout, right? So you can't, to avoid those kind of, people can't help but fix it on visual design, right? So to avoid that, you just don't put any colors in. You don't worry about the fonts. You don't put any images in, right? You worry about organization. Copy is great to have a real copy in there. Anyway, has anyone used InVision? So InVision is a great tool. Usually the way, you know, mockups, the designer will say, great, Dan, here's your zip file with your 50 mockups in it, right? And it's like, what the heck am I supposed to do with that? Well, the cool thing with InVision is you can upload those to InVision. And then, you know, in Valsamic, the controls have built in click, you know, kind of click paths that you can specify. The way it works in InVision is you overlay a rectangle and say, okay, I'm gonna put a rectangle over the sign up button. And when the person clicks on that hotspot rectangle, then it's gonna go to this other one and go to this other one. So it's a way to string together, again, a set of high fidelity mockups. You're not gonna be able to express every path, but you're gonna express what we call the happy path, where you expect people to go. And this is a great way. I mean, at this level, when you've got clickable or tappable mockups that are high fidelity, that's a great place to get customer feedback and is a lot less expensive than doing it with, like, live code and having to go back to dev and change the code and things like that, right? So again, you wanna talk to groups five to eight, get to the point where they're happy with it, and then you can proceed confidently to live product, right? So this is words about features to live product. These are kind of the intermediate steps that I recommend that people go through. I do recommend that you test your live product because these don't have any browser compatibility issues or database performance issues or anything like that. Sometimes when you send stuff into dev, it doesn't come out exactly the way you wanted it to. So it's good to test it, but that's a way to kind of de-risk, work your way through the design iteratively and de-risk it. Just to be super clear, is this a wireframe or a mockup? It's a wireframe. Sometimes people mix up the term. So wireframe is reserved for lower fidelity. This is actually a balsamic wireframe. You can see the cool thing about balsamic and all these tools is they have widget and stencil libraries. You're not creating things from scratch. You should just be like, okay, I need an iOS phone. I need a button. I need this. I need this. I need this, right? Does it have any colors? No, does it have any fonts? No, does it have any images? No, we can't get into stupid disagreements and arguments about that because it's not there. We have to focus on what's on the screen. Are we missing anything on the screen? Should it be there? Is that the right size or not, right? Here's a mockup of the exact same app, where now we do have colors, we do have fonts, we do have pictures, right? So the whole point is work on this first, don't jump to this, then I'll just tell you this. Many of you may know this already. There's a big difference between visual designers who do this and interaction designers who do this. And these days we call them UI designers and UX designers, right? And I can tell when I work with a team and the first thing they show me is high fidelity mockups. That means they have a visual designer and they don't have interaction design skills on their team, right? And so what usually happens is they haven't thought through the information architecture, the interaction design. So anyway, all right, once we have our prototype then or our live product, then we can go and test with customers, right? And just again to tie it back, now we've worked our way all the way through here and we've got this UX prototype. That UX prototype again embodies all the assumptions and decisions we've made throughout the whole process and now we want to close the loop. So it's actually important, one other thing too is to make sure you close the loop with actual target customers, not just anybody on the street or random people because you want to kind of close the loop and see where you're at with product mark fit. All right, I just want to close out with an end-to-end case study of it being applied in the real world. People have said this is super helpful to kind of bring it, make it real for them. The project was marketingreport.com. It was a consulting client of mine. It was a startup. The CEO hired me. They had an existing product already but he had an idea for a new product basically, right? And he wanted to test out this new product idea and he had a very small dev team. So he's the one that said, hey Dan, we can't code anything. You have to do as much validation as you can without coding. And I was like, great, I love doing that. We're gonna use clickable prototypes. We're gonna do this. And he wanted to see if there was a business opportunity worth pursuing there or not. The idea was a marketing product. Who gets junk mail here? Anybody? Yeah. Nice euphemism for junk mail is direct mail. But basically the idea was, say I come home after this talk and I go to my mailbox and there's a coupon for cat litter for me. Why did somebody mail me a coupon for cat litter? Because in some marketing database in the cloud somewhere it thinks Dan Olson has a cat. That's why, right? And so the idea was to give people control and transparency. The best way to explain it is probably by analogy through the credit report stuff, right? So today you can get your credit scores, you can get your credit report. If something's wrong, you can fix it, right? But 20, 25 years ago, you'd apply for a loan, you'd put in your social security number, it'd go off into the cloud database and it'd be like, oh, okay, oh no, sorry. You didn't pay your bills on time. No, you're denied for that loan. And you wouldn't really know why. So that was the idea. It was like to give you transparency. Why are you getting the junk mail you're getting and help you kind of potentially fix it or make it more relevant. That was the idea. So I said, okay, great, sounds good. By the way, the chart customer, as you saw, was pretty much everybody. It was a broad customer offer. Step one is who's your chart customer. It was a broad consumer offering. Anybody that got junk mail in the US, right? So I said, great, let's get clear on our problem space and let's talk about what some of our top feature ideas are. The key problem space idea was basically learn why I received the junk mail. I received like, why am I getting this junk mail? And the top MVP feature ideas that they had were a marketing report, which was analogous to a credit report. It basically consisted of a marketing profile which is like all those different attributes that they have. Like, okay, you have a cat, you're married, you have kids, whatever it is. And then a marketing score, which is meant to be like a credit score. Some numerical measure and higher is better. And that was the core idea. And then one of the executives said, hey, in addition to that, I'm interested in exploring money saving offers. What if I send you a cat coupon and you wanna say, well, you know, I don't have a cat, I have a dog. Can I opt in for dog food coupon? Can I opt in for dog related coupon? So can I opt in for relevant money saving offers? He wanted to test that. He had hypotheses that people also wanted to compare their shopping habits and how much they spend and things like that with other people. So that was that box. And then because social networking was hot at the time, social networking. It's a pure like, you know, key stakeholders slamming a solution into the prod, into the thing. All right, cool, that's fine. We weren't clear on what exactly why people would want that, but sure. The other executive didn't care about any of the blue stuff. He cared about the green stuff and he was also interesting exploring like, hey, what about just the overall idea of just reducing the junk mail? Like, let's just test that idea. And then we brainstormed a secondary benefit. Hey, if we're really, truly reducing a bunch of junk mail, maybe we can make some claim that we're saving trees and being environmental because there's a lot less paper going around, right? So I iterated to get this to this point and we got to this point. I said, okay, is everyone's idea on the board? And they said, yes. And then I said, okay, cool. Now it's time to figure out our MVP. I looked at this and I said, this is way too big for one MVP. This does way too much stuff. So what I did is I actually created two MVPs. I created one, again, the green stuff is the core idea. They just wanted to test the secondary stuff in addition, right? So I created one MVP that had the core green stuff plus the yellow and we call it the marketing shield because it was kind of shielding you from junk mail, right? So that was our MVP feature set for that and then we created a set of prototypes for that. The second MVP was the green stuff plus the blue stuff and that was the MVP feature set and we created a set of prototypes. Obviously, these were the same kind of, this was similar between the two, but these were different. Then we proceeded on to doing UX design. This is an example of the prototype we created, the screen we created, I'd call it medium fidelity. It's a little design, but we didn't really spend a lot of time obsessing about visual design. When you clicked in, then you got to this main page where there would be like a module for each of those features from the MVP feature to list. There'd be a module and then you would click down to learn more and that's where you'd go to like a whole screen where you would interact with that feature. So that's how we didn't get away with not having two completely different UI designs and we would just mix and match the modules and then if you're in the saver one and have the saver modules or if in the shield one have the shield modules. So now that we had our clickable prototypes, we recruited people and I moderated them through the prototypes. What did we learn? Here's that same diagram now color coded with red, yellow, green. When I ran it through with people, there were tons of questions and concerns and comments, right? They really weren't super excited about it. So green doesn't mean like, oh yeah, they're ready to sign up. Green just means like, well, we feel like we have a good handle on the questions and concerns that they raise and that we can address them and that if we address them, we think there's some underlying value that can be realized there. That's what green meant. Yellow meant, you know, it wasn't really too appealing and red meant like they just didn't want to have anything to do. They either didn't get it or didn't want it, right? So the first thing when you do this is do you have any green? So luckily we had some green. There's no guarantee the first time, especially the first time you do it, you're gonna get green. So we had green here and we green here. What was the core idea that we were testing? It was this. Do we have any green in the core idea that we were testing? No. So it's like, thank God we tested something else, right? That happens all the time. You get out there and talk to people and say, oh, I really have, tell me about this one. They're like, well, that's not really important. Let me tell you about something else. That happens all the time, right? The second thing is also the red. The red is just as valuable. I'm not surprising the thing that was bolted on didn't play well, right? Do you guys know what a marketing score is? Really? I don't either, right? It was ill-defined. I didn't bother going down that path until we tested, but I know what it would have taken. It would have taken us going and signing some expensive third-party data licensing agreement to get all this data on people. It would have taken hiring and some smart engineers to develop these algorithms and figure out this whole score thing. It would have taken a lot of time and money to educate consumers about what the heck a marketing score was and why they should care about it. But guess what? Because people didn't like it, we don't have to do any of that work and waste that resources. Remember back to the unimportant stuff, like to avoid the unimportant stuff, so we avoided it. And you hear people say you need to pivot? Well, here's what we had to pivot. We were here and we had to decide are we gonna pivot to that green or are we gonna pivot to that green, right? And this happens all the time. Just by getting out and talking to people, you'll learn about adjacent opportunities that are more important than what you think you originally go out to talk to them about. For example, Slack, I'm sure a lot of you guys use Slack, did not start out saying we're gonna build a communication platform. They actually started building a game and they had a distributed team and they were using Slack to communicate and collaborate and they decided that that wasn't much more of a thought product. Flickr, same thing, it didn't start out as a photo. So there's a numerous examples of you start out one path and then you realize by getting in market and talking to people that there's a slightly different path that's better. So we had to pivot. We had to pivot here or here. We decided to pivot here for three reasons. One, as I mentioned, the company already had an existing product and brand. This was more consistent with that. Secondly, money saving offers would have taken a lot of time to go out and find partners and do biz dev deals and sign those contracts. It would have taken like 12 to 18 months which would have killed our time to market. And also there were already a lot of people doing money saving offers. So back to the value prop, we weren't really clear on how we could be better or different. So for those reasons, we pivoted here and because we had done no coding, we had no qualms about just tossing those prototypes and starting with the blank sheet of paper. Because we hadn't done any coding. And so we started from scratch and what we did is we took the six pages of notes we had from the shield sessions and made sure the new prototypes addressed all the questions and concerns that we learned. So we pivoted to junk mail freeze. So marketreport.com was gone. Here's the new prototype of the new homepage that we did, right? And I learned a lot of stuff by talking to customers, by exploring the problem space with them. I learned a lot of stuff. I learned that not all junk mail is the same. Like peeling the onion, right? Here we said reduce junk mail. Well, it turns out not all junk mail is the same. Some junk mail is more annoying than others to people. And what we found is the most annoying junk mail generally was the financial related junk mail. Like people were getting cashed advance check, pre-approved cashed advance checks, pre-approved credit card offers. And they basically said, I really hate those. And a lot of people, they lived in houses that just had mailboxes that weren't locked. And they said, well, I'm just worried that somebody's gonna come steal my mail and use my cash advance check and defrive my account or take my identity. So we learned that. So we put that front and center. And the second time around, people would like hone right in on that. You could see their temple kind of start going, their vein going, because they were like getting angry just thinking about the possibility that somebody would do that. So it was very effective. Other things, I said, hey, before I showed them the prototype, they're like, hey, can you tell me about what you do with your mail today? They're like, let me tell you what I do, Dan. I go to my mailbox, I grab my stack of mail, and I go straight to my paper shredder and I go shred, shred, shred, shred. I'm like, well, how long are you shredding? Like five minutes a day. That's 30 minutes a week. That's 1500 minutes a year. So there's a whole save time benefit that we didn't even have in our promises map that we learned about. So we put right here, spend less time shredding mail, right? So that's an example of how we took everything that we learned and we put it into the new concept. Silly things. We said we're gonna save trees. Remember, people said how many, multiple people, like, well, how many trees are you gonna save? So we put in 43 trees, just so like we didn't get that question, we didn't get that question anymore, right? Yeah, yeah, right, right? You could pick your tree, right? All kinds of stuff. And there was other stuff too about like, you know, I'm not showing here about like UI for how you configure which mail gets through and which mail doesn't. It's like, hey, you gotta block the catalogs because they clog up my mailbox, but my pottery barn catalog, that's gotta get through like all these little things like that, right? And both times, I didn't say this, but both times we asked people at the end, hey, would you pay $10 a month for this service? And it's always a little iffy to ask people if they would pay, if they don't actually have to bust out their wallet and pay, but we saw like a night and day difference. The first time around, nobody was going, nobody had any interest in that. The second time around, what they said is, well, I'd want a 30 day trial of your product, but if it does what it says here, I would gladly pay you $10 a month. So it was like a night and day difference. And then the other thing that gave me evidence that we had really kind of improved product market fit by iterating was after the test was done, I'd be like, great, thank you so much for your time. Here's your check for $100, you know, we're done, right? Almost every single person after we gave them the check and the test was completely done and they had every right to leave was like, so is this product live now? Can I go use this? Like no, no, we're still building it. It's not quite ready yet. So can I give you my email? Can you email me when this thing comes out? Like nobody was doing that the first time around. So I was like yet another kind of indicator that we had achieved more product market fit. So I was pretty excited about that. So to sum up again, lean product process, start out by getting clear on who your target customer is, use importance and satisfaction to understand which of their needs are underserved. Use the Kano model to get clear on your value proposition, how are you gonna be better, different than the competition. Figure out what your MVP feature set should be so you don't overbuild before you course correct the direction that you're going in. Go through the UX design and create a prototype. Again, if you can create a clickable tapable prototype without coding, then great, you'll be able to move faster. Get out of the building and test with customers and basically iterate. Go through that hypothesized design test learn loop with the goal of improving product market fit as you iterate. So as I mentioned, I run a group down in Mountain View. We have monthly speakers. The next one is Tuesday, December 12th. Give Biddle a friend of mine. He's the ex-former VP of product from Netflix. He's gonna give a great talk on product strategy and how to translate into execution. If anybody's interested, you can go there. The other thing is I occasionally, like two or three times a year, I give public workshops that anybody can come to. I'm actually giving one on November 30th in the city. I've created a special 20% off code for Peace School, for product school. If anybody's interested, if you have any questions, you can ask me about that. But basically we go all day with group exercises and really get into it. So with that, I'd be happy to answer any questions that anybody has. Yeah. B2B is too hard to forget about it, man. Yeah. So it's funny, because I get, a lot of times I'll get that question and sometimes it's worded more strongly. People say, oh, this doesn't apply to B2B. This doesn't apply to enterprise. You ask it more like what's different. And when people say, hey, it doesn't apply. I'm like, well, don't you have customers? Don't they have needs? Don't you have competition? So you do. So there's more to it's, high level, more to it's similar than different. What comes up with enterprise is, certain techniques may not be as relevant. Like, I didn't actually mention, did I mention AB testing? I didn't. This is a very qualitative approach. In the book, I do talk about AB testing, but that's more later after you launch. But AB testing may not be as amenable, right? If you've got 10 important B2B clients, you may not be able to do AB testing. Whereas if you're Facebook, it's like, hey, let's take 2% of traffic and AB test and see what happens and get your results in a few hours, right? So AB testing may be harder for you to do. The other thing is, other techniques, you may not be able to, you may not be able to survey people as much, right? I didn't mention using surveys either. So it doesn't impact that. The thing that people tend to complain about is the avail access and availability to customers. Basically when it's B2C, it's pretty easy to get people. If you're targeting like Fortune 500 CMOs and I was like, hey, can I meet with you to show you my prototype? They may be more reluctant, right? So that's usually one of the material differences. And a lot of times you may have blockers and gatekeepers like sales or customer success keeping you from talking to those people. So, but you gotta figure out some way to actually get some of the tactics that are used there is like your customer advisory council or customers opt in to be pestered with questions or what the other people would think, right? They opt in so once a month, once a quarter, you have a meeting, you have a few hours with them and that's when you get feedback on your prototypes and your ideas. The other strategy to go is to go with a private beta and like a pilot, you know, where you're going deep with one or two clients to really work through the MVP and the UI design before you roll it out more broadly. Usually those people, if you guys are Red Cross in the chasm, there's like the difference between like the early adopters and later adopters. You wanna find the subset of your customers that feel that pain so much that they're willing to jump in the foxhole with you and the carrot you have for them is, hey, you can help us co-create this. So if you're co-creating this new product, it's gonna really meet your needs and there's an opportunity there. A lot of times they'll also get grandfathered in on lower pricing or free or something like that. So those are some of the differences that I see. Conversely, the willingness to pay is usually easier to assess because if you're solving being a real business need, they will pay you. Whereas the consumers, it can be kind of harder. Maybe you're monetizing through other means like advertising or something like that, you're not sure. So those are some of the differences that I see. Yeah, that's right. That's right. Yeah. Yeah. Yeah. Yeah, so I would actually recast it. The question was, hey, a lot of user testing I've seen is very task focused, but how do you assess for delight? I would rephrase it as the way I don't have slides here, but in the all day workshop, I talk about the distinction between usability testing and product market fit testing. That's how I would call it, right? And basically like old school usability testing is literally focused on like task completion rates. And it's like, okay, we've got a product to let you book flights. We're gonna like put on our white coat and get someone to come in and say, please try to, excuse me, sir, please try to book a flight from SFO to Dallas on this day. And we just see if they can do it. And we just either, you know, five out of 10 people did. We had a 50% task completion rate. That's like hardcore usability testing. I don't think that's as valuable for what we're talking about. So what I'm talking about is just more of a blend. Yeah, you'll get usability feedback. And I didn't get into time and to share my advice on how to conduct these sessions, but they're much less directed. I'm not telling people what to do, right? Marketing Report.com, right? They're just going there and doing what they would do, right? I mean, like junkmailfreeze.com, you know, I'm not sitting here going, okay, try to sign up, try to do this, try to do this. Like if I do, I'm kind of pushing them through the wickets. It's not a real test, right? What I want it to be is as realistic as if, no, you were just, this was in your browser, a friend of you said, hey, go check out junkmailfreeze.com. So my main hack for that is at the beginning of the interview when I show them, at the point in the interview when I show them the prototype, they're used to kind of being mice in a maze and I'll show it and I just shout out, I'll just be quiet. And it's a little awkward, but then they'll be like, so what do you want me to do? And I have a pat answer, it's, well, do whatever you would do if you're home by yourself and I just shut up again. And then sometimes almost always it'd be like, so should I register? And what do you think I say? Do whatever you would do if you're home by yourself. And then they realize, okay, Dan's not gonna hold my hand and push me through the maze. They realize it and so then that's the number one trick. It's not about giving them a task and seeing if they can do it. It's about getting their reaction to it. You'll naturally, if there are usability issues, they'll naturally emerge because people won't be able to navigate or figure things out, right? But it's a better way of assessing that product market fit. So that's kind of how I, and then again in the book and then the workshop I talk about so those techniques on how to ask, a lot of the thing too is how you ask questions. It's very important how you ask questions. It's really easy to learn how to ask questions the right way. Any other questions? Yeah. So if you get put on a project towards the top of the product, you need pyramids. Yeah. And further down the pyramid, it's not really ruffling the feathers. I see. It's alive and out at the whole time. I see. What are some of the best? That's good, well, that's a good, that's a good quote. Let's just pull up the pyramid so we can get clear about what we're talking about. I know what you mean. So you're saying like, hey, I'm like the UX designer and they've already pre-specified all that other stuff to me or I'm a PM and I'm jumping in. So I'm jumping in at step four or five and some other people in the org have decreed one, two and three and I'm inheriting that and I may or may not believe that's right or I may have a feeling that it's probably not right. How can I influence that? Well, I mean, the number one thing is to test with customers. Then the question becomes kind of a blame game. Oh, they didn't like it. Well, why didn't they like it? Well, maybe it's your UI design that you did or maybe it's your feature set that you didn't do. So it's hard to attribute that. If you think that target customer is wrong, then you can say, hey, guys, why don't we actually talk to two sets of customers? That's an easy one actually, because most people, again, they don't like to say no. It's like, yeah, yeah, you're right. In case we fail with customer group one, maybe we'll hit with customer group two. That sounds like it's gonna increase our odds, right? So that's one thing you can do. You can actually get good data on this from your interviews. So I recommend, like I said, starting out with discovery and then going into the prototype. In the discovery phase, I'll be like, hey, can you tell me how do you get that need met today? And you'll learn. It's a great way to get competitive intelligence. People say, I use this, I use this, I use this. And then you can get into, well, why did you pick that product? What other products did you evaluate? What do you like about that one that's better, right? And then later, when you test your product, you see people making comments like, wow, this isn't any better than what I use today or it's worse because of this, then now you can isolate it to this value prop layer. Does that make sense? Yeah, so those are some techniques. Other questions? Yeah. Well, we have to define quality can mean a lot of different things. And you just mentioned reliability. So what, can you tell me more about? Yeah, right, right, right. So it's hard because prototypes are always reliable, right? They don't go down, the server doesn't go down. So that's one more where I would say, if that was your hypothesis, that our competitor has low reliability and we're gonna have higher reliability, the first thing I would do is I would confirm that the competitor's customers actually are saying they have low reliability. So I'd go interview 20 Uber users and I wouldn't say, tell me about reliability. I'd just see if they brought it up. And if they don't bring it up at all, then your hypothesis is wrong. If they do bring it up, then you go, okay, cool. And then you can try to, then you can like, you don't need the witness, but if they bring it up, then you go, oh, hey, what do you, if you had to estimate it, like, what do you think it is? 50%, 75%, you know, like some rough numbers. And then you can kind of estimate it, right? So there's, you can validate your hypothesis about the problem that you're gonna do a better job on. As far as how you confirm that you're doing a better job, that's tough because until you actually get out there and do it, you don't know. But if you have a reason, a strong reason to believe that you're gonna achieve higher reliability, as long as you just, and people say it's important, right? They won't bring it up if it's not important, but you wanna kind of quantify how important is that reliability. So you're kind of quantifying importance and satisfaction for that reliability. And you're hoping, you're saying it's in the upper left quadrant. So you wanna validate that. If it is, and you know that your reliability is gonna be like this much better, you almost don't even need to test it. If you know it's important and you know they're here and you know that you're gonna be here or you're confident you're gonna be here, you should be in good shape. You know what I mean? So you could explicitly ask people, hey, what if our product was, you know, you just told me Uber's 50% reliable. What if our product was 90% reliable? I'm just gonna say, yeah, that sounds good. That's what they're gonna say, right? So yeah. Yeah, yeah. Yeah. So I mean, that's how I would go about trying to building confidence and evidence. Yeah. A specific question, but so right now the sort of market's moving towards the consumerization of software or enterprise software is really focused on UIUX. So if there's a person that's transitioning from like a non-technical background to products, and they only have one thing they can invest in in terms of like extra time to spend on strengthening other skill sets, would you say a non-technical PM should invest more in design and UIUX stuff or strengthening the engineering background? That's a tough call and there's a little bit, I mean, I'm happy to give them, share my opinion, it's a little bit of it depends. Like if you're gonna go join a machine learning startup then maybe you should learn about machine learning, right? Or if you're joining a real techie startup then maybe you need the technical background. But with those exceptions in general, and I think in general a lot of employers over index on technical skills, and I think the reason they do that, they a lot of them say, hey, CS degree required for a PM. A lot of them, they don't maybe fully understand PM and they're just using it as a hack because they're saying, hey, if you have a CS degree then you're gonna be able to work with engineers well. It's like a lazy shorthand for, you'll probably be able to work with engineers well and it's not necessarily true. I mean, you cannot have a CS degree and be able to work with engineers well. Certainly, yeah, that too, right? You could, right, but yeah. And just to be clear, they just wanna avoid the clueless PM who goes and asks for things that would take years to build. So there's like the extremes, right? There's the extremes, there's the programmer who gets into PM and then just acts like a programmer and working on solutions. And there's the PM who has no technical skills and doesn't even know what they don't know and they ask for silly stuff. But in general, with the exception of places where you truly need a specific skill like machine learning or technical skills, I personally would, I would invest more in not even UI design, UX design, like wire framing skills and also customer research skills, right? We just got done saying, hey, I interviewed these customers, I did these sessions, like who's gonna do that? A lot of times in a company, everyone kind of looks around like who's gonna do it? And sometimes it's like, oh, we have a professional UX researcher, you often don't have that. And so it's gonna be like the designer and the PM look at each other going, who wants to do, who hates doing this less kind of thing, right? As a PM, I would embrace that and jump in and do it. I wanna be as close to customers as I can, right? So in those UX research skills, like interviewing skills don't take a lot of time. It's not a huge investment to learn that. Learning UX skills, it's like learning a tool like Balsamic, if you're not, you need to have a go-to wire framing tool that you're fairly proficient in and comfortable with. So when somebody does come at you with a product idea, you can kind of relatively quickly bang out some basic wire frame and say, what do you think, you know? That's what you wanna get to the point of doing it. The other thing as I talk about in the book and my other talk is just like there's often the lack of UX research, there's often a design gap, especially in startups. They'll be like, okay, we got a bunch of engineers, then we have a PM, that's what we got. So that's where the wire framing skills come in handy. No one's expecting you to be the world's best wireframer, but when you're in that kind of situation, even in established companies, you think, oh, you worked at Facebook, you must have designers falling out of your pockets, right? No, on certain teams at certain points in time, they're understaffed or the designers are on higher-party things or they're unfilled positions. Even in big companies, there's a design gap at time, so it can be good to know that. And also, just even if you do have a designer, you'll be able to partner with them much more effectively, so hope that helps. And the other thing I would say is some companies are very analytical in their DNA, like a Zynga or something like that. So for those companies, maybe it's best to invest in analytics, right? So there's certain exceptions, like machine learning, data science, technical for technical products, you know, analytics for highly analytical cultures, but all other ones I would invest in UX design and wireframing and UX research.